Utility-Based Multiagent Coalition Formation with Incomplete Information and Time Constraints *

Transcription

1 Utlty-Baed Multagent Coalton Formaton wth Incomplete Informaton and Tme Contrant * Leen-Kat Soh Computer Scence and Engneerng Department Unverty of Nebraa Lncoln, Nebraa loh@ce.unl.edu Abtract - In th paper we propoe a coalton formaton model for a cooperatve multagent ytem n whch an agent form ub-optmal coalton n vew of ncomplete nformaton about t noy, dynamc, and uncertan world, and t need to repond to event wthn tme contrant. Our model ha two tage: (1) when an agent detect an event n the world, t frt comple a lt of coalton canddate that t thn would be ueful (coalton ntalzaton), and (2) then negotate wth the canddate (coalton fnalzaton). A negotaton an exchange of nformaton and nowledge for contrant atfacton untl both parte agree on a deal or one opt out. Each ucceful negotaton add a new member to the agent fnal coalton. Th paper tal about the tep we have degned to enhance the fnalzaton tage. Keyword: Multagent ytem, negotaton, coalton formaton. 1 Introducton In th paper we decrbe a coalton formaton model for agent wth ncomplete nformaton and tme contrant wthn a dynamc and uncertan world. A coalton a group of agent that collaborate to perform a coordnated et of ta that may be a repone to an event that ha occurred n the envronment. A dynamc coalton one that formed a a repone to an event and dolved when the event no longer ext or when the repone completed. A coalton neceary when an agent cannot repond to an event all by telf due to lac of nformaton, nowledge, or functonal capablte. Ideally, the agent would prefer to form an optmal coalton to maxmze the yeld of the ytem a a whole. However, uch optmal ratonalzaton requre the agent to have complete nformaton about t world and t neghborng agent, and alo about the uncertanty aocated wth all factor related to the multagent nfratructure. When that nformaton not readly avalable and the collecton of that nformaton too cotly, an agent cannot afford uch optmalty. In the followng, we elaborate on ome of the problem charactertc. Cota Tatoul Department of Electrcal Engneerng and Computer Scence Unverty of Kana Lawrence, KS tatoul@ttc.u.edu (1) Our model apple to an envronment where each agent ha ncomplete nformaton about t world. Incomplete nformaton may be due to pollng and updatng cot, contraned reource, and decentralzed nformaton bae. (2) An optmal ratonalzaton for coalton formaton may not be poble due to (a) noe and uncertanty n the envronment, and (b) tme contrant. For example, the communcaton channel among the agent may be congeted or faulty, meage may be noy or lot, perceved event may be qualfed naccurately, and o on. (3) We aume all agent are peer there no herarchy among the agent. Each agent able to ene t envronment, reve t own percepton, and form t own coalton. Th allow the agent to be reactve to envronmental change, wthout havng the drectve paed from a hgher-up agent whle encouragng dverty n nformaton tored at each agent. (4) We propoe ung negotaton to refne a coalton. We ee negotaton a an exchange of neceary nformaton pertnent to ndvdual contrant, percepton, and commtment. Th exchange of nformaton performed only when the coaltonntatng agent approache potental coalton partner to requet for help. Thu, the ntal coalton can be le than optmal and be computed hatly a the negotaton wll refne or fnalze the electon. (5) Our model expect coalton member to refue to on n a coalton, epecally n a reource-contraned envronment and alo plan for faled communcaton due to congeton, noe, or meage lo. Thu, the ntal coalton may not urvve after negotaton a the worng coalton fnalzed. Brefly, our propoed model wor a follow. When an event detected n a multagent ytem, one of the agent ntate the coalton formaton proce n hope of organzng a group of cooperatve agent to perform ta * /03/$ IEEE.

2 n repone to the event. Th ntatng agent (alo nown a the computng agent [3])) houlder the reponblty of degnng the bet coalton gven the tuated nformaton to ncreae the chance of formng a worng and ueful coalton at the end of the proce. The model cont of two tage. Frt, durng the coalton ntalzaton, the ntatng agent extract a raned lt of ueful agent. Then, the ntatng agent approache the potental coalton partner and requet for negotaton durng a coalton fnalzaton tep. Our negotaton baed on a cae-baed reflectve argumentatve model [7]. Th paper focue on the dfferent tratege durng the ntalzaton tage and the tep we have degned to enhance the fnalzaton tage. Before further dcuon, here we outlne ome aumpton about our agent. In general, our model aume that agent have the followng charactertc: autonomou, ratonal, communcatve, honet, and cooperatve. There are n general three reaon why agent cooperate [6]. Frt, an agent cannot perform a pecfc ta by telf. Second, an agent can perform a pecfc ta, but other agent are more effcent n performng the ta. Thrd, an agent can perform a pecfc ta, but worng on t collaboratvely wll ncreae the beneft from the ta (or reduce the cot). We alo aume that each agent ext n a neghborhood where t now ome bac properte of t neghbor (uch a functonal capablte) and can communcate to them drectly. Each agent ha a neghborhood and can communcate drectly wth all t neghbor, and each neghbor can communcate wth the agent drectly a well. It from th neghborhood that an agent form a coalton. Reader are referred to [10] for a detaled dcuon on varou agent charactertc. 2 Related Wor A defnton from the ratonal coalton theory outlned n [1] tate that a coalton game wth tranferable utlty n normal charactertc form. The value of a coalton the total utlty that the member of the coalton can acheve by coordnatng and actng together. However, n our problem doman, the agent do not have the nformaton they need to compute accurately the value of a coalton but we have propoed general heurtc for etmatng uch value. In our model, the value of a coalton not ndependent of nonmember acton a n ome tude n charactertc functon game [9]. Sandholm and Leer [3] ntroduce a bounded ratonalty n whch agent are guded by performance profle and computaton cot n ther coalton formaton proce. The author bounded ratonalty model requre each agent to pay for computatonal reource that t ue for delberaton. Our model alo affected by the current tatu of an agent, uch a the avalablty of negotaton thread, and the envronment, uch a the avalablty of the communcaton channel. Zlotn and Roenchen [11] decrbe a coalton drven by ta-orented utlte. In a ta-orented doman (TOD), a coalton can coordnate by redtrbutng ther ta among themelve. In our model, we addre the problem from the vewpont of an agent. That, an agent can be of two or more coalton multaneouly a each agent autonomou and capable of reactng to eparate event. Shehory et al. [6] relax ome of the retrctve aumpton of theoretcal coalton formaton algorthm for a real-world ytem. Ther model aume that the agent are group-ratonal and the agent populaton doe not change durng the coalton formaton. There are everal dfference between th model and our. Frt, the author model aume that all agent now about all of the ta and the other agent. In our model, an ntatng agent now only the agent n t neghborhood and now about partally the updated tatu of a electve ubet of the neghbor after negotaton. Second, the detal of ntra-coaltonal actvty are not neceary for agent outde of the coalton n the author model. On the contrary, n our model, an agent can and doe belong to multple coalton concurrently. Tohmé and Sandholm [9] tude coalton formaton among elf-ntereted agent that cannot mae depayment reward each other wth payment for agreement to on ome coalton, mang the evaluaton of a coalton olely on t utlty, and propoe a model that guarantee convergence or tablty n t coalton oluton. However, n our model, there are external factor uch a the dynamc event and noy communcaton channel that may thwart the ucceful completon of a negotaton, renderng a negotaton outcome unpredctable. Sen and Dutta [4] propoe an order-baed genetc algorthm (OBGA) a a tochatc earch proce to dentfy the optmal coalton tructure. A gnfcant dfference between the author wor and our model the cope of coalton formaton. The author algorthm for earchng for an optmal coalton tructure, whch cont of all the agent n the envronment grouped nto one or more coalton. Fnally, note that our coalton formaton actvte dffer from that preented n [2] whch defne coalton formaton a three nteractng actvte: (1) Coalton tructure generaton where agent wthn each coalton coordnate ther actvte but do not coordnate between coalton. Th mean parttonng the et of agent nto exhautve and dont coalton

3 and the partton called a coalton tructure. In our model, an agent form a coalton from t neghborhood. Some neghbor may be part of the coalton, may be part of other coalton, or may be mply dle. Coalton n our model may alo overlap. (2) Solvng the combnatoral optmzaton problem of each coalton whoe obectve to maxmze the utlty of the coalton. Th mean poolng the ta and reource of the agent n the coalton, and olvng ther ont problem. In our model, the ntatng agent houlder th approxmaton ung mperfect nformaton to ncreae the chance for a ucceful coalton. (3) Dvdng the value of the generated oluton among agent. There no uch explct value dtrbuton n our model. Our agent are altrutc and drected to help f poble to acheve global goal, and thu do not requre addtonal motvaton uch a reward or value. However, our agent do want to manage ther own reource effcently and th motvate negotaton and the ta allocaton among agent. 3 Methodology Our methodology ha two tage: (1) when an agent detect an event n the world, t frt comple a lt of coalton canddate that t thn would be ueful (coalton ntalzaton), and (2) then negotate wth the canddate (coalton fnalzaton). A negotaton an exchange of nformaton and nowledge for contrant atfacton untl both parte agree on a deal or one opt out. Each ucceful negotaton add a new member to the agent fnal coalton. 3.1 Coalton Intalzaton We wll brefly dcu th ntalzaton tage here. Reader are referred to [8] for detal. The frt tage of the utlty-baed multagent coalton formaton model the determnaton of the et of the ntal coalton Λ a, e for agent and event canddate, denoted a ( ) e n. We denote a canddate a α. In our model, the ntatng agent canddate, n ( a, e ) n a ( a, e ) frt generate the ntal coalton Λ, to deal wth an event. Λ repreent the neghbor that the agent thn can be of help to repond to a e. To fnd out whether thee canddate are wllng to help, the ntatng agent need to negotate. Due to reource contrant, our degn frt ran the canddate on ther potental utlty value to the coalton o that the ntatng agent can negotate wth the agent wth the hghet utlty value frt. For a canddate α Λ a, e, we bae t e n ( ) potental utlty, PU α,, on three et of attrbute: (1) a the pat relatonhp between the ntatng agent and the canddate, rel pat, a ( α, t), where t the pont n tme when the et of attrbute-value par n the relatonhp collected, (2) the current relatonhp between the ntatng agent and the canddate, relnow, a ( α, t), and (3) the ablty of the canddate n handlng the event, ablty α, e t. All thee ub-utlty meaure map ( ) a, nto R : 0K1 and each aymmetrc uch that rel α t rel a, t, where t denote tme. ( ) ( ) pat, a, pat, α Fnally, the potental utlty, PU α, a, t, of a canddate α a weghted um of relpat, a ( α ), relnow a (, t) and ablty (, e t) PU α, a α : a,, α, = W Λ n ( a, e ) (1) α t rel α, t ablty α, e, t [ rel ( ) ( ) ( )] pat, a, now, a where ( ) [ w w w ] T W Λ n a, e = pat, a, e now, a, e ablty, a, e a and w + w w = 1. pat, a,,, + e now a e 3.2 Coalton Fnalzaton ablty, a, e In our model, we ue a real-tme cae-baed logcal negotaton protocol to dctate the rule of encounter or the negotaton tratege between two agent. Intereted reader are referred to [7] for a detaled preentaton of the logcal protocol. In th paper, we focu on the tep that we have taen to enhance the fnalzaton proce : (1) awarene, and (2) relaxaton and termnaton. Snce our negotaton protocol mult-tep, t facltate nteracton between negotaton thread. An ntatng agent can nvoe a hot of concurrent negotaton, bounded by Λ can _ approach ( a, e ). Whle the negotaton thread are actvely engaged n ther repectve negotaton, the ntatng agent contnue to montor t world, examne t ta, communcate wth other agent, and watch the tatu of t negotaton thread. Snce each of thee thread now how to negotate on t own, all t need from tme to tme for the parent agent to update the agent current belef and ntenton that mght nterrupt the negotaton or change the negotaton ue. The parent agent thu able to nfue a hgh-level of awarene n the negotaton thread, relax the negotaton ue (le demandng or more concedng, for example), and abort negotaton wth dmnhng return. In the followng ubecton, our dcuon focu on the ndrect nter-thread actvte that the agent provde for t negotaton thread.

4 Awarene Snce the envronment dynamc, an ongong negotaton may become uele. For example, f the negotaton part of a repone to an event e and e become fale, then the negotaton ha to be termnated. Snce a negotaton thread handle the negotaton emautonomouly, t mut be aware of uch a tuaton, and the parent agent ha to provde uch awarene. Th coalton awarene ha everal beneft. Frt, t allow an agent to free up t negotaton thread, communcaton channel, and communcaton bandwdth for other negotaton ta. Second, t allow an agent to mmedately abandon falng coalton, re-ae t envronment, and tart another coalton formaton. Thrd, by termnatng uele negotaton, an agent able to bae t reaonng on updated, more correct tatu profle. A negotaton thread conduct t negotaton followng a logcal real-tme protocol that pell out what t hould do n each negotaton tep, and a negotaton trategy that dctate how the thread hould negotate how much tme t ha, how concedng t hould be, what nd of argument t ha, whch argument t hould end frt, and o on [7]. It alo need to now the context of the negotaton the requet, the amount of reource to gve up, the counterpart agent, and o on. When actvated, a negotaton thread download the negotaton context and trategy from the parent agent. Then, f the thread doe not hear from the parent agent, t now how to negotate on t own and report bac to the parent agent only when the negotaton completed. The parent agent, on the other hand, carre out t normal ta uch a montorng the world, actuatng t enor, and o on. It hold a hared data obect wth each negotaton thread. When the event change or the current tatu of the coalton change, the agent evaluate the current tatu of each negotaton thread and mae a decon a to whether to relax, to termnate, or refne the negotaton. Th decon together wth t pertnent nformaton tored at that data obect. We call th hared data obect the awarene ln, or AL for the β negotaton thread of agent, where AL ha X z negotaton thread, β 1, β 2, L, β. Both the agent and the negotaton thread can tore and acce data on th awarene ln. Wth th degn, the parent agent houlder the ta of feedng t negotaton thread addtonal ntructon. There are everal reaon why we adopt th awarene ln degn n our model. Frt, the envronment dynamc and real-tme crtcal. It doe not mae ene for each negotaton thread to etup t own enor, montor, and even decon maer to determne the current tatu of the agent and change t negotaton behavor accordngly, nce t would have to eve through a { X } a, β a z, β z unrelated nformaton and data and that would be tme conumng. Second, t natural for the agent to dpere the nformaton to all t negotaton thread. A negotaton thread doe not now the tatu of a coalton (e.g., whether the coalton falng or ucceedng); only the agent now that. Wth that nowledge, an agent can decde whether to cale bac on ome of t negotaton, or mae other modfcaton. Th way, the chan of command drect and le confung, and certanly le computatonally ntenve. Thrd, wth each negotaton thread havng t own dedcated awarene ln, the nformaton or data paed through the parent agent and that partcular thread doe not nterfere wth the other thread. Th way, each negotaton thread can concentrate on the ntructon pecfcally drected to t from the parent agent. In our model, the parent agent chec the negotaton tatu of t negotaton thread wthn a framewor of ta. It chec t meage, t enor for event, ta, and the negotaton, and then repeat. Th lfecycle vare n t duraton, a the envronment dynamc and uncertan. For the negotaton thread, we propoe a gradated cheme baed on the percentage of tme elaped. For example, a thread chec and update t tatu (1) le frequently n the begnnng, (2) more frequently toward the end, (3) le frequently when t progreng accordng to plan, and (4) more frequently when t falng. Th becaue we aume that the event tatu tll relatvely contant n the begnnng of the negotaton and only change after a certan tme perod ha paed. We alo aume that when a negotaton progreng well and ucceedng, that negotaton thread hould carry on and complete the negotaton unle ome gnfcant event occur and call t off. In th manner, the agent doe not loe the utlty of uch a negotaton and learn to be effcent. We alo aume that when a negotaton not dong well, after reportng t to the parent agent, the negotaton thread can expect further ntructon from the parent agent, hence the ncreae n t acce of the awarene ln. Relaxaton and Termnaton Each agent reponble for the coordnaton among t negotaton thread a the negotaton thread do not tal to each other drectly. The agent montor the tatu of the negotaton and mae decon. Two of the decon t can mae are relaxaton and termnaton. From the ntatng agent tandpont, th relaxaton reult n a maller demand; from the repondng agent tandpont, th relaxaton reult n a more yeldng tance. Snce th paper focu n coalton formaton, we wll dcu relaxaton and termnaton from the vewpont of an ntatng agent.

5 Suppoe n a 1-to-1 ta allocaton problem, we have approached ( a e ) > F Λ,. That, the number of canddate that the agent approache greater than the number of ta requred to repond to the event e. At tme t, a wth a partal-agnment a actvate all t negotaton thread, each = f. At tme t +, α, ome change have been detected uch that F now F. If F F, then the agent need to (1) mmedately termnate all negotaton thread wth where = α, f, and (2) label th change a a new event F and proceed from there accordngly. If F F, then the agent need to relax the negotaton. In a many-to-1 ta allocaton problem, uppoe that the negotaton thread n queton ha = α, { f 1, f 2 }. Then, the agent can compute ( α, f t) and ( α, f t) benefta 1, benefta 2, (aumng dont ta). It then can decde to drop or from t orgnal demand or eep both. The f 1 f 2 algorthm become: Algorthm Many-to-1 Relaxaton and Termnaton: (1) for all ongong negotaton wth f f beneft α, f = f where and f F do: (1.1) compute ( f t) a,, α for all f f and f F, (1.2) chec the current tatu of the negotaton and denote th a progreβ x ( α, f, t) for negotaton thread β x, (1.3) compute the expected utlty of contnung wth th negotaton for all f f and f F : EU PU β x ( α, f, t) ( α, f, t) beneft ( α, f, ) α, f, a + progre t β x a = 2 (2) and (1.4) f EU β ( α f, t) x, greater than what the agent can afford to pend n reource, then the agent retan n the negotaton; otherwe, t drop f from the negotaton. The expected utlty bacally the potental utlty of the coalton member to the orgnal event repone plu the utlty of contnung wth the negotaton. The latter utlty ay that f the negotaton progreng well and the eventual outcome wll beneft the envronment, then the agent hould not dcard the ongong effort. The relaxaton and termnaton behavor both ratonal and altrutc. An agent hould not be conductng negotaton that ue t reource when thoe negotaton f have become uele. Nether hould an agent mpoe or tranfer that cot to t repondng agent by ntng on uele negotaton. Wthout th relaxaton and termnaton capablty, the ntatng agent would have to be more careful n t ntalzaton nce t ha le room for error. That would mean for the ntatng agent to collect more nformaton n t ratonalzaton whch would n turn decreae the autonomy and robutne of the multagent ytem. So, the couplng of the ntalzaton and relaxaton/termnaton very mportant n our utltybaed, dynamc coalton formaton model. Fnally, at the end of all negotaton, we have the Λ a, e and fnal coalton fnal ( ) ( a, e ) Λ ( a, e ) Λ ( a, e ) Λ. fnal approached 4 Implementaton and Reult The drvng applcaton for our ytem multenor target tracng, a dtrbuted reource allocaton and a contrant atfacton problem [7]. The obectve to trac a many target a poble and a accurately a poble ung a networ of enor. Each enor ha a et of conumable reource, uch a beam-econd (the amount of tme a enor actve), battery power, and communcaton channel, whch each enor dere to utlze effcently. Each enor at a fxed phycal locaton and, a a target pae through t coverage area, t ha to collaborate wth neghborng enor to trangulate ther meaurement to obtan an accurate etmate of the poton and velocty of the target. Here we report on ome prelmnary experment reult for the behavor analy of our multagent ytem, pecfcally for multenor target tracng. We conder here an exemplary run that we ued to adut our ytem parameter. In th run, the total number of attempt to form a coalton wa 150. The total number of coalton uccefully formed (after coalton fnalzaton) wa 30, or 20%. The total number of coalton confrmed by all coalton member wa 26, or 86.7% of all uccefully formed coalton. Fnally, the total number of coalton executed on tme wa 18, or 61.5% out of all uccefully confrmed coalton. Frt, the percentage of uccefully formed coalton wa only 20.0%. Out of the 120 faled attempt, 86 (71.7%) of them were caued by one of the coalton member outrght refung to negotate, 17 (14.2%) were caued by the communcaton channel beng ammed, and 17 (14.2%) were caued by buy negotaton thread. When an ntatng agent ntate a negotaton requet to a canddate and that canddate mmedately refue to entertan the negotaton, t can be due to (1) the repondng agent doe not have dle negotaton thread, or (2) the repondng agent cannot proect the requeted n

6 ta nto t ob queue. Thu, we expect th falure rate to decreae once we ncreae the number of negotaton thread allocated per agent. When an agent fal to end a meage to another agent, or fal to receve an expected meage, we label th a a communcaton channelammed problem. When an ntatng agent fal to approach at leat two canddate, t mmedately abort the other negotaton proce that t ha nvoed for the ame coalton. Th caue the coalton to fal. Second, the probablty of a uccefully formed coalton gettng confrmed completely wa 86.7%. For each coalton uccefully formed, three confrmaton were requred. Out of 30 coalton, 4 coalton were confrmed only by two of the member. The caue were (1) the acnowledgment meage ent out by the ntatng agent wa never receved by the repondng agent expectng a confrmaton, and (2) the agreed ta had been removed from the ob queue before the confrmaton arrved. The frt caue happened nce communcaton channel could be ammed. The econd caue happened becaue of a contenton for a lot n the ob queue by two ta. For example, uppoe agent A receve a requet from agent B to trac a target tartng at 8:00 a.m. Agent A repond to the requet and tart a negotaton. Then later on, agent A receve a requet from agent C to trac a target tartng alo at 8:00 a.m., but ung a dfferent enng ector (each enor ha three). Agent A chec t ob queue and ee that t free at that tme and thu agree to negotate. Note that a ta nerted nto the ob queue only after the agent agree to perform t. Now, uppoe that both negotaton are ucceful. The negotaton between A and B end frt and then that between A and C. When the frt negotaton end, agent A add the ta requeted by B to the ob queue. Immedately after, when the econd negotaton alo end uccefully, agent A add the econd ta, requeted by C to the ob queue, and th caue the econd ta to replace the frt ta. Th a problem wth over-commtment. Our prelmnary reult were promng a the agent are able to form coalton ung the outlned model and methodology. However, the reult howed that there are tmng and ta chedulng ue that are currently beng addreed. 5 Concluon In th paper, we have ntroduced everal tep that we have degned to enhance a utlty-baed multagent coalton formaton model. The model ue awarene ln and flexble relaxaton and termnaton cheme to deal wth the dynamm n a coalton formaton proce due to ncomplete nformaton and tme contrant. The prelmnary reult were nghtful and promng. Acnowledgement The author would le to than Xn L for her programmng and runnng the experment. The wor decrbed n th paper wa partally ponored by the Defene Advanced Reearch Proect Agency (DARPA) and the Ar Force Reearch Laboratory (AFRL). Reference [1] J. P. Kahan and A Rapoport, Theore of Coalton Formaton, Lawrence Erlbaum, [2] T. W. Sandholm, K. Laron, M. Anderon, O. Shehory, and F. Tohmé, Coalton tructure generaton wth wort cae guarantee, AI, vol. 111, no. 1-2, pp , [3] T. W. Sandholm and V. R. Leer, Coalton formaton amongt bounded ratonal agent, n Proc. IJCAI 1995, Montreal, Canada, pp , [4] S. Sen and P. S. Dutta, Searchng for optmal coalton tructure, n Proc. ICMAS 2000, July 7-12, Boton, MA, pp , [5] O. Shehory and S. Krau, Formaton of overlappng coalton for precedence-ordered ta-executon among autonomou agent, n Proc. ICMAS 96, Kyoto, Japan, pp , [6] O. M. Shehory, K. Sycara, and S. Jha, Mult-agent coordnaton through coalton formaton, n Proc. ATAL 97, Provdence, RI, [7] L.-K. Soh, and C. Tatoul, Reflectve negotatng agent for real-tme multenor target tracng, n Proc. IJCAI 01, Augut 4-10, Seattle, WA [8] L.-K. Soh, and C. Tatoul, Allocaton Algorthm n Dynamc Negotaton-Baed Coalton Formaton, n Proc. IJCAAMAS Worhop on Teamwor and Coalton Formaton 02, July 15-16, Bologna, Italy [9] F. Tohmé and T. Sandholm, Coalton formaton procee wth belef revon among bounded-ratonal elf-ntereted agent, J. Logc & Computaton, vol. 9, no. 6, pp , [10] M. Wooldrdge and N. Jennng, Intellgent agent: theory and practce, Knowledge Engr. Rev., vol 10, no. 2, pp , [11] G. Zlotn and J. S. Roenchen, Coalton, cryptography and tablty: mechanm for coalton formaton n ta orented doman, n Proc. AAAI, July, Seattle, WA, pp , 1994.