Cross-Domain Multi-Event Tracking via CO-PMHT

Transcription

1 31 Coss-Domain Muti-Event Tacking via CO-PMHT TIANZHU ZHANG and CHANGSHENG XU, Chinese Academy of Sciences and China-Singapoe Institute of Digita Media With the massive gowth of events on the Intenet, efficient oganization and monitoing of events becomes a pactica chaenge. To dea with this pobem, we popose a nove CO-PMHT (CO-Pobabiistic Muti-Hypothesis Tacking) agoithm fo cossdomain muti-event tacking to obtain thei infomative summay detais and evoutionay tends ove time. We coect a agescae dataset by seaching keywods on two domains (Goooge News and Fick) and downoading both images and textua content fo an event. Given the input data, ou agoithm can tack mutipe events in the two domains coaboativey and boost the tacking pefomance. Specificay, the bidge between two domains is a semantic posteio pobabiity, that avoids the domain gap. Afte tacking, we can visuaize the whoe evoutionay pocess of the event ove time and mine the semantic topics of each event fo deep undestanding and event pediction. The extensive expeimenta evauations on the coected dataset we demonstate the effectiveness of the poposed agoithm fo coss-domain muti-event tacking. Categoies and Subject Desciptos: H.4.3 [Infomation Systems Appications]: Communications Appications Infomation bowses; I.4.8 [Image Pocessing and Compute Vision]: Scene Anaysis Tacking; I.5.4 [Patten Recognition]: Appications Signa pocessing Genea Tems: Agoithms, Expeimentation, Pefomance Additiona Key Wods and Phases: Coss-domain, muti-modaity, muti-event tacking, PMHT, CO-PMHT ACM Refeence Fomat: Tianzhu Zhang and Changsheng Xu Coss-domain muti-event tacking via CO-PMHT. ACM Tans. Mutimedia Comput. Commun. App. 10, 4, Atice 31 (June 2014), 19 pages. DOI: 1. INTRODUCTION With the exposion of Intenet bandwidth, moe and moe socia media sites, such as Fick, YouTube, Facebook, and Googe News, have spung up. These sites ae a popua distibution outet fo uses to shae thei pesona pefeences (hobbies, tastes, contact detais, comments to diffeent socia events, etc.) and inteests and to successfuy faciitate the infomation ceation, shaing, and diffusion among the Web uses. As a esut, a popua event that is happening aound us and aound the wod can spead vey fast, and thee ae substantia amounts of events with muti-modaity (e.g., images, videos, This wok is suppoted in pat by the Nationa Pogam on Key Basic Reseach Poject (973 Pogam, Poject No. 2012CB316304) and Nationa Natua Science Foundation of China ( , ), and aso by the Singapoe Nationa Reseach Foundation unde Intenationa Reseach Cente at Singapoe Funding Initiative and administeed by the IDM Pogamme Office. Authos addesses: T. Zhang and C. Xu (coesponding autho), Nationa Lab of Patten Recognition, Institute of Automation, Chinese Academy of Sciences, Beijing , P. R. China and China-Singapoe Institute of Digita Media, Singapoe , Singapoe; emai: csxu@np.ia.ac.cn. Pemission to make digita o had copies of a o pat of this wok fo pesona o cassoom use is ganted without fee povided that copies ae not made o distibuted fo pofit o commecia advantage and that copies bea this notice and the fu citation on the fist page. Copyights fo components of this wok owned by othes than ACM must be honoed. Abstacting with cedit is pemitted. To copy othewise, o epubish, to post on seves o to edistibute to ists, equies pio specific pemission and/o a fee. Request pemissions fom pemissions@acm.og. c 2014 ACM /2014/06-ART31 $15.00 DOI: ACM Tansactions on Mutimedia Computing, Communications and Appications, Vo. 10, No. 4, Atice 31, Pubication date: June 2014.

2 31:2 T. Zhang and C. Xu Fig. 1. The eft panne incudes the Web pages etuned by queying Aab Sping in a coss-domain (Googe News and Fick). Though the event tacking method, we coud mine the whoe evoutionay pocess of Aab Sping denoted with a ed tajectoy as in the ight pane. Hee, we show the ocation and its happening time on the map fo bette undestanding. and textua content) on the Intenet. Theefoe, it is impotant to automaticay identify and tack the inteesting socia events fom massive socia media data. Muti-event tacking is hepfu to bowse, seach, and monito socia events and undestand thei evoution by uses o govenments. Uses may be inteested in mutipe events and thei evoution. Fo instance, they might want to know the evoution of the wa in Afghanistan, the deveopment of the pesidentia eections in the U.S., o just be infomed if anything new takes pace in Asia o in the technoogica fied. Thus, fo bette use undestanding, it is impotant to obtain the whoe pocesses of mutipe events by muti-event tacking. Fo govenment, it is aso usefu to know the evoution of events fo pubic secuity. Let s take a concete exampe. Consideing the govenment wants to know the whoe pocess of the event The Aab Sping fom beginning to end, infomation etieva techniques ae quite effective fo coecting infomation given we-defined queies. When seaching fo the ecent popua event The Aab Sping on Googe News o Fick, one coud get some infomation as shown in the eft pane of Figue 1. Howeve, a the esuts etuned on the fist page ae showing what has happened ecenty. Moeove, the eevant infomation is too scatteed and one coudn t captue the whoe event evoution. The pobem becomes toughe when we need to foow the gadua evoution of events though time. To dea with these issues, event tacking [Aan 2002; Yang et a. 1999] is one of the best ways. Given an event initiaized with one stoy, we need to ecognize which subsequent stoies descibe the same evoving and changing event. As a esut, fo the event The Aab Sping we can know its evoution in diffeent counties ove time as shown in the ight pane of Figue 1, denoted with a ed tajectoy. Theefoe, the event tacking task coud povide infomative summay detais on what has happened in the ea wod and coud yied impotant knowedge on the evoutionay tends of these events ove time. Tacking the evoution of mutipe events ove socia media sites is a chaenging pobem, as socia media data ae inheenty heteogeneous, noisy, and ambiguous. Thee ae a wide vaiety of ea-wod events on poitics, economics, heath, spots, education, entetainment, sciences, etc., anging fom popua, widey known events (e.g., the 2012 United States pesidentia eection) to smae events (e.g., a oca confeence, o an annua convention). What s wose, since most of the infomation in these socia media sites is geneated by the uses, they contain much noise. Fo exampe, on Fick, fo the same event, uses may have diffeent comments and upoad diffeent images. Sometimes, the comments and the images ae totay ineevant to the event. In addition, some events ae vey simia (hee, event simiaity is defined as its featue simiaity), such as that Occupy Wa Steet and ACM Tansactions on Mutimedia Computing, Communications and Appications, Vo. 10, No. 4, Atice 31, Pubication date: June 2014.

3 Coss-Domain Muti-Event Tacking via CO-PMHT 31:3 Fig. 2. The fowchat of ou coss-domain muti-event tacking via the poposed CO-PMHT. The input is the coss-domain muti-modaity data coected fom Googe News and Fick incuding images and documents. Based on the input data, ou agoithm can tack mutipe events in a coss-domain coaboativey and boost the tacking pefomance. Afte tacking, fo each event, it can be visuaized with documents and images ove time. Meanwhie, we can mine thei semantic topics. Geek potests that have simia topics (peope, against, govenment, and so on). Theefoe, this infomation has bad quaity and might often be miseading o ambiguous, making the event tacking in the socia media patfom a vey chaenging pobem. Event tacking is nomay studied with textua featues [Chieu and Lee 2004; Kuma et a. 2004; Lin and Liang 2008]. In addition to textua infomation, events aso have ich visua infomation [Wu et a. 2008]. An event in diffeent sites may have diffeent textua infomation due to diffeent comments by uses. Howeve, it may have vey simia visua infomation, such as images o videos, that ae usefu fo bidging evoving event stoies acoss time and sites. Fo exampe, the event 2012 United States pesidentia eection, with the pictue of Obama has stoies that ae highy eated. Theefoe, muti-modaity fusion is usefu. In addition, diffeent patfoms can compensate and enhance each othe. Fo exampe, most of events on Googe News ae officia, but have a ot of comments by Web uses on Fick. Theefoe, items of infomation in diffeent domains can hep each othe, especiay when the stengths of one domain compement the weaknesses of the othe. The citica chaenge is how to find the most effective way to bidge the domain gap. Inspied by the pevious wok [Aan 2002; Chieu and Lee 2004; Lin and Liang 2008], we popose a nove CO-PMHT (CO-Pobabiistic Muti-Hypothesis Tacking) agoithm fo coss-domain mutievent tacking. The detais of ou famewok ae shown in Figue 2. The coss-domain dataset is buit by seaching keywods on two domains (Goooge News and Fick) and downoading both images and textua content. Inputting the data to ou agoithm, it can tack mutipe events in the two domains coaboativey and boost the tacking pefomance. Specificay, the inkage between two domains has semantic infomation and effectivey bidges the domain gap. Afte tacking in socia media, based on the esuts, we can visuaize the whoe evoutionay pocess of an event ove time and discove the semantic topics of events as shown in Figue 2. Compaed with existing methods, the contibutions of this wok ae theefod. (1) To the best of ou knowedge, we ae the fist to tack mutipe events with muti-modaity infomation acoss diffeent domains. Specificay, the bidge is a semantic posteio pobabiity that avoids the domain gap. ACM Tansactions on Mutimedia Computing, Communications and Appications, Vo. 10, No. 4, Atice 31, Pubication date: June 2014.

4 31:4 T. Zhang and C. Xu (2) We popose a nove CO-PMHT method to tack mutipe events in diffeent domains coaboativey, thus boosting the tacking pefomance fo each othe. (3) We coect a dataset fo eseach on coss-domain muti-event tacking with muti-modaity infomation and wi eease it fo academic use. The est of the atice is oganized as foows. In Section 2, we summaize wok most eated to ous. The poposed tacking appoach and optimization methodoogy ae pesented in Sections 3 and 4, espectivey. In Section 5, we epot and anayze extensive expeimenta esuts. Finay, we concude the atice with futue wok in Section RELATED WORK We eview eated wok in two aeas: event tacking in socia media and mutitaget tacking in compute vision. Event Tacking. The event tacking inks togethe evoving and histoica stoies. It is a chaenging pobem because socia media data ae inheenty heteogeneous, noisy, and ambiguous. The topic detection and tacking (TDT) task [Aan 2002] was studied in a notabe coective effot to discove and oganize news events in a continuous steam. With the exposive gowth of Web data, event tacking pays a vita oe in TDT fo descibing the time-evoving natue of topics. Most methods ae based on textua infomation, visua infomation, o muti-modaity infomation [Makkonen et a. 2004; Xie et a. 2004; Kende and Naphade 2005; Zhai and Shah 2005; Duyguu et a. 2005; Wu et a. 2008; Zhang et a. 2012b, 2013c]. Makkonen et a. [2004] extacted meaningfu semantic featues such as names, time efeences, and ocations and eaned a simiaity metic that combines these metics into a singe custeing patition. Xie et a. [2004] appied hieachica HMM modes ove the ow-eve audio-visua featues to discove spatio-tempoa pattens, atent semantic anaysis to find text custes, and then fused these mutimoda tokens to discove potentia stoy topics. In Kende and Naphade [2005], authos studied the coeation between manuay annotated visua concepts (e.g., sites, peope, and objects) and topic annotations, then used gaph-cut techniques in stoy custeing. In Zhai and Shah [2005], the authos poposed techniques to ink news stoies fom diffeent TV channes by textua coeation and keyfame matching. In Duyguu et a. [2005], the authos pesented techniques to detect and tacepeated sequences of news shots. Topic custes ae discoveed based on the association of textua and visua cues. Wu et a. [2008] pesented a system buit on visua nea-dupicate constaints that is appied on top of text to impove the stoy custeing and mining. Consideing the effectiveness of muti-modaity infomation fo event tacking, we wi aso adopt textua and visua infomation fo event desciption. Howeve, diffeent fom existing wok, ou aim is to tack events in diffeent domains jointy. Mutitaget Tacking. The Mutitaget Tacking (MTT) estimates the dynamica states of the tagets given a sequence of measuement data eating to the states. The pincipa difficuty in the MTT pobem is due to the unknown data association between tagets and measuements due to the pesence of noise and cutte [Yu and Medioni 2009; Zhang et a. 2013a, 2012a]. To dea with this pobem, many data association methods have been poposed in the iteatue, such as Joint Pobabiistic Data Association fite (JPDA) [Ba-Shaom et a. 2009] and methods based on the patice fite [Zhang et a. 2012c; Knan et a. 2005]. Howeve, consideing infomation of futue feedback and past simutaneousy is usuay moe effective to ovecome the ambiguities in the tacking pocess, which is the main conception of cuent methods such as mutipe hypotheses tacking (MHT) [Cox and Hingoani 1979], Pobabiistic Muti-Hypothesis Tacking (PMHT) [Steit and Luginbuh 1995], MCMC data association [Yu and Medioni 2009], gaph mode [Shitit et a. 2011; Liu et a. 2012], and hieachica association [Xing et a. 2009]. Most of the existing methods ae based on a simia fomuation and ACM Tansactions on Mutimedia Computing, Communications and Appications, Vo. 10, No. 4, Atice 31, Pubication date: June 2014.

5 Coss-Domain Muti-Event Tacking via CO-PMHT 31:5 conception as MHT. The efficient MHT is the most cassica appoach fo data association in tacking by a detection famewok, which has been fomey widey and successfuy used fo point tacking in ada and sona signa pocessing. The PMHT uses soft posteio pobabiity associations between measuements and tagets. Compaed with MHT, the PMHT has much ess computationa compexity. Moeove, a of the existing methods focus on taget tacking in a singe domain ony. Diffeent fom the existing tacking methods, ou CO-PMHT is nove and can tack mutipe events in a coss-domain coaboativey. 3. MULTI-EVENT TRACKING VIA CO-PMHT In this section, we give a detaied desciption of ou poposed muti-event tacking method CO-PMHT that makes use of coss-domain infomation to enhance the obustness of event tacking. It is aso woth mentioning that in this wok we focus on event tacking, not detection, since the event detection is consideed as input to ou system by seaching fom Googe News and Fick. Indeed, ou appoach coud be combined with any existing event detection agoithms that can automaticay discove a new event, then ou system wi tack the events ove time. 3.1 Pobem Definition An event is something that occus at specific pace and time associated with some specific actions [Yang et a. 1999] and consists of many stoies ove time. Theefoe, event tacking aims at inking togethe evoving and histoica stoies. Ou task is to tack mutipe events ove time. Assume that thee ae M events within a siding-window time T,andthem th event obsevation mode is defined as y m (t) = H m (t) x m (t) + w m (t) (1) fo t = 1, 2,...,T,andm = 1, 2,...,M. Hee, as usua, x m (t) epesents the tajectoy of the m th event at time t and y m (t) is its coesponding obsevation. The obsevation matix H m is known and based o how to obtain obsevation y m based on the state x m. The andom sequences {w m (t)} ae assumed white, zeo mean, Gaussian, and mutuay independent, with E{w m (t)wm T (t)} =R m(t), which is the obsevation covaiance. Fo a muti-event tacking pobem, the key pobem is the data association, namey, how to detemine which measuements (stoies) come fom which events. Hee, the measuements mean stoies of events ove time in event tacking. Fo each t we define { (t), z (t)} n t =1 such that z (t) = y k(t) (t), (2) meaning that the th measuement at time t comes fom event (t) and, of couse, (t) is unknown. As an aside, pease note that we have denoted the numbe of obsevations (measuements) at time t as n t, which is not necessaiy identica to the numbe of events M. Fo simpicity, we ewite a paametes as foows. X = {x m (t)}, whee x m (t) is the state of event m at time t. Z = {z (t)}, whee z (t) is the th measuement vecto at time t. K = { (t)}, whee (t) is the event fom which the th measuement at time t aises. About the pobabiistic stuctue fo the measuement/event associations K, they ae defined as p( (t) = m) = π m (3) and a ae independent andom vaiabes. Based on the pevious definition, ou aim is to estimate the X and K with the obsevation Z ove time. Next, we wi intoduce how to ean these paametes with ou CO-PMHT agoithm. ACM Tansactions on Mutimedia Computing, Communications and Appications, Vo. 10, No. 4, Atice 31, Pubication date: June 2014.

6 31:6 T. Zhang and C. Xu 3.2 Ou Poposed CO-PMHT Ou aim is to tack mutipe events by using the infomation fom two domains, namey, Googe News and Fick, to enhance the obustness of each othe. And ou method is to find the best estimate of the event states, X (1) and X (2), when the measuement souce K is unknown. Hee, fo simpicity, we adopt the supescipt v ={1, 2} to denote diffeent domains. It does this by teating the assignments K as missing data using EM. In EM teminoogy, the compete data ae (X (1), Z (1), X (2), Z (2), K), the incompete data ae (X (1), Z (1), X (2), Z (2) ), and K is the missing data. The auxiiay function is the expectation of the compete data og-ikeihood ove the missing data, which takes the foowing fom. Q ( X (1), X (2), X (1), X (2) ) = 2 v=1 Q ( X (v), X (v) ) ( λ Q X (1), X (2), X (1), X (2) ) Eq. (4) denotes ou coss-domain Q function as the sum ove each domain Q function minus a penaty tem Q that quantifies the disageement of the modes Q in each domain and is eguaized by λ. When the eguaization paamete λ is zeo, then Q(X (1), X (2), X (1), X (2) ) = v Q(X(v), X (v) ). In each step, EM then maximizes the v tems Q(X (v), X (v) ) independenty. It foows fom Dempste et a. [1977] that each Q(X (v), X (v) ) inceases in each step and theefoe v Q(X(v), X (v) ) is maximized. Next, we wi intoduce the definitions of the two tems in Eq. (4), espectivey. (1) Q(X (v), X (v) ). In Eq. (5), we show the definition of Q(X, X ). Hee, fo simpicity, we wi ignoe the supescipt v when thee is no confusion. In addition, the foowing independence assumptions ae made: (a) a state sequences ae independent of each othe; (b) the unknown tue assignments ae independent, identicay distibuted andom vaiabes with a pio pobabiity mass as shown in Eq. (3); and (c) a measuements ae conditionay independent given the assignments and the states of the tagets. Q(X, X ) = E { } og p(x, Z, K) Z, X = p(k Z, X ) og p(x, Z, K) = og p(x) + p(k Z, X ) og p(z, K X) (5) K Hee the summation is ove a pemutations of the assignment vaiabe K. The agoithm is an iteative agoithm, that asymptoticay appoaches a oca maximum of the EM auxiiay function by efining estimates of the states X. Attheth iteation, the estimated states ae denoted as X. The iteations ae epeated unti the auxiiay function conveges and the agoithm s output is the fina estimate. In addition, p(x), p(k Z, X ), andp(z, K X) ae defined as [ ] T og p(x) = og p(x (0)) p(x (t) X (t 1)), (6) m=1 t=1 [ M ] M T = og p(x m (0)) p(x m (t) x m (t 1)), m=1 m=1 t=1 M M T = og [p(x m (0))] + og [p(x m (t) x m (t 1))], t=1 K m=1 t=1 T T n t p(z, K X) = p(z (t), K (t) X (t)) = p ( z (t) x k (t) ) π k (t), (7) t=1 =1 ACM Tansactions on Mutimedia Computing, Communications and Appications, Vo. 10, No. 4, Atice 31, Pubication date: June (4)

7 Coss-Domain Muti-Event Tacking via CO-PMHT 31:7 p(k Z, X) = T n t p( (t) z (t), X (t)), (8) t=1 =1 whee n t is the numbe of measuements at time t, T is the numbe of time sampes in the batch, and M is the numbe of events. Combining Eqs. (7), (8), and (5), we obtain Q(X, X ) = { T } n t T n og[p(z (t) x k (t))π k (t)] p( (t) z (t), X (t)) + og p(x). (9) K t=1 =1 t=1 =1 Then, the Q function in each domain is obtained as [ M T nt ] M T n t Q(X, X ) = og p(x) + w,m (t) og [π m (t)] + w,m (t) og [p(z (t) x m (t))], (10) m=1 t=1 whee w,m (t) at each iteation is =1 w,m (t) = π m (t) p(z (t) x m (t)) p(z (t) X (t)) m=1 t=1 =1 = π m (t) N (z (t) ; Hx m (t), R m (t)) M k=1 π k (t) N (z (t) ; Hx k (t), R k (t)). (11) whee N (X; μ, ) is Gaussian density in vaiabe X, with mean μ and covaiance. (2) Q. The disageement tem Q shoud satisfy a numbe of desideata. Fist, since we want to minimize Q, it shoud be convex. Second, fo the same eason, it shoud be diffeentiabe. Given (X (1), X (2) ), we woud ike to find the maximum of Q(X (1), X (2), X (1), X (2) ) in one singe step. We woud, thid, appeciate if Q was zeo when the views totay agee. Theefoe, the Q is defined as in Eq. (12). The Q can be thought of as a paiwise Kuback-Leibe divegences [Kuback and Leibe 1951] of the posteios between a domains and is convex. Q ( X (1), X (2), X (1), X (2) ) = v u T n t M t=1 =1 =1 p ( (t) z (v) Based on Eq. (10) and Eq. (12), we ewite Eq. (4) as Q ( X (1), X (2), X (1), X (2) ) 2 = Q (v)( X (v), X (1), X (2) ), = v=1 v=1 t=1 =1 =1 (t), X (v) (t) ) og p( (t) z (v) (t), X (v) (t) ) p ( (t) z (u) (t), X (u) (t) ) (12) 2 T n t M p (v) ( (t) z (t), X (t)) (13) og [ π (v) (t) p ( z (v) (t) x (v) (t) )] + og p ( X (1)) + og p ( X (2)), p (v) ( (t) z (t), X (t)) = p ( (t) z ( v) (t), X ( v) (t) ), (14) p ( (t) z (v) (t), X (v) (t) ) = w (v), (t), (15) whee w (v), (t) is defined as in Eq. (11) fo domain v, λ = 1, and m =. The poof fo Eq. (13) is given in Appendix 6. We wi adopt the EM agoithm to maximize Eq. (13). In ode to impement the M-step, ACM Tansactions on Mutimedia Computing, Communications and Appications, Vo. 10, No. 4, Atice 31, Pubication date: June 2014.

8 31:8 T. Zhang and C. Xu Fig. 3. The basic idea of ou poposed CO-PMHT agoithm fo mutipe-events tacking in a coss-domain (Googe News and Fick). In each iteation, we attempt to estimate the states x m (t) and assignments (t). To enhance the obustness, infomation in two domains inteacts by using w (v), (t), which is the posteio pobabiity as defined in Eq. (16). Due to the posteio pobabiity, the bidge between two domains is semantic and without a domain gap. we have to maximize Q(X (1), X (2), X (1), X (2) ) given (X (1), X (2) ) and set the deivative to zeo. Because paametes (X (1), X (2) ) occu in the ogaithmized posteios, we diffeentiate a sum of ikeihoods within a ogaithm as in Eq. (13) to faciitate the optimization. Afte optimization, we can obtain the states (X (1), X (2) ) and achieve muti-event tacking. The basic idea of ou CO-PMHT is shown in Figue 3. Ou aim is to estimate the states of mutipe events ove time by consideing the coss-domain infomation. Ou bidge between two domains is the posteio pobabiity w (v), (t), which is semantic and educes the domain gap. 4. OPTIMIZATION In this section, we intoduce how to optimize the Q function as shown in Eq. (13) to estimate the states of mutipe events in a coss-domain ove time by the unsupevised EM agoithm. The M-steps can be executed independenty in the domains but the pobem is how the E- and M-steps shoud be inteeaved. EM can be impemented such that a goba E-step is foowed by M-steps in a domains o, atenativey, we can iteate ove the domains in an oute oop and execute an E- and an M-step in the cuent domain in each iteation of this oop. We adopt the atte stategy because consecutive M-steps in mutipe domains impose the isk that the two modes fip thei dissenting opinions. We obseve empiicay that this effect sows down the convegence of optimization. In Eq. (13), paametes (X (1), X (2) ) can be obtained by maxmizing Q(X (1), X (2), X (1), X (2) ) and they occu ony in the ogikeihood tems. It now becomes cea that the M-step can be executed by finding paamete estimates of p(z (v)(t) x(v) (t)) and π (v) (t) independenty in each domain v. The E-step can be caied out by cacuating and aveaging the posteios p (v) ( (t) z (t), X (t)) accoding to Eq. (14), which specifies how the domains inteact. Theefoe, in ou EM agoithm impementation, the M-step is cacuated in each domain independenty, and the inteaction is in E-step by the posteios p (v) ( (t) z (t), X (t)) as shown in Eq. (14). Based on the peceding discussion, we intoduce how to impement the E-step and M-step fo domain v at the th iteation. When the execution has eached time step fo domain v, the paametes X ( v) +1 have aeady been estimated in the case with two domains. In the E-step, we can theefoe ACM Tansactions on Mutimedia Computing, Communications and Appications, Vo. 10, No. 4, Atice 31, Pubication date: June 2014.

9 Coss-Domain Muti-Event Tacking via CO-PMHT 31:9 detemine the posteio p( (t) z v (t), X ( v) +1 (t)) using the most ecent paamete estimates. In the succeeding M-step, the Q function in domain v is maximized ove the paamete X (v). 4.1 E-Step Ou coss-domain Q function is defined in Eq. (13). In the E-step, ou aim is to cacuate w (v), (t) fo each domain v. Hee, w (v), (t) has the intepetation that it is the posteio pobabiity (conditioned on Z and X in domain v) that the th measuement at time t aises fom event (t). At the th EM iteation, the w (v), (t) shoud be the posteio pobabiity based on othe domains as shown in Eq. (14). Fo mutipe-events tacking in two domains, the w (v), (t) in domain v is defined as in Eq. (16). w (v), (t) = p ( (t) z ( v) (t), X ( v) (t) ) = π ( v) M =1 π ( v) (t) N ( z ( v) (t) ; Hx ( v) (t) N ( z ( v) (t), R ( v) (t) ) (t) ; Hx ( v) (t), R ( v) (t) ) (16) 4.2 M-Step In the M-step fo domain v, the coesponding auxiiay function Q is shown in Eq. (13). Next, we give the detais about how to estimate othe paametes of the mode. In Eq. (13), we shoud obtain the soution π (v) (t) and it has a constaint M =1 π (v) (t) = 1, (17) which means that {π (v) (t)} shoud be a pope pobabiity vecto. The soution is achieved by using a Lagangian M T n t L = w (v), (t) og [ π (v) (t) ] T M + γ t 1 (t), (18) =1 t=1 =1 whee γ t is the Lagange mutipie. Diffeentiating the Lagangian with espect to π (v) (t) and setting the esut to zeo gives the necessay condition. In addition, using the constaint in Eq. (17), we obtain the updated pio estimate π k (t) = 1 n t n t =1 t=1 =1 π (v) w (v), (t), (19) that is, the weights eative fequency fo time t. In Eq. (13), consideing the event states and the measuements fo domain v, we obtain Q X = og p ( X (v)) T n t M + t=1 =1 =1 w (v), (t) og [ p ( z (v) (t) x (v) (t) )]. (20) By maximizing Q X, we can obtain the state X. This is a maximum ikeihood pobem with weighted measuements. The objective ikeihood function in Eq. (20) is the same as the ikeihood function of an unambiguous measuement pobem, except fo the measuement tem. The unambiguous measuement case is maximized by the Kaman smoothe when thee ae inea Gaussian statistics. Fo inea ACM Tansactions on Mutimedia Computing, Communications and Appications, Vo. 10, No. 4, Atice 31, Pubication date: June 2014.

10 31:10 T. Zhang and C. Xu Gaussian statistics, Eq. (20) becomes Q X = M T =1 t=1 + M =1 ( z (v) og [ p ( x (v) (t) x (v) (t 1) )] og [ p ( x (v) (0) )] + C + (t) Hx (v) (t) ) T R 1 M T n t =1 t=1 =1 w (v), (t) 2 (t) ( z (v) (t) Hx (v) (t) ), (21) whee C is a constant facto due to the measuement density nomaization and we ignoe the supescipt v fo R and H fo simpicity. Though agebaic manipuation, Eq. (21) can be ewitten as whee and Q X = M T =1 t=1 + M =1 (v) ( z og [ p ( x (v) (t) x (v) (t 1) )] (22) og [ p ( x (v) (0) )] + C + (t) Hx (v) z (v) (t) = (t) )T R 1 (t) 1 nt =1 w(v), (t) R (v) (t) = nt n t =1 R (v) M =1 t=1 T 1 2 (v) ( z (t) Hx (v) (t) ), w (v), (t) z (v) (t), (23) =1 w(v), (t). (24) The ikeihood function in Eq. (22) is the same as the ikeihood function optimized by the Kaman smoothe. Howeve, instead of the measuement pobabiity density fo mode p(z (v)(t) x(v) (t)), a modified measuement density with covaiance R k (t) appeas. The measuement centoid z k (t) acts as a synthetic measuement. Theefoe, the event state estimate x k (t) can be found using a Kaman smoothe whee the measuement pobabiity density is given as in Eq. (25). (v) p ( z (t) x (v) (t) ) = 2π R (v) (t) { 1 2 exp 1 (v) ( z k 2 (t) Hx (v) (t) )T R 1 (v) (t) ( z (t) Hx (v) (t) )} (25) 4.3 Convegence Anaysis The whoe opeation of ou poposed CO-PMHT is shown in Agoithm 1. In each step, the oca function Q (v) inceases in each domain v. Since a othe Q ( v) ae constant in X (v), this impies that aso the goba function Q inceases. In each iteation of the egua EM agoithm, p(z X +1) p(z X ) 0. This method can be guaanteed to convege and the poof is easiy deived fom the convegence guaantees of egua EM [Dempste et a. 1977]. In addition, Q is convex and can guide the seach to a paamete egion of ow eo due to its popety. In ou expeiments, we show that easonaby accuate soutions ae avaiabe afte 3 6 iteations and thee is sedom any change at a afte 9 15 iteations. Figue 4 shows an exampe of the iteation pocess. ACM Tansactions on Mutimedia Computing, Communications and Appications, Vo. 10, No. 4, Atice 31, Pubication date: June 2014.

11 Coss-Domain Muti-Event Tacking via CO-PMHT 31:11 Fig. 4. An exampe of the iteation pocess of ou optimization. ALGORITHM 1: Ou Poposed CO-PMHT Agoithm fo Coss-Domain Mutipe-Events Tacking. Input : Data Z (1), Z (2). Output: X = ( ) X (1), X (2). Detemine initia vaues fo the tajectoy vaiabes X (1) 0, X(2) 0 fo a events m = 1, 2,...,M and a times t = 1, 2,...,T ; Let the EM iteation index = 1. whie Q in Eq. (13) not conveged do fo v = 1,...,2 do Cacuate the posteio association pobabiities w (v), (t) fo a events = 1, 2,...,M and fo a times t = 1, 2,...,T, and measuements = 1, 2,...,n t, accoding to Eq. (16). (Note w (v), (t) in domain v is cacuated by the infomation fom domain v). Cacuate the synthetic measuements z (v) (t) and thei associated (synthetic) measuement covaiances R (v) (t) fo a events = 1, 2,...,M and times t = 1, 2,...,T, accoding to Eq. (23) and Eq. (24), espectivey. Fo each event = 1, 2,...,M, use a Kaman smoothing agoithm to obtain the estimated tajectoy x (v) (t), accoding to the Eq. (25), but with the diffeence that the synthetic measuements and covaiances { z (v) (t)} and { R (v) (t)} ae used. end Incement = +. end 5. EXPERIMENTAL RESULTS In this section, we pesent extensive expeimenta esuts on ou coected dataset in ode to vaidate the poposed appoach. Fist, we intoduce the detais about the dataset constuction. Then, we show the featue extaction by using both textua and visua infomation. Finay, we give muti-event tacking esuts and anaysis. 5.1 Dataset Constuction Since thee is no coss-domain muti-modaity dataset avaiabe fo muti-event tacking in the mutimedia society, we coect the dataset ouseves. In ode to compehensivey evauate the pobem of ACM Tansactions on Mutimedia Computing, Communications and Appications, Vo. 10, No. 4, Atice 31, Pubication date: June 2014.

12 31:12 T. Zhang and C. Xu Tabe I. Detais of Ou Coected Coss-Domain Dataset Googe News Fick Event ID Event Name Stat Time End Time #Image #Text #Image #Text 1 Senkaku Isands Dispute Occupy Wa Steet Cannabis Legaization in The United States Noway Attacks Chinese Mik Scanda Gangnam Stye United States Pesidentia Eection Potests against the Iaq Wa Disputes in the South China Sea Mass Kiings in Ameica Geek potests Aab Sping s Euopean Soveeign Debt Cisis Mas Reconnaissance Obite the Wa in Afghanistan Bahaini Upising Noth Koea Nucea Pogam Geat Recession coss-domain muti-event tacking, we manuay define 18 events. In ode to cove the whoe evoutionay pocess of each event, we manuay ceate the intoduction page of each event o downoad it fom the Wikipedia page 1. In ou impementation, the timeines of events Senkaku Isands Dispute, 2012 Gangnam Stye, and Mass Kiings in Ameica ae manuay obtained based on the infomation on the pages 2,3,4, espectivey. Fo othe events, we make use of the avaiabe Wikipedia pages to obtain the timeines. Taking event Occupy Wa Steet as an exampe, its timeine is on the page 5. The eades can obtain the timeines by seaching keywods incuding event name, timeine, and Wikipedia with Googe. Fo each tem of the timeine, we seect sevea keywods as Googe News and Fick seach queies. Then, we seach and downoad eated Web pages fom both Googe News and Fick based on the keywods fo each event ove time. Fo each quey, the coesponding texts (on Fick, they ae desciptions, tags, and tite whie on Googe News, they ae officia news atices) and images ae downoaded and we denote them as one stoy of the event. Finay, ou cossdomain muti-modaity dataset is constucted. The detais of ou coected dataset shown in Tabe I. The coected 18 events cove a wide ange of topics ae incuding poitics, economics, techniques, entetainment, miitay, and society. Totay, thee ae 3583 documents and 6742 images fom Googe News and 4356 documents and 4356 images fom Fick. Fo the same quey with mutipe etuned esuts, we adopt the max pooing stategy to obtain the fina epesentation. The time ange diffes fom 6 months to 10+ yeas. Based on the stat time and the end time of each event, it is cea that many events have time oveap, making the tacking Isands dispute. 3 Stye of Occupy Wa Steet. ACM Tansactions on Mutimedia Computing, Communications and Appications, Vo. 10, No. 4, Atice 31, Pubication date: June 2014.

13 Coss-Domain Muti-Event Tacking via CO-PMHT 31:13 pobem difficut. Based on the expeimenta esuts, we can see that ou agoithm can successfuy tack mutipe events with time oveap. 5.2 Featue Extaction Textua Featue. Fo each stoy of an event, fo the text infomation, two Engish an anguage pepocessing tasks ae conducted: the fist task is segmenting the document by-engish wod segmentation pogam to get the sepaate featue items in wods. The second is to emove the stop wods, such as moda patices, pepositions, conjunctions, and punctuation, that contibute ess to the text cassification. Afte the pepocessing, we extact the emaining wods as the featue items of the event texts which can be expessed as (k 1, k 2,...,k n). This atice takes the commony used vecto space mode to epesent the text featues [Saton et a. 1975], that expesses a document as a space vecto, that is, W = (k 1,w 1 ; k 2,w 2 ;...; k n,w n ). Hee w i is the coesponding weight of k i. In this atice, the coesponding weight of the featue item is cacuated by TF-IDF [Saton and Buckey 1988] vaue. The fomua of cacuating weights is geneay w i = TFIDF i = TF i og ( N / ) DF i, (26) in which the expession of TF i is TF i = tf i / n j=0 tf j, whee tf i means the fequency of featue item k i appeaing in the document d, N epesents the tota numbe of a taining documents, and DF i is the numbe of documents containing the featue item k i. Visua Featue. Fo image desciption, we adopt the popua spase coding-based methods which show good pefomance compaed with othe state-of-the-at methods [Liu et a. 2011; Zhang et a. 2013b]. It incudes the extaction of oca descipto, codebook design, oca desciptos coding and pooing, and a spatia ayout into the fina featue. The oca desciptos can be obtained by extaction of SIFT fom images. The codebook (o dictionay) is buit to epesent the oca desciptos with K-means. Fo the oca descipto coding, the Locaized Soft-assigment Coding (LSC) [Liu et a. 2011] is adopted. The next step is to obtain a compact desciption fom the obtained codes with pooing, such as max pooing. Finay, the Spatia Pyamid Matching (SPM) step is usuay expoited to incude some spatia ayout infomation. Such vectos of fixed size can then viewed as a desciption of the image. The basic idea of LSC [Liu et a. 2011] is to use the k visua wods in the neighbohood of a oca featue to obtain the codes. Let b i (b i R d ) denote a visua wod o a basis vecto, whee d is the dimensionaity of a oca featue. The tota numbe of visua wods is n. AmatixB = [ ] b 1, b 2,...,b n denotes a visua codebook o a set of basis vectos. Let x i (x i R d )betheith oca featue in an image. Let z i (z i R n ) be the coding coefficient vecto of x i,withz ij being the coefficient with espect to wod b j. exp ( α xi b j 2 ) 2 z ij = n k=1 exp ( α x i b k 2 ), b k N k (x i) (27) 2 Finay, we concatenate the textua and visua featues and obtain the fina epesentation fo each stoy of an event. 5.3 Event Tacking Resuts and Anaysis In this section, we wi show the muti-event tacking esuts. We can see that ou poposed CO-PMHT agoithm can coaboativey tack mutipe events in a coss-domain with muti-modaity infomation and boost the tacking pefomance. In Sections and 5.3.2, we give a quaitative and quantitative anaysis of the poposed CO-PMHT agoithm, as we as compae it with sevea baseine methods with diffeent expeimenta setups. The expeiments show that CO-PMHT poduces moe obust and accuate tacks. ACM Tansactions on Mutimedia Computing, Communications and Appications, Vo. 10, No. 4, Atice 31, Pubication date: June 2014.

14 31:14 T. Zhang and C. Xu Quaitative Evauation. In Figue 5, we show tacking esuts of ou CO-PMHT on a subset of the events due to imited space. In Figue 5(a), we show tacking esuts of fou events on Googe News. They ae 2008 Chinese Mik Scanda, 2012 United States pesidentia eection, Occupy Wa Steet, and Senkaku Isands Dispute, espectivey. In Figue 5(b), we give the tacking esuts of fou events on Fick, namey, Occupy Wa Steet, 2008 Chinese Mik Scanda, Geek potests, and Bahaini Upising, espectivey. To show the tacking esuts effectivey, we andomy seect six stoies fom each event and show each event as one ow in Figue 5(a). Fo each stoy, we seect sevea keywods to epesent the coesponding textua infomation and aso use one image to denote the visua infomation. Fom Figue 5(a) and Figue 5(b), we can see some stoies denoted with ed bounding boxes that epesent these stoies incoecty tacked. Fo exampe, fo event Occupy Wa Steet, the thid stoy in Figue 5(a) is tacked as the event Potests against the Iaq Wa. It is because this stoy is vey simia to the event Potests against the Iaq Wa. Howeve, fom the esuts, it can be seen that ou CO-PMHT tacking agoithm can coecty tack most of the stoies Quantitative Evauation. To give a fai quantitative compaison among the tackes, we obtain manuay abeed gound-tuth tacks fo a the events, which can be obtained when we downoad the data by queies. Hee, we compae ou poposed CO-PMHT with the popua tacking method PMHT [Steit and Luginbuh 1995] with the same visua featues, textua featues, and fusion featues (combination of the same visua and textua featues), espectivey. Fo simpicity, they ae denoted as Visua, Textua, and Fusion, espectivey, as shown in Tabe II. The detais of the featue extaction ae intoduced in Section 5.2. We know the gound tuth fo stoies of a events and can adopt the popua mean aveage pecision (MAP) to evauate ou poposed CO-PMHT agoithm. Fo each event, thee ae many stoies, and we know thei gound tuth. Afte tacking, we know the associated abe fo each stoy. Based on the tacking esuts, we can cacuate the aveage pecision fo each event, finay, cacuating the mean of the aveage pecisions of mutipe events and obtaining the MAP. In Tabe II, we show the muti-event tacking esuts on Googe News and Fick. Note that, fo PMHT, the MAP (Googe) means ony on Googe and the MAP (Fick) means ony on Fick. Fo CO-PMHT, the MAP (Googe) means Googe-based event tacking with the hep of Fick and the MAP (Fick) means Fick-based event tacking with the hep of Googe. Fom the esuts, we can see the foowing. (1) Tacking pefomances on Googe News ae much bette than the esuts on Fick. This is because infomation on Googe News is officia, wheeas infomation on Fick is fom the Web uses, thus has much moe noise. (2) The esuts with muti-modaity fusion ae the best compaed with both visua-infomation- and textua-infomation-based methods except fo PMHT on Fick. Fo PMHT on Fick, if we combine the visua featues and textua featues, the esut is wose than ony using textua infomation, which shows that a simpe fusion of diffeent featues cannot guaantee pefomance impovement. In addition, textua featues show much bette pefomance than visua featues. (3) Ou CO-PMHT shows much bette pefomance compaed with the popua tacke PMHT. This is because ou poposed CO-PMHT tacks mutipe events in the two domains coaboativey and it can compement the weaknesses of theefoe each boosting the tacking pefomance. This shows ou CO-PMHT eaning is effective and pecise. (4) Ou CO-PMHT shows significant impovement. On Fick, ou CO-PMHT shows 29.69%, 35.12%, 41.82% impovement with visua featues, textua featues, and fusion of visua featues ACM Tansactions on Mutimedia Computing, Communications and Appications, Vo. 10, No. 4, Atice 31, Pubication date: June 2014.

15 Coss-Domain Muti-Event Tacking via CO-PMHT 31:15 Fig. 5. Tacking esuts of ou CO-PMHT on eight events. Six stoies fom each event ae seected to show the esuts. Each stoy incudes one image and sevea keywods. Stoies tacked incoecty ae denoted with ed bounding boxes. Fo a bette view, pease see the oigina PDF fie. ACM Tansactions on Mutimedia Computing, Communications and Appications, Vo. 10, No. 4, Atice 31, Pubication date: June 2014.

16 31:16 T. Zhang and C. Xu Tabe II. Compaison Resuts on Googe News and Fick with Thee Diffeent Featues Agoithm PMHT CO-PMHT Featue Visua Textua Fusion Visua Textua Fusion MAP (Googe) MAP (Fick) Visua (visua featue), Textua (textua featue), and Fusion (visua featue and textua featue). Fig. 6. Confusion matix fo muti-event tacking esuts. and textua featues, espectivey. On Googe News, with diffeent featues, the impovements of CO-PMHT ae 2.88%, 2.08%, and 1.53%, espectivey. (5) Googe News and Fick can compement each othe. As shown in Tabe II, the tacking pefomances on Googe News and Fick ae boosted via ou CO-PMHT. The impovement of CO-PMHT on Fick is much bette than that on Googe News. This is because the stength of Googe News is bette than Fick and Googe News can significanty boost the tacking pefomance on Fick. In addition, the tacking pefomance on Googe News is aso impoved even though the tacking pefomance on Fick is not high due to noise. (6) Ou CO-PMHT can guaantee convegence in two domains. Fo the poposed CO-PMHT with any kinds of featues, the pefomances on two domains ae amost simia, demonstating the convegence of ou agoithm. Fom the peceding esuts, we can see that Googe News and Fick can compement each othe via ou poposed CO-PMHT agoithm and the tacking pefomance on Fick is significanty boosted by Googe News. In a wod, the expeimenta esuts demonstate the effectiveness of muti-modaity and coss-domain infomation fo mutipe-events tacking. ACM Tansactions on Mutimedia Computing, Communications and Appications, Vo. 10, No. 4, Atice 31, Pubication date: June 2014.

17 Coss-Domain Muti-Event Tacking via CO-PMHT 31:17 Tabe III. Resuts of Event Topic Mining China maijuana beivik mik Obama Mas Japan cannabis poice meamine Romney MRO isands medica Noway food Repubican Obite Senkaku aw Nowegian Chinese pesident wate Taiwan possession Oso scanda eection science watch deciminaize Juy chiden state Matian miitay US attacks Sanu US aunch Fo each event, we show the top 7 wods. We aso show the confusion matix as in Figue 6. Fom the esuts, we can see some events have bad tacking esuts because they ae simia to othe events. Let s take the events Geek potests and Occupy Wa Steet as an exampe. We can see that some stoies fom Geek potests and Occupy Wa Steet ae confused, because the two events have some simia topics. Moeove, ou tacke confuses Chinese Mik Scanda with 2012 United States pesidentia eection o Aab Sping. This is because the miscassified stoies fom 2012 United States pesidentia eection o Aab Sping have much highe pobabiity of beonging to the event Chinese Mik Scanda than othe events based on ou agoithm Appications. Afte tacking, we can do two appications: event visuaization and event topic mining. Based on the timeine of each event, we can do event visuaization ove time as shown in Figue 2. Moeove, based on the tacking esuts, each tajectoy coesponds to an event. Note that we do not know the topics of this event, athe we ony have its documents and images. Fo bette undestanding, we can adopt any existing topic modeing methods [Bei et a. 2003] to mine its topics. Some esuts ae shown in Tabe III. Hee, we show the esuts fo six diffeent events and thei top seven wods. Based on the esuts, we can see that these wods amost exacty descibe the events, which can hep us undestand these events. 6. CONCLUSION In this atice, we fomuate muti-event tacking in a coss-domain as a CO-PMHT eaning pobem that incopoates effective infomation between domains and can be efficienty optimized by an EM agoithm. We extensivey anayze the pefomance of ou tacke on chaenging ea-wod events and show that it outpefoms othe baseine methods. This is because ou agoithm can adopt infomation in a coss-domain to hep each othe, especiay when the stengths of one domain compement the weaknesses of the othe. In the futue, we wi investigate event detection and combine the methods to buid a ea system. APPENDIX Q ( X (1), X (2), X (1), X (2) ) = T n t M v u = v u t=1 =1 =1 T n t M t=1 =1 =1 p ( (t) z (v) p ( (t) z (v) (t), X (v) (t), X (v) (t) ) og p( (t) z (v) (t), X (v) (t) ) p ( (t) z (u) (t), X (u) (t) ) (t) ) og p( (t), z (v) (t) X (v) (t) ) p ( z (u) (t) X (u) (t) ) p ( (t), z (u) (t) X (u) (t) ) p ( z (v) (t) X (v) (t) ) (28) ACM Tansactions on Mutimedia Computing, Communications and Appications, Vo. 10, No. 4, Atice 31, Pubication date: June 2014.

18 31:18 T. Zhang and C. Xu = v u T n t M t=1 =1 =1 T n t M + v u = T n t M v u = t=1 =1 =1 t=1 =1 =1 2 T n t M v=1 t=1 =1 =1 p ( (t) z (v) p ( (t) z (v) p ( (t) z (v) [ p ( k (t) z (v) (t), X (v) (t), X (v) (t), X (v) (t), X (v) (t) ) og π (v) π (u) (t) p ( z (v) (t) x (v) (t) ) (t) p ( z (u) (t) x (u) (t) ) (t) ) og p( z (u) (t) X (u) (t) ) p ( z (v) (t) X (v) (t) ) (t) ) og π (v) π (u) (t) ) p ( (t) z ( v) (t) p ( z (v) (t) x (v) (t) ) (t) p ( z (u) (t) x (u) (t) ) + (t), X ( v) T n t v u t=1 =1 (t) )] og [ π (v) og p( z (u) (t) X (u) (t) ) p ( z (v) (t) X (v) (t) ) (t) p ( z (v) (t) x (v) (t) )] + 0 Q ( X (1), X (2), X (1), X (2) ) = v Q( X (v), X (v) = 2 T n t M v=1 t=1 =1 =1 + og p ( X (v))] ) ( Q X (1), X (2), X (1), X (2) ) [ p ( k (t) z ( v) (t), X ( v) (t) ) og [ π (v) (t) p ( z (v) (t) x (v) (t) )] (29) REFERENCES J. Aan Topic Detection and Tacking: Event Based Infomation Retieva. Kuwe Academic Pess. Y. Ba-Shaom, F. Daum, and J. Huang The pobabiistic data association fite. IEEE Conto Syst. Mag. 29, 6, D. M. Bei, A. Y. Ng, and M. I. Jodan Latent diichet aocation. J. Mach. Lean. Res. 3, 3 4, H. L. Chieu and Y. K. Lee Quey based event extaction aong a timeine. In Poceedings of the 27 th Annua ACM SIGIR Intenationa Confeence on Reseach and Deveopment in Infomation Retieva. I. Cox and S. Hingoani An efficient impementation of eid s mutipe hypothesis tacking agoithm and its evauation fo the pupose of visua tacking. IEEE Tans. Patten Ana. Mach. Inte. 24, 6, A. Dempste, N. Laid, and D. Rubin Maximum ikeihood fom incompete data via the em agoithm. J. Roya Statist. Soc. B39, 1, P. Duyguu, J. Y. Pan, and D. A. Fosyth Towads auto-documentay: Tacking the evoution of news stoies. In Poceedings of the 12 th Annua ACM Intenationa Confeence on Mutimedia (Mutimedia 05) J. R. Kende and M. R. Naphade Visua concepts fo news stoy tacking: Anayzing and expoiting the nist tecvid video annotation expeiment. In Poceedings of the IEEE Compute Society Confeence on Compute Vision and Patten Recognition (CVPR 05) Z. Knan, T. Bach, and F. Deaet MCMC-based patice fiteing fo tacking a vaiabe numbe of inteacting tagets. IEEE Tans. Patten Ana. Mach. Inte. 27, 11, S. Kuback and R. A. Leibe On infomation and sufficiency. Ann. Math. Statist. 22, 1, R. Kuma, U. Mahadevan, and D. Sivakuma A gaph-theoetic appoach to extact stoyines fom seach esuts. In Poceedings of the 10 th ACM SIGKDD Intenationa Confeence on Knowedge Discovey and Data Mining. F. Lin and C.-H. Liang Stoyine-based summaization fo news topic etospection. Decis. Suppot Syst. 45, 4, L. Liu, L. Wang, and X. Liu In defense of soft assignment coding. In Poceedings of the IEEE Intenationa Confeence on Compute Vision (ICCV 11) S. Liu, S. Yan, T. Zhang, C. Xu, J. Liu, and H. Lu Weaky-supevised gaph popagation towads coective image pasing. IEEE Tans. Mutimedia 14, 2, J. Makkonen, H. A. Myka, and M. Samenkivi Simpe semantics in topic detection and tacking. Inf. Ret. 7, 3 4, G. Saton and C. Buckey Tem-weighting appoaches in automatic text etieva. Inf. Pocess. Manag. 24, 5, G. Saton, A. Wong, and C. S. Yang A vecto space mode fo automatic indexing. Comm. ACM 18, 1, H. Shitit, J. Becaz, F. Feuet, and P. Fua Tacking mutipe peope unde goba appeaance constaints. In Poceedings of the IEEE Intenationa Confeence on Compute Vision (ICCV 11) ACM Tansactions on Mutimedia Computing, Communications and Appications, Vo. 10, No. 4, Atice 31, Pubication date: June 2014.