UAV-based monitoring of pedestrian groups Florian Burkert, Friedrich Fraundorfer Technische Universität München, 04.09.2013, UAV-g 2013, Rostock 1
Motivation 2
Overview State of the art UAV imagery for pedestrian group behaviour Scenario planning and data acquisition Pre-processing and dataset Group behaviour model of the complex event detector (CED) Experimental results Discussion 3
State of the art Classification of recurring individual trajectories [Nascimento et al, 2010] Recognition of pairwise human interaction [Oliver et al., 2000] Unusual event detection in pedestrian groups [Mehran et al., 2009] * 1 Recognition of motion patterns in dense crowds [Rodriguez et al., 2011] *² Only a few pedestrians analyzed Recurring events crucial for learning No classification of the situation in a crowd * 1 http://mha.cs.umn.edu/movies/crowd-activity-all.avi *² http://www.di.ens.fr/willow/research/datadriven/ 4
State of the art Goal: Complex event detection in public scenes Perfect birds-eye visibility provided by UAV imagery Contribution: Interpretation of pedestrian group behaviour in unknown scenes Declaration of the specific type of event New UAV test dataset 5
Overview State of the art UAV imagery for pedestrian group behaviour Scenario planning and data acquisition Pre-processing and dataset Group behaviour model of the complex event detector (CED) Experimental results Discussion 6
UAV dataset planning Composition of scenarios Goal: Representation of possible situations in pedestrian crowds Input: - Research on Social Force Model [Helbing] - Everyday life experience - Media showing pedestrian crowds 7
UAV dataset planning Composition of scenarios Goal: Representation of possible situations in pedestrian crowds Input: - Research on Social Force Model [Helbing] - Everyday life experience - Media showing pedestrian crowds Volunteers Simulating predefined scenarios Minimal information provided natural behaviour Material UAV: Battery life + recharging Marking of volunteers Physical obstacles 8
UAV dataset data acquisition Location: pitch in Munich UAV: AscTec Falcon 8 + Panasonic DMC Lumix LX3 Altitude ~85m ~1,5cm ground resolution, coverage 48x27m ROI: 25x25m; Size of pedestrian: 30x45 pixels Flight sessions of 12 minutes 9
UAV dataset pre-processing Image alignment Homography-based transformation using SIFT correspondences Manual pedestrian tracking Trajectories 10
UAV dataset pre-processing Image alignment Homography-based transformation using SIFT correspondences Manual pedestrian tracking Trajectories 11
UAV dataset pre-processing Image alignment Homography-based transformation using SIFT correspondences Manual pedestrian tracking Trajectories 12
UAV dataset Name norm. pace fast pace #sequ. normal #sequ. fast # pics normal #pics fast #pics total 1 Parallel group motion 4 4 76 38 114 2 Diverging 4 2 42 13 55 3 Converging 4 2 47 20 67 4 Random walking 1-127 - 127 5 Individual crossing standing/walking group 8 8 135 71 206 6 Groups crossing standing/head-on/sidewards 12 10 224 88 312 7 Group overtaking group 4-50 - 50 8 Group passing wide gap 4 4 70 44 114 9 Group passing narrow gap 4 4 79 44 123 10 Group passing corridor 4 4 93 52 145 11 Groups passing corridor head-on 4 2 90 38 128 12 Groups merging triple junction 3 3 95 56 151 13 Group avoiding obstacle 4 4 89 41 130 14 Groups brawling 2-102 - 102 15 Group escaping 2-27 - 27 13
Overview State of the art UAV imagery for pedestrian group behaviour Scenario planning and data acquisition Pre-Processing and dataset Group behaviour model of the complex event detector (CED) Experimental results Discussion 14
CED group behaviour model Goal: detection of specific scenarios in public scenes How do pedestrians behave when they are organized in groups? Pedestrian group behaviour model (Considerably influenced by UAV dataset) [Burkert, F., Butenuth, M., 2012. Complex Event Detection in Pedestrian Groups from UAVs. ISPRS Annals I-3, Congress 2012] 15
Overview State of the art UAV imagery for pedestrian group behaviour Scenario planning and data acquisition Pre-Processing and dataset Group behaviour model of the complex event detector (CED) Experimental results Discussion 16
Experimental results Corridor scenario 17
Experimental results Bottleneck scenario 18
Experimental results Bottleneck scenario Robustness: 13 of 16 bottleneck scenarios can be detected (various paces, various gap sizes) 19
Overview State of the art UAV imagery for pedestrian group behaviour Scenario planning and data acquisition Pre-Processing and dataset Group behaviour model of the complex event detector (CED) Experimental results Discussion 20
Discussion UAV imagery or airplane imagery? + monitoring period + flexibility + ground resolution o weather sensitivity o viewing angle Summary Dataset with 15 group motion scenarios Pedestrian group behaviour model Event detection of specific complex scenarios Monitoring of pedestrian groups from UAVs 21
Discussion Outlook Dataset Additional dataset with real-world scenes Image sequences with higher frame-rates Complex Event Detector Probabilistic framework for CED Optional learning of the environment to include obstacles in a scene model 22
Thank you for your attention florian.burkert@bv.tum.de friedrich.fraundorfer@tum.de 23