SHERPA. Smart collaboration between Humans and groundaerial Robots for improving rescuing activities in Alpine environments

Size: px

Start display at page:

Download "SHERPA. Smart collaboration between Humans and groundaerial Robots for improving rescuing activities in Alpine environments"

Bethany Dixon
6 years ago
Views:

1 SHERPA Smart collaboration between Humans and groundaerial Robots for improving rescuing activities in Alpine environments Lorenzo Marconi CASY-DEI University of Bologna

2 SHERPA in Short Integrated Project FP7-ICT (IP Grant Agreement no.: ) Project Officer: Dr. Olivier Da Costa 4 Years (1 February 2013, 31 January 2017) 7 Universities, 2 SMEs, 1 Public organization 10 Workpackages, 38 Research tasks 70 Persons involved (919 Person Months) Advisory Board M total cost 8.55 M financial contribution 2

3 The SHERPA Team Part. # Institution Country Leading scientists 1 (coord) Università di Bologna Lorenzo Marconi 2 University of Bremen Micheal Beetz 3 ETH Zurich Roland Siegwart 4 University of Twente Raffaella Carloni & Stefano Stramigioli 5 University of Leuven Herman Bruyninckx 6 Linkopings University Patrick Doherty 7 Università di Napoli Federico II Vincenzo Lippiello 8 Aslatech (SME) Andrea Sala 9 Bluebotics (SME) Nicola Tomatis 10 Club Alpino Italiano Andreina Maggiore 3

4 The complementary skills of the academic partners Control UNIBO Low level control Supervisory control UT Technology integration Tele-operation ETHZ Sensor fusion Autonomous navigation KUL Architecture (cognition/control) Cogni0on (AI) UNIHB Abstract reasoning Cognitive aspect UNINA Human perception & interpretation LKU Multi-agent cooperation And planning

5 SHERPA Operative Conditions Statistics from CAI 5

The SHERPA Environment Search and rescue in unfriendly and possibly hostile environments, like the ones in which civil protection, alpine rescuers and forest guards usually operate Features:

6 The SHERPA Environment Search and rescue in unfriendly and possibly hostile environments, like the ones in which civil protection, alpine rescuers and forest guards usually operate Features: Unstructured environments Severe operative conditions (weather, terrain,..) Presence of human (rescuer) often focused on demanding tasks Team (Human & Machine) capabilities Emotional aspects

CAI - CNSAS CAI non-profit, non-political

000 operations per year It is a national

7 CAI - CNSAS CAI non-profit, non-political organization, devoted to search and rescue operations in mountains and caves more than volunteer technicians > operations per year It is a national body of the National Service for Civil Protection. 7

8 Italian Statistics OPERATIONS ,000 8,000 8,299 7,000 6,000 5,000 4,000 3,000 2,000 1, ,103 2,148 2,183 1,873 1,344 1,406 1,453 1,521 1, ,037 1, ,403 2,803 2,867 4,342 3,766 3,830 3,893 6,504 6,256 5,810 5,8985,813 5,563 5,568 5,188 4,775 4, ,013 8

9 Italian Statistics Rescuers 40,000 35,000 36,517 30,000 25,000 20,000 22,862 21,764 20,137 19,376 19,521 18,600 19,019 29,983 30,222 28,540 28,894 27,519 27,538 26,685 25,437 25,241 15,000 10,000 5, ,632 11,659 11,650 10,319 10,290 10,859 9,789 9,862 8,5208,833 9,053 9,100 8,079 6,383 6,579 6,946 4,400 4,934 5,513 4,475 3,212 3,647 1,988 2, , , ,213 1,381 1,733 2,309 2,590 2,466 14,407 14,

10 Italian Statistics 4,500 # Helicopters 4,000 3,500 3,000 3,713 3,440 3,424 3,390 3,261 3,144 3,120 3,1803,175 3,883 3,831 3,707 3,060 2,500 2,725 2,405 2,730 2,462 2,000 1,500 1, ,308 1,369 1,390 1,180 1,834 1,

11 Italian Statistics June September: about 50% December February: about 30% 11

12 SAR Operations Many different types of rescue operations: WINTER: Sky (resort / off-piste) Avalanches Ice climbing High altitude SUMMER Search of missing persons Paragliding / Biking / Canyoning Rock climbing High altitude 25/10/2013 SSRR 2013, Linkoping 12

13 SAR Operations Different environmental conditions WINTER: Cold temperatures (-30 o ) Shorter days Mobility difficulties (snow) Generally: worse weather 13

14 SAR Operations Different environmental conditions SUMMER: Temperatures (-10o +40o) Stronger thermal activity (wind) Mobility problems related to vegetation Generally: much larger areas to be patrolled 14

15 THE SHERPA VISION 15

16 The SHERPA Animals Patrolling hawks Busy Genius Trained wasps Intelligent donkey 16

17 The SHERPA Animals Busy genius A human rescuer, expert of the specific rescuing mission or surveillance activity (e.g. a mountain guide, a specifically trained specialist, a forest guard, ) The human transmits wirelessly his position to the robotic platform and communicates to it through handy and easy-to-operate technological devices busy genius Incomparable cognitive capabilities Often distracted by demanding rescuing activity ( sketchy inputs ) 17

The SHERPA Animals Trained wasp Small-scale UAVs with onboard cameras/receivers Improved surveillance, capability of reaching rapidly a target location in the neighborhood of the team, increasing the

18 The SHERPA Animals Trained wasp Small-scale UAVs with onboard cameras/receivers Improved surveillance, capability of reaching rapidly a target location in the neighborhood of the team, increasing the patrolled rescuing area ( flying eyes ) Handy devices: Technically conceived to be supervised by the human in simple and natural way Safe and operable in the vicinity of the human (hand deployment) Multi-rotors configurations 18

19 CNSAS interest Interest in wasps Quadcopters equipped with cameras (RGB, thermal, ) or other sensors fast survey of limited areas able to flight in a forest and scan areas with cameras (normal or thermal); feedback video for ground station; flying eyes for the rescuer exploration of steep walls rescue operations on cableways (cable cars) avalanche equipped with beacon possible new (more efficient) search methods 19

The SHERPA Animals Patrolling hawks High-altitude, High-Payload

high-altitude (50-100m) and complementing the capabilities of

Constructing rough 3D maps of the rescuing area, serving as a

20 The SHERPA Animals Patrolling hawks High-altitude, High-Payload unmanned aerial vehicles, patrolling a large area at high-altitude (50-100m) and complementing the capabilities of the small-scale UAVs. Constructing rough 3D maps of the rescuing area, serving as a communication hub. Fixed-wing, Rmax Complementarities with respect to the small scale UAVs (operational synergies) 20

21 CNSAS interest Interest in hawks RMAX: deployment of wasps to improve efficiency Carry and deploy first aid kit for missing people (sleeping bad, medicines, thermos flask) Radio/Wi-Fi/GSM bridge for not covered areas Flying in the night with laser scanners when real helicopters cannot flight Fixed wing: first and fast scan of incident area Equipped with high resolution cameras could send images of the area to the ground station and with computer vision algorithms detect on snow proofs of potential victim (gloves, ski, backpacks) Radio/Wi-Fi/GSM bridge for not covered areas 21

22 The SHERPA Animals Intelligent donkey Ground rover: intelligent carrying vehicle equipped with a multifunctional robotic arm Recovering/re-charging base for the aerial vehicles, able to improve the operative radius and the overall autonomy of the combined system, carrying vehicle-20 Kg payload The SHERPA Box 22

23 CNSAS interest Interest in donkeys Aircraft carrier : docking and deployment station for wasps to increase efficiency and recharge batteries. Radio bridge. Almost fixed ground station with limited movements. Payload limited for batteries and hardware Useless in winter 23

24 The SHERPA Vision Limited and implicit communication. when human teams successfully perform complex rescue missions (few words and simple gestures especially mountaineers!) Intuitive and adaptive cooperation Context-sensitive information filtering Dynamic cognitive map (multi-faced information) Distributed situation awareness paradigm Hierarchical vs Heterarchical architectures Robust control and mechanics 24

behave like kites virtually pulled by the rescuer ) Ground and aerial deployment Team

25 The SHERPA Benchmarks Virtual ground leashing ( The rescuer virtually pulls the rover, precisely as an owner would do with his pet on the leash ) Virtual aerial leashing ( the UAVs behave like kites virtually pulled by the rescuer ) Ground and aerial deployment Team coordina0on The Avalanche ( the ultimate ) ü Test site of the Institute for Snow and Avalanche Research of ETHZ

26 The SHERPA vision A. Enhancing Cognition through Multi-Modal Interfacing and Interaction ü Distributed Situation Awareness Dynamic Cognitive Map ü Multimodal human robot interaction and gestures as a modality for cognition and control B. Decision Support Systems for Cognitive Decision Making and Cooperation ü Knowledge Representation and Reasoning Technologies ü Reactive Planning and Execution Monitoring ü Delegation, Distributed Planning and Team Collaboration C. Single and Multi-Robotic Control, Actuation and Mechanical Design ü Low-Level Reactive Planning, Control and Data Fusion ü Mechanical Design and Integration D. Distributed Architectures Supporting Cognition, Cooperation and Control ü Architectural Issues and Objectives ü Architectural Support for the Dynamic Cognitive Map E. Validation and Benchmarks 26

27 Conclusions Mountains: a very challenging environment for S&R missions, with many complex operational scenarios Some of these scenarios have been outlined Robotic technology may help, and we have the vision, but FOLLOW US: 27

28 First Experiments Follow us: 28

NSW Mines Rescue Unmanned Aerial Vehicle (UAV)

NSW Mines Rescue Unmanned Aerial Vehicle (UAV) 2015 IMRB Conference Hanover Emergency Response Q. Will the systems providing this required information remain operational once an incident occurs? Likely