www.sherpa-project.eu SHERPA Smart collaboration between Humans and groundaerial Robots for improving rescuing activities in Alpine environments Lorenzo Marconi CASY-DEI University of Bologna
SHERPA in Short Integrated Project FP7-ICT (IP Grant Agreement no.: 600958) Project Officer: Dr. Olivier Da Costa 4 Years (1 February 2013, 31 January 2017) 7 Universities, 2 SMEs, 1 Public organization 10 Workpackages, 38 Research tasks 70 Persons involved (919 Person Months) Advisory Board 11.270 M total cost 8.55 M financial contribution 2
The SHERPA Team Part. # Institution Country Leading scientists 1 (coord) Università di Bologna Lorenzo Marconi 2 University of Bremen Micheal Beetz 3 ETH Zurich Roland Siegwart 4 University of Twente Raffaella Carloni & Stefano Stramigioli 5 University of Leuven Herman Bruyninckx 6 Linkopings University Patrick Doherty 7 Università di Napoli Federico II Vincenzo Lippiello 8 Aslatech (SME) Andrea Sala 9 Bluebotics (SME) Nicola Tomatis 10 Club Alpino Italiano Andreina Maggiore 3
The complementary skills of the academic partners Control UNIBO Low level control Supervisory control UT Technology integration Tele-operation ETHZ Sensor fusion Autonomous navigation KUL Architecture (cognition/control) Cogni0on (AI) UNIHB Abstract reasoning Cognitive aspect UNINA Human perception & interpretation LKU Multi-agent cooperation And planning
SHERPA Operative Conditions Statistics from CAI 5
The SHERPA Environment Search and rescue in unfriendly and possibly hostile environments, like the ones in which civil protection, alpine rescuers and forest guards usually operate Features: Unstructured environments Severe operative conditions (weather, terrain,..) Presence of human (rescuer) often focused on demanding tasks Team (Human & Machine) capabilities Emotional aspects
CAI - CNSAS CAI non-profit, non-political organization, devoted to search and rescue operations in mountains and caves more than 7.000 volunteer technicians > 6.000 operations per year It is a national body of the National Service for Civil Protection. 7
Italian Statistics OPERATIONS 1955-2012 9,000 8,000 8,299 7,000 6,000 5,000 4,000 3,000 2,000 1,000 0 2,103 2,148 2,183 1,873 1,344 1,406 1,453 1,521 1,176 80584689921 9921,037 1,028 631 1391901231058012820315522519520 267 2422267 362385409428 493 570 676 3,403 2,803 2,867 4,342 3,766 3,830 3,893 6,504 6,256 5,810 5,8985,813 5,563 5,568 5,188 4,775 4,874 1955 1956 1957 1958 1959 1960 1961 1962 1963 1964 1965 1966 1967 1968 1969 1970 1971 1972 1973 1974 1975 1976 1977 1978 1979 1980 1981 1982 1983 1984 1985 1986 1987 1988 1989 1990 1991 1992 1993 1994 1995 1996 1997 1998 1999 2000 2001 2002 2003 2004 2005 2006 2007 2008 2009 2010 2011 2012 5,013 8
Italian Statistics Rescuers 40,000 35,000 36,517 30,000 25,000 20,000 22,862 21,764 20,137 19,376 19,521 18,600 19,019 29,983 30,222 28,540 28,894 27,519 27,538 26,685 25,437 25,241 15,000 10,000 5,000 0 13,632 11,659 11,650 10,319 10,290 10,859 9,789 9,862 8,5208,833 9,053 9,100 8,079 6,383 6,579 6,946 4,400 4,934 5,513 4,475 3,212 3,647 1,988 2,067 9251,244 8319117818621,022 9631122 1,213 1,381 1,733 2,309 2,590 2,466 14,407 14,699 1955 1956 1957 1958 1959 1960 1961 1962 1963 1964 1965 1966 1967 1968 1969 1970 1971 1972 1973 1974 1975 1976 1977 1978 1979 1980 1981 1982 1983 1984 1985 1986 1987 1988 1989 1990 1991 1992 1993 1994 1995 1996 1997 1998 1999 2000 2001 2002 2003 2004 2005 2006 2007 2008 2009 2010 2011 2012 9
Italian Statistics 4,500 # Helicopters 4,000 3,500 3,000 3,713 3,440 3,424 3,390 3,261 3,144 3,120 3,1803,175 3,883 3,831 3,707 3,060 2,500 2,725 2,405 2,730 2,462 2,000 1,500 1,000 877 1,308 1,369 1,390 1,180 1,834 1,768 500 0 529 591 531 342 0 0 0 0 0 0 0 0 0 0 0 0 211336537054 97 193 122 169 231213237275 33 370 462 138 1955 1956 1957 1958 1959 1960 1961 1962 1963 1964 1965 1966 1967 1968 1969 1970 1971 1972 1973 1974 1975 1976 1977 1978 1979 1980 1981 1982 1983 1984 1985 1986 1987 1988 1989 1990 1991 1992 1993 1994 1995 1996 1997 1998 1999 2000 2001 2002 2003 2004 2005 2006 2007 2008 2009 2010 2011 2012 10
Italian Statistics - 2012 June September: about 50% December February: about 30% 11
SAR Operations Many different types of rescue operations: WINTER: Sky (resort / off-piste) Avalanches Ice climbing High altitude SUMMER Search of missing persons Paragliding / Biking / Canyoning Rock climbing High altitude 25/10/2013 SSRR 2013, Linkoping 12
SAR Operations Different environmental conditions WINTER: Cold temperatures (-30 o ) Shorter days Mobility difficulties (snow) Generally: worse weather 13
SAR Operations Different environmental conditions SUMMER: Temperatures (-10o +40o) Stronger thermal activity (wind) Mobility problems related to vegetation Generally: much larger areas to be patrolled 14
THE SHERPA VISION 15
The SHERPA Animals Patrolling hawks Busy Genius Trained wasps Intelligent donkey 16
The SHERPA Animals Busy genius A human rescuer, expert of the specific rescuing mission or surveillance activity (e.g. a mountain guide, a specifically trained specialist, a forest guard, ) The human transmits wirelessly his position to the robotic platform and communicates to it through handy and easy-to-operate technological devices busy genius Incomparable cognitive capabilities Often distracted by demanding rescuing activity ( sketchy inputs ) 17
The SHERPA Animals Trained wasp Small-scale UAVs with onboard cameras/receivers Improved surveillance, capability of reaching rapidly a target location in the neighborhood of the team, increasing the patrolled rescuing area ( flying eyes ) Handy devices: Technically conceived to be supervised by the human in simple and natural way Safe and operable in the vicinity of the human (hand deployment) Multi-rotors configurations 18
CNSAS interest Interest in wasps Quadcopters equipped with cameras (RGB, thermal, ) or other sensors fast survey of limited areas able to flight in a forest and scan areas with cameras (normal or thermal); feedback video for ground station; flying eyes for the rescuer exploration of steep walls rescue operations on cableways (cable cars) avalanche equipped with beacon possible new (more efficient) search methods 19
The SHERPA Animals Patrolling hawks High-altitude, High-Payload unmanned aerial vehicles, patrolling a large area at high-altitude (50-100m) and complementing the capabilities of the small-scale UAVs. Constructing rough 3D maps of the rescuing area, serving as a communication hub. Fixed-wing, Rmax Complementarities with respect to the small scale UAVs (operational synergies) 20
CNSAS interest Interest in hawks RMAX: deployment of wasps to improve efficiency Carry and deploy first aid kit for missing people (sleeping bad, medicines, thermos flask) Radio/Wi-Fi/GSM bridge for not covered areas Flying in the night with laser scanners when real helicopters cannot flight Fixed wing: first and fast scan of incident area Equipped with high resolution cameras could send images of the area to the ground station and with computer vision algorithms detect on snow proofs of potential victim (gloves, ski, backpacks) Radio/Wi-Fi/GSM bridge for not covered areas 21
The SHERPA Animals Intelligent donkey Ground rover: intelligent carrying vehicle equipped with a multifunctional robotic arm Recovering/re-charging base for the aerial vehicles, able to improve the operative radius and the overall autonomy of the combined system, carrying vehicle-20 Kg payload The SHERPA Box 22
CNSAS interest Interest in donkeys Aircraft carrier : docking and deployment station for wasps to increase efficiency and recharge batteries. Radio bridge. Almost fixed ground station with limited movements. Payload limited for batteries and hardware Useless in winter 23
The SHERPA Vision Limited and implicit communication. when human teams successfully perform complex rescue missions (few words and simple gestures especially mountaineers!) Intuitive and adaptive cooperation Context-sensitive information filtering Dynamic cognitive map (multi-faced information) Distributed situation awareness paradigm Hierarchical vs Heterarchical architectures Robust control and mechanics 24
The SHERPA Benchmarks Virtual ground leashing ( The rescuer virtually pulls the rover, precisely as an owner would do with his pet on the leash ) Virtual aerial leashing ( the UAVs behave like kites virtually pulled by the rescuer ) Ground and aerial deployment Team coordina0on The Avalanche ( the ultimate ) ü Test site of the Institute for Snow and Avalanche Research of ETHZ
The SHERPA vision A. Enhancing Cognition through Multi-Modal Interfacing and Interaction ü Distributed Situation Awareness Dynamic Cognitive Map ü Multimodal human robot interaction and gestures as a modality for cognition and control B. Decision Support Systems for Cognitive Decision Making and Cooperation ü Knowledge Representation and Reasoning Technologies ü Reactive Planning and Execution Monitoring ü Delegation, Distributed Planning and Team Collaboration C. Single and Multi-Robotic Control, Actuation and Mechanical Design ü Low-Level Reactive Planning, Control and Data Fusion ü Mechanical Design and Integration D. Distributed Architectures Supporting Cognition, Cooperation and Control ü Architectural Issues and Objectives ü Architectural Support for the Dynamic Cognitive Map E. Validation and Benchmarks 26
Conclusions Mountains: a very challenging environment for S&R missions, with many complex operational scenarios Some of these scenarios have been outlined Robotic technology may help, and we have the vision, but FOLLOW US: www.sherpa-project.eu 27
First Experiments Follow us: www.sherpa-project.eu 28