Probabilistic Methods of Risk Assessment: Industry Views. Dr. Allison Ferguson Director, Airspace Research

Transcription

1 Prbabilistic Methds f Risk Assessment: Industry Views Dr. Allisn Fergusn Directr, Airspace Research

2 1. Intrductin Prblem Statement Summary 2. Operatinal Risk Assessment in Practice Pathfinder (Fcus Area Tw) EVLOS Lcalized BVLOS Lessns learned (s far) 3. Mving Frward Building the data pl Engaging with industry

3 PrecisinHawk LATAS: Our Gals 1 Reduce risk f drne peratins 2 3 Enable advanced peratins such as BVLOS and package delivery thrugh technlgy Build backbne datasets

4 Mitigating Risk thrugh Infrmatin Data Infrastructure Services 3D Obstacle Maps Buildings Trees Terrain Manned Aircraft Radar / ADS-b UAS Surveillance Airspace Cellular Cverage Clud services Cellular Security Bandwidth cnsideratins Registratin Tracking Reprting Flight planning Flight authrizatin Operatr validatin But with each new additinal cmpnent we intrduce new failure mdes hw t assess impact n verall system risk?

5 Mitigating Risk thrugh Infrmatin All risk statements are prbabilistic. The prblem is getting t thse prbabilities: Evaluating ttal system risk (target level f safety) inferentially is.prbably nt a gd idea. There are distributins we CAN access inferentially that have an impact n verall system risk (e.g. metrics f peratr respnse t manned aircraft in peratinal area). Hw gd are the mdels that link what we can measure with verall system risk? Especially in the presence f real-wrld (i.e. messy) data Hw gd d they have t be? Can we mitigate the risk f using a risk mdel?

6 Thrwback Tuesday? Weibel, Rland E., and R. Jhn Hansman. "Safety cnsideratins fr peratin f different classes f UAVs in the NAS." AIAA 4th Aviatin Technlgy, Integratin and Operatins Frum, AIAA 3rd Unmanned Unlimited Technical Cnference, Wrkshp and Exhibit. 2004

7 Risk Mdels and Real Data Histrical Simulatin Field Data A risk mdelling apprach isn t abut ignring real surces f infrmatin, it s abut cllating them in a way that has sme lgical backing. These are prblems with lts f variables. The temptatin is t assume that all surces f variatin are equally imprtant t the final utcme (they are nt).

8 Pathfinder Phase I & II Overview: EVLOS EVLOS is an Unmanned Aircraft System (UAS) peratin whereby the Unmanned Aircraft (UA) may be perated marginally beynd the visual range f the remte pilt in cmmand (hereafter referred t as the PIC) prvided he r she maintains uninterrupted visual surveillance fr manned aircraft within the planned UA perating airspace. Within the EVLOS cnstruct, the UA must remain within the preplanned perating area and UA cntrl authrity must be maintained by the PIC at all times. Phase I Research fcused n identifying the range where an intruder aircraft culd be visually identified. Phase II Research fcused n the PIC ability t react t an intruder aircraft within the EVLOS range. Fcus is n cllectin f quantitative data t supprt a general peratinal risk assessment

9 Phase I & II Overview: EVLOS Industry Impact: Cmmercial waiver t Part 107 VLOS requirement issued t PrecisinHawk (August 2016). Operatinal risk assessment based n Phase I/early Phase II results. Full Phase II results t be incrprated int ASTM F38 standard (2017) t supprt rulemaking effrts.

10 Pathfinder Phase III: Lcalized BVLOS Wide industry applicability. This same picture applies t: Emergency respnse Insurance Frestry/Natural Resurce Management Lcalized BVLOS EVLOS (2.5 nm) EVLOS with RVO (5 nm)

11 Phase III: Lcalized BVLOS Drive creatin f lcalized beynd visual line f sight cncept f peratins in Class G Current peratins t understand risk: Pathfinder, UAS test sites, ASSURE Twards perating n a Natinal Level Shift fcus frm visual base case t technlgy assisted case BUILD ON RESULTS TO DEFINE RISK PROFILE ON NATIONAL SCALE

12 Phase III Missin Statement Evaluate the impact f assistive technlgy (i.e. LATAS) n peratinal risk assciated with flight f suas in BVLOS applicatins in the NAS. What des the technlgy need t d t achieve an acceptable level f safety? In what ways can it fail? Hw d we mitigate the risk f failure? Quantitative answers t these questins prvide regulatrs with the infrmatin they require t define an peratinal standard fr rulemaking.

13 Overall Risk Phase III Operatinal Risk Assessment Risks in the technlgy-assisted case can be lsely gruped int three categries: Risks assciated with technlgy nt functining as designed Risks assciated with imprper use f the technlgy (human factrs) Risks inherent in the envirnment Already we see the need fr cllabratin amngst researchers! + Similarly t the ORA fr the EVLOS case, we will ideally define a series f perfrmance-based requirements ANY assistive technlgy wuld have t meet. + Envirnmental risk is related t cnditins that feed int the risk mdel. Fr example: + Lw altitude traffic patterns + Ppulatin density

14 Technlgy Risk Phase III Operatinal Risk Assessment Risks assciated with the technlgy nt functining as designed (sftware & hardware):

15 Technlgy Risk Phase III Operatinal Risk Assessment Risks assciated with the technlgy nt functining as designed (sftware & hardware): + Frm a map f the prcess we can begin t itemize the ptential hazards at each prcess step. + One specific set f hazards is related t GPS & cellular service at altitude: Prcess Step Transmit telemetry until peratin cmplete Hazard i. Device experiences mechanical r electrical failure ii. GPS issues: GPS service becmes unavailable r intermittently available during peratin, invalid GPS string (imprperly frmed), incrrect GPS data (prperly frmed string) iii. Cellular issues: N cellular service, pr cellular receptin due t lcal cnditins, intermittent cnnectivity due t LTE handver issues + Cellular service (and LTE in particular) is ne f the bigger pen questins: + Lag? + Intermittency? + Hw much f either f these is acceptable? Risk mdels need t infrm the initial answer here.

16 Human Factrs Risk Phase III Operatinal Risk Assessment Risks assciated with imprper use f the technlgy: + Imprtant t ask questins abut hw and when infrmatin shuld be displayed t the peratr:

17 Human Factrs Risk Phase III Operatinal Risk Assessment Risks assciated with imprper use f the technlgy: + MITRE will evaluate a set f prpsed assistive technlgy displays via Human-inthe-Lp (HITL) simulatin, fllwed by field experiment:

18 Human Factrs Risk Phase III Operatinal Risk Assessment Risks assciated with imprper use f the technlgy: + MITRE will evaluate a set f prpsed assistive technlgy displays via Human-inthe-Lp (HITL) simulatin, fllwed by field experiment: Intruder w/ Amber Highlight Relative altitude and distance Blcked Headings ( Cne f Death ) Altitude Meter: Blue dt is yur AGL altitude Amber band is altitude blcked by intruder Well Clear Zne

19 Risk Mdels and Real Data: Part Deux Simulatin Field Data Histrical This prcess is nn-trivial. Real data is messy (especially if humans are part f the study). It s nt just science it s plitics. Different CAAs will have different pinins abut which variables are imprtant. We dn t exactly have a target risk. We were able t start measuring withut it, but cming up with this limit will be imprtant sn. If cmputatinal and research cmplexity nw leads t regulatry and peratinal simplicity later, it s wrth it. But let s get all the help we can.

20 Sme Views FROM Industry Technlgy is mving rapidly. Gathering flight heritage n rare events t assess risk is prblematic. S we need variables we can measure ver shrter cycles t assess the risk cnditinal prbability based risk mdels are a gd way t determine what thse variables shuld be. Mitigatins need t be palatable. That may ALSO depend n the rate f a particular risk ccurring. If the mitigatin results in ttal lss f the UAS, whether r nt that is acceptable depends n hw ften that wuld be likely t ccur. If cmputatinal and research cmplexity nw leads t regulatry and peratinal simplicity later, it s wrth it. But let s get all the help we can.

21 Sme Views FOR Industry Is yur technlgy is mving t rapidly? Or in the wrng directin? Adding features desn t help (and can vercmplicate the safety case) unless yu re actually slving a key prblem at this pint. Risk mdelling can help identify thse blckers. We are ALL in a data-starved envirnment right nw. We did it that ne time and nbdy died is nt a sufficient evaluatin f lng term risk. There are pprtunities fr participatin fr all sizes f industrial partner (i.e. ASSURE) If cmputatinal and research cmplexity nw leads t regulatry and peratinal simplicity later, it s wrth it. But let s get all the help we can.

22 Thank Yu Ally Fergusn