Report on Phase 2 Causal Modeling for Schiphol Airport Oswaldo Morales, Roger Cooke, Dorota Kurowicka EWI, TU Delft, April 25, 2006 Introduction. This document reports on the activities of EWI during the second phase of the CATS project; that is 3 months between the end January and the end of April 2006. The project kicked of in July 2005, although preparatory work was initiated in the spring of the same year. The previous report [3] described the developments of mathematical models for for Distribution-Free Continuous /Discrete Bayesian Belief Nets DFCDBBNs, the development of elicitation protocols for the dependence information required as input for such models, the development of software implementation for DFCDBBNs and a prototype application for Controlled Flight into Terrain (CFIT). In this report we present: New results on the prototype application for Controlled Flight into Terrain (CFIT). Development of software implementation for Distribution-Free Continuous /Discrete Bayesian Belief Nets. (DFCDBBNs) Advances in the human error probability model. The CFIT Model. The CFIT model was first presented in [2]. The prototype version developed with the methods described in [1] was introduced in [3]. Figure 1 presents the discrete BBN version of the prototype CIFIT model. Each node represents the marginal distribution of the variables in the model. The expectation of each variable (together with standard deviations after the ± sign) is shown at the bottom of the node. The variables entered in the model are: 1. Fuel Weight: Measured in kilograms and is the remaining fuel at arrival based on data for 172 flights of a Boeing 737 at Schiphol airport. 2. Visibility: Measured in meters and is based on a sample of 27 million observations over Europe.
Figure 1. CFIT Prototype Model Figure 2. CFIT Prototype Model (Conditioned in Separation in air (0, 2.5) Nm)
3. Crew Alertness: Measured by the Stanford Sleepiness Scale in an increasing scale from 1 to 7, where 1signifies feeling active and vital; wide awake and 7 stands for almost in reverie; sleep onset soon; struggle to remain awake the distribution used for this study comes from field studies by the Aviation Medicine Group of TNO Human Factors in 1,295 flights. 4. Speed Deviation at 500 ft: Deviation from bug at 500 ft. The data comes from 13,753 approaches of a major European airline. 5. Mean Cross Wind: Usually expressed as a combination of speed (in knots) and direction (compass course) of the wind at any direction not favorable for the aircraft, the cross wind distribution comes from 380,000 takeoffs and landings conducted on three large European airports. 6. Separation in Air: Longitudinal distance (in nautical miles) between the landing aircraft and the preceding aircraft in the approach path. The distribution was retrieved from a sample size of 2,382 landings at Schiphol airport. 7. Missed Approach Execution: Number of missed Approach Execution per 100,000 flights at Schiphol airport. The Expectation of the number of missed approach executions divided by 100,000 would be an estimate of the probability of executing a missed approach maneuver. 8. Condition for missed approach: This node is deterministic in the sense that assigns probability one to its state yes when either one of the following conditions holds alone or together with any other: a. Visibility < 360 meters. b. Speed Dev. at 500 ft (-5, 10) knots. c. Mean Cross wind > 20 knots. d. Separation in air < 2.5 Nautical miles. An accident situation may happen when a missed approach is not executed when the conditions for the missed approach are present. Hence the interest is in observing the probability of NOT executing a missed approach when the conditions are present. Figure 3 presents the increase in risk according to evidence entered in the model described in Figure 1. Scenarios A and B describe situations where there are NO reasons to execute a missed approach the difference being the confidence with which one may say there are no reasons to perform a turn around. In A all variables are at their safest level whereas in B they are close to give indication that a turn around should be performed. The reader may observe that the difference in the estimated probability of not executing a missed approach is of approximately 11%. Situations C and D present the same situations as A and B respectively except that the condition for missed approach is obtained by observing a separation in air (0, 2.5) nautical miles. The estimates of the probability of not executing a missed approach maneuver in these situations are rather high ( 0.78 and 0.68 for C and D respectively), however the uncertainty over the estimates is also rather high. For C the estimate can be as low as 0.29 and for D may be also as low as 0.20. The model allows for analyses similar to the one discussed previously and in Figure 2. The probability of NOT executing a missed approach when conditions are present would be input for the fault trees developed by DNV.
1.000 0.900 0.800 0.700 0.600 0.500 0.400 0.300 0.200 1.0000 0.9718 1.0000 1.0000 1.0000 0.8612 0.7829 0.7442 0.6793 0.4166 0.2888 0.1991 0.100 0.000 A B C D Figure 3. Probability of NOT Executing a Missed Approach Maneuver (and 2σ bounds). A: No Condition Separation in Air (14,100) Mean Cross Wind (0,2.5) Speed Dev at 500 ft (0,5) Visibility > 40000 B: No Condition Separation in Air (2.5,4) Mean Cross Wind (17.5,20) Speed Dev at 500 ft (5, 10) Visibility (360,10000) Current Status of Uninet. C: Condition Separation in Air (0,2.5) Mean Cross Wind (0,2.5) Speed Dev at 500 ft (0,5) Visibility >40000 D: Condition Separation in Air (0, 2.5) Mean Cross Wind (17.5, 20) Speed Dev at 500 ft (5, 10) Visibility (360, 10000) The previous steps for working with UNINET are summarized next, for more information see [3]: 1. Specify in UNICORN the variables that will enter the BBN later. 2. From UNICORN go to the Run menu and launch UNINET. 3. Create in UNINET the BBN of interest. 4. Run the model and create the sample file required by NETICA. 5. Prepare a discrete version of the model in NETICA. 6. Import the sample file created in step 4 to populate the probability tables required in 5. Steps 5 and 6 are avoided in the present version of UNINET. As shown in Figure 4 once the BBN of interest has been sampled, the user may define the number of intervals each node should have. This applies for the case of equal length intervals and intervals defined by quintiles (the length of the interval is defined such that each one will have equal mass). User defined intervals are also supported in the current version of UNINET (Figure 5)
Figure 4. UNINET Discretization Panel. Figure 5. UNINET Discretization Panel (Custom intervals). Once the discretization has been chosen for each variable, the next step is to Run Export to NETICA (Figure 6) and the program will automatically launch NETICA incorporating the dependence information specified in UNINET into the probability tables. A model like the one in Figure 1 will be created where analysis such as the one presented in the previous section can be performed.
Figure 6. Exporting the BBN from UNINET to NETICA. Advances in the Human Error Probability Model. Weekly meetings have been held since March 17 between NLR, TBM-TU Delft and EWI-TU Delft with the purpose of building up a model for Human Error Probability. This model will be later quantified using the tools described in [1, 3]. The first version of the model is presented in figure 7 below. Figure 7. First Version of the Model for Human Error Probability.
The variables entering the model are: 1. Human Error Probability: As in the prototype application for CIFIT the Human Error probability per flight phase can be determined as the expectation of: # Human erros # Flight phases 2. Situation: A list of possible events in which an accident could happen. Possibly in an ordinal scale. 3. Crew Composition: Variable taking into account the homogeneity or heterogeneity of the pilot s and co-pilot s backgrounds. 4. Experience: Based on the difference between Captain s and First Officer s experience. 5. Mother Tongue: Same mother tongue between Pilots Yes/No. 6. Crew Skills: To be defined. 7. Technical Skills. To be defined. 8. Non-Technical Skills: To be defined. 9. Experience: Total number of hours flown since the pilot s license obtaining. The number of airliner types is limited today and in the near future and the difference between the types is relatively small. Therefore adding the experience per a/c type as a measure for experience would only make the model unnecessarily complex. 10. Training: # of days since last training/recurrence check. From the human factors literature this is supposed to be the most determining factor and not for example the total number of training hours received. 11. Fatigue: Stanford Sleepiness Scale. See crew alertness in the previous section. 12. Weather: Colour scale (ordinal) representing weather conditions. Based on: a. Visibility (m). b. Wind [speed (m/s) and direction (degrees) relative to a/c or wind component] c. Rain (mm/unit Time) 13. Safety culture. To be defined. Future work. Future work from the EWI-TU Delft group includes: 1. Conditioning with UNINET. This will be done as an alternative for doing the conditioning in NETICA. 1. Online software tool to support the elicitation of the dependence information required by the BBNs. Possibly included as a feature in UNINET. 2. Finalize and Quantify the BBN for Human error probability. Further work to be performed as agreed on the technical meetings.
Basic bibliography. [1] Hanea A.M. et al., Hybrid Methods for Quantifying and Analyzing Bayesian Belief Nets, Proceedings of the 2005 ENBIS5 Conference, 2005. [2] Roelen A.L.C. et al., Causal Modelling of Air Safety. Demonstration Model. National Aerospace Laboratory, NLR-CR-2002-662, December 2002. [3] Cooke R.M. et. al, Report on Phase 1 Causal Modeling for Schiphol Airport, in Causal Model for Air Transport Safety (CATS). Third Interim report. 4 February 2006. Risk Centre TU Delft.