George Cleland 6 th December 2011

Transcription

1 Safety Case Templates and Safety Case Review Templates George Cleland 6 th December 2011 Adelard, Exmouth House, 3-11 Pine Street, London EC1R 0JH +44 (0)

2 DOSG Safety Case Guidance Project recently completed for the Defence Ordnance Safety Group Joint project between Adelard and DOSG Equipment safety case template publicly available ASCE version HTML version Original in Claim-Argument-Evidence Also conversion to GSN 2

3 Project Context UK defence sector is (was) self regulating Normally requires the use of an Independent safety Auditor/ assessor Provides Duty Holder with impartial advice and reviews safety management except for Nuclear propulsion and weapons: DNSR (also NII) Ordnance, Munitions and Explosives (OME) DOSG All changing now post Haddon-Cave MAA DSEO 3

4 DOSG Regulatory Function PTs (Project Teams) procure OME from suppliers against DefStan PTs run internal safety management systems against POSMS (generic safety) and JSP 520 (OME safety) DOSG Advisors provide OME safety advice to PTs PTs are required to present a Safety Case Report to an Ordnance Safety Review Panel JSP 520 lays down a required format for this report 4

5 The problem PT should be maintaining a live Safety Case covering all aspects of safety The OSRP Safety Case Report should be a relevant projection of the OME aspects of safety DOSG wanted to provide guidance on the whole safety case lifecycle Developing an overall safety case Producing the OSRP Report 5

6 Safety Case Guidance/Examples Lots of guidance on safety case reports DefStan 00-56, POSMS, JSP 520, MAA ALWR Little (no) guidance on Safety Case approaches Few available example cases DOSG tasked Adelard to develop this guidance and a template Safety Case Also to take forwards earlier work on Safety Case Review 6

7 Safety Case Report Templates Examples JSP OME POSMS MAA Air Launched Weapon Release Standards encoded as ASCE template Guidance from standard included Replace guidance with content to create report ASCE DNR Plugins provide change tracking and dynamic re-mapping of content ASCE Export Functions support instant publication into Word, PDF or HTML 7

8 JSP 520 Ordnance Safety Review Panel Submission 8

9 Ch 1 Executive summary POSMS Ch 2 Summary of Sys... Ch 7.1 Operational Envelopes Ch 7.2 Limitations on use/ operatio... Ch 7 Operational Information Ch 7.4 Main risk areas Ch 7.4 Balancing operational imperative against safety risk Ch 7.5 Operating and maintena... Ch 3 Assumptions Ch 8 ISA Report Ch 4 Progress against the Programme A 1 Hazard Log Summary Ch 9 Conclusions... A.4 Hazardous materials Ch 5.1 Safety definition Ch 5 Meeting safety Ch 5.2 Safety satisfaction A.2 Safety case argum... Appendices A.5 Handling Procedures A.3 Calculations A.6 Safety certificates Ch 6 Emergency/ Contingency Arrangements Lifecycle Concept/Initial Gate Lifecycle Assessment/ Main Gate Lifecycle Demonstration/ Manufacture and Trials Lifecycle In Service Lifecycle Disposal 9

10 PRELIMINARY PAGES MAA Air Launcher Weapons Release OTHER ReadMe B.1 Weapon Description and Associated Operating B.2 Weapon Interface Control Documentation B.3 Weapon Performance/ Effectiveness B.4 Weapon Delivery Envelope PART A AIRWORTHINESS & DOCUMENT MANAGEMENT PART B ALW DESIGN PERFORMANCE AND OPERATING LIMITATIONS A.1 Release Statement A.2 Introduction A.3 Description A.4 Weapon Service Life A.5 Statement of Operating Intent and Usage A.6 Engineering Actions from Weapon Safety Case A.7 Weapon Configuration A.8 Related Documents A.9 Weapon and Component Security Classification A.10 Incident and Fault Reporting A.7.4 Other equipment A.8.1 Weapon Document Set A.2.1 Purpose A.2.2 Structure A.2.3 Ammendment A.2.3 Responsibilities A.7.1 Design Standard A.7.2 Designer Modifications A.7.3 Service Modifications A.8.2 Other Documentation A Weapon Software Standard A A A A B.5 Aircraft Self Damage B.6 Explosive Hazard Classification B.7 Environmental Conditions B.8 TEACASE Limitations B.9 Electromagnetic Compatibility C.1 Manufacture To Target/DisposAl Sequence C..2 Maintenance and In-Service Testing C.3 Weapon Loading and Unloading C.4 Special to Type Tools Test and Support Equipment C.5 Packaging Containers C.6 Storage Conditions C.7 Transportation C.8 Weapon Embarkation PART C SYSTEM LIMITATIONS & CONSTRAINTS PART D AIRCRAFT INTEGRATION, LIMITATIONS & CONSTRAINTS PART E TEMPORARY INFORMATION D.1 Aircraft Type & Mark D.2 Configuration Requirements D.3 {Aircraft Identifier} Interface Control Document D.4 {Aircraft Identifier} Limitations and Constraints D.5 {Aircraft Identifier} Certificates of Design C.10 Laser Safety C.11 Emergency and Contingency Arrangements C.11 Disposal PART F SERVICE DERIVED INFORMATION G1 Information Audit G2 Specialist Advice PART G AUDIT TRAIL G.3 Safety Case 10 G4 Integrated Test Evaluation and Acceptance Plan G.5 Non-Compliance/Further Work

11 Introduction JSP520 (Issue 3.0) Part 1 Safety Cases and the Claims Argument Evidence approach Definitions History Guidance/Regulation JSP520 + Leaflets AOP 15 DefStan POSMS System overview Certification / approvals framework The equipment is safe for its application in its environment of use Current status of safety case Operating manuals Assumptions JSP520 (Issue 3.0) Part 2, A1 System Requirements Document Prerequisites JSP520 (Issue 3.0) Part 2, B1 JSP520 (Issue 3.0) Part 2, B2 JSP520 (Issue 3.0) Part 2, B3 JSP520 (Issue 3.0) Part 2, C1 JSP520 (Issue 3.0) Part 2, C2 JSP520 (Issue 3.0) Part 2, C3 JSP520 (Issue 3.0) Part 2, C4 JSP520 (Issue 3.0) Part 2, C5 JSP520 (Issue 3.0) Part 2, C6 JSP520 (Issue 3.0) Part 2, D1 JSP520 (Issue 3.0) Part 2, E1 JSP520 (Issue 3.0) Part 2, E2 JSP520 (Issue 3.0) Part 2, E3 The safety are met (the equipment is Safe and Suitable for Service) AOP-15 (Edition 3) Safety are adequately defined The equipment continues to be safe in service (Issue 4) Part (Issue 4) Part 2 POSMS Safety Case Review Criteria The equipment is safe through life Risk from equipment is tolerable and ALARP Other safety met Safe disposal Safety management defined Prescriptive safety identified by adequate reivew of legislation, standards and policy Safety criteria adequately defined Integration (if any) identified Risk from equipment is tolerable Risk from equipment has been reduced SFAIRP by design All reasonably practicable operational controls have been identified and implemented (or communicated to users) Compliance with applicable legislation, policy and standards Platform integration met Safety management are met In-service safety is adequately monitored and managed Equipment is safe under change and maintenance Disposal plan Overall safety management system Register of applicable legislation, policy and standards Review the applicability of identified legislation / standards / policy Broadly acceptable and tolerable risk criteria thresholds defined ALARP risk criteria Integration All reasonably practicable design controls have been identified and implemented Equipment design follows all relevant good practice Documentation, training and OME Safety Instructions Safety management plan and SMS Periodic review of Incident validity of assumptions reporting and and adherence to evaluation prerequisites Periodic review of actual safety performance Unplanned change management is safe Planned change management is safe All hazards identified and risk assessment made All reasonably practicable equipment level design controls have been identified All identified equipment level design controls have been implemented All relevant good design practice has been identified The design follows the identified good practice Change management procedures Maintenance schedule Compliance matrix Preliminary hazard identification and analysis report System hazard analysis and risk assessment System hazard log Derived safety (in the SRD) V&V evidence Safety Case Template A 1 Hazard Log Summary A.2 Safety case argument structure A.3 Calculations Executive Summary System description Safety Risk level category Safety Management Plan Safety assessment Emergency arrangements Conclusions Appendices A.7 References Section 1 Capability Section 1 Safety User Requirements Section 1 Risk Management Section 1 Risk Management A.4 Hazardous materials A.5 Handling Procedures A.6 Safety certificates Section 2 Predicted Service Environment Section 2 Safety System Requirements Section 2 Safety Programme Section 2 Design Assessment Section 3 Lifecycle sequence Section *from JSP520 issue 1 OME Safety Criteria Section 3 Review and Audit Section 3 Safety Trials Section 4 System definition Section *from POSMS Assumptions Section 4 Responsibilities Section 4 Range and Laser Safety Section 5 Environmental Management and Assessment 11 OME Safety Case Report

12 Embedded links connect each element of the Safety Case Template to relevant regulation Linking to Guidance If we open the Risk from equipment has been reduced SFAIRP by design node this show links to relevant guidance on how this argument fragment should be elaborated Clicking these links navigates to the relevant guidance Embedded links show how elements of the Safety Case maps out to populate the OME SCR 12

13 Top level template decomposition System overview Operating environment Certification / approvals framework(s) System Requirements Document The equipment is safe for its application in its environment of use Current status of safety case Prerequisites and limitations Operating manuals Assumptions [>] C1 Safety are adequately defined [>] C2 The safety are met (the equipment is Safe and Suitable for Service) [>] C3 The equipment continues to be safe in service 13

14 Requirements definition leg [>] Safety are adequately defined Safety management defined Prescriptive safety identified by adequate reivew of legislation, standards and policy Safety criteria adequately defined Integration (if any) identified Overall safety management system Register of applicable legislation, policy and standards Review the applicability of identified legislation / standards / policy Broadly acceptable and tolerable risk criteria thresholds defined ALARP risk criteria Integration 14

15 Equipment safety leg [>] The safety are met (the equipment is Safe and Suitable for Service) Risk from equipment is tolerable and ALARP Other safety met Risk from equipment is tolerable Risk from equipment has been reduced SFAIRP by design All reasonably practicable operational controls have been identified and implemented (or communicated to users) Compliance with applicable legislation, policy and standards Safety management are met All reasonably practicable design controls have been identified and implemented Equipment design follows all relevant good practice Documentation, training and OME Safety Instructions Platform integration met Safety management plan and SMS All hazards identified and risk assessment made All reasonably practicable equipment level design controls have been identified All identified equipment level design controls have been implemented All relevant good design practice has been identified The design follows the identified good practice Compliance matrix Preliminary hazard identification and analysis report System hazard analysis and risk assessment Derived safety (in the SRD) V&V evidence System hazard log 15

16 Through life safety leg [>] The equipment continues to be safe in service The equipment is safe through life Safe disposal In-service safety is adequately monitored and managed Equipment is safe under change and maintenance Disposal plan Periodic review of validity of assumptions and adherence to prerequisites Incident reporting and evaluation Unplanned change management is safe Planned change management is safe Periodic review of actual safety performance 16 Change management procedures Maintenance schedule

17 In Practice The template as delivers is a guidance document To use in practice Break out the template part and the report part as separate ASCE documents (must copy into same Schema, ASCAD 1.2) Follow the guidance to elaborate the Safety Case Use ASCE Node Mapping Plugin to pull content from the Safety Case into the Safety Case report Repeat for POSMS and ALWR 17

18 Populating the OME Safety Case Report The SCR does not contain any native content All content is mapped in from the Safety Case Mapped content is tracked and changes recorded One-click export creates a MS Word SCR document in a DE&S template Word macros transform content according the Defence Writing Style and flatten ASCE DNR tables 18

19 Case Studies CRV7 IGMR PT, Bristol Aerospace Brimstone 2 SAM PT 19

20 Roles and responsibilities Safety and environment management defined Overall safety management Register of applicable legislation, policy and standards Prescriptive safety and environment identified by adequate review of legislation, standards and policy Review the applicability of identified legislation / standards / policy Safety and environmental are adequately defined Safety and environmental criteria adequately defined Broadly acceptable and tolerable risk criteria thresholds defined ALARP risk criteria Environmental Case Documents Integration (if any) identified Integration ALARP statements Risk from equipment is tolerable All hazards identified and risk assessment Preliminary hazard identification and analysis report made System hazard analysis and risk assessment System overview All reasonably practicable equipment level design controls have been identified System hazard log Derived safety (in the SRD) Certification / approvals framework All reasonably practicable design controls have been identified and implemented System Requirements Document All identified equipment level design controls have been implemented V&V evidence Risk from equipment is tolerable and ALARP Risk from equipment has been reduced SFAIRP by design All relevant good design practice has been identified Equipment design follows all relevant good practice The equipment is safe for its application in its environment of use The safety are met (the equipment is Safe and Suitable for The design follows the identified good practice Service) All reasonably practicable operational controls have been identified and implemented (or communicated to users) Documentation, training and OME Safety Instructions Current status of safety case Compliance matrix Operating manuals Assumptions Compliance with applicable legislation, policy and standards Limitations and prerequisites Other safety met Safety and environmental management are met Safety management plan and SMS Operating environment Integration (if any) are met Integration Evidence Periodic review of validity of assumptions and adherence to prerequisites In-service safety is adequately monitored and managed Periodic review of actual safety performance Review and audit Incident reporting and evaluation The equipment is safe through life Unplanned change management is safe Change management procedures The equipment continues to be safe in service Equipment is safe under change and maintenance Planned change management is safe Maintenance schedule Safe disposal Disposal plan CRV7 Case Study Going from this: to a production quality OME SCR for OSRP submission takes about three minutes 20

21 Brimstone 2 Ongoing 21

22 Safety Case Review Initial Work ISA on a major programme Many safety Case to review Equipment Many variants Installation Many platforms Operational trials Tight timescales Usually decreasing Developed a Safety Case Review Production Line Based on DefStan issue 2 Original concept by Tim Clement 22

23 Benefits Consistency Efficiency Objective Coverage Constructive 23

24 Current work Update original with current standards DefStan issue 4 JSP 520 issue 3 JSP 454 issue 5 Generic version OME version 24

25 Red nodes are used to write a Orange nodes report on the contain OME aspect of the review guidance safety case linked Blue nodes contain generic to The the guidance Full Template review guidance ReadMe System description System description enables full understanding of how the system works OTHER References Predicted service environment for OME References Summary Introduction to the safety case review OTHER Definitions Safety OME specific legislation, policy and standards Safety have been identified and validated **DELETE** Additional subclaims OTHER OTHER Safety case relevance Safety case maintenance OTHER Compliance evidence Test and assessment evidence has been generated and is adequate System complies with prescriptive safety XYZ safety case supports current predicted use of system Current status of safety case is clearly defined Safety case status Emergency COSHH arrangements Hazard log contains COSHH relevant occupational assessment hazards Adequate hazard identification has been conducted and the hazard log contains relevant hazards Hazard log contains relevant hazards of functional failure Hazard log Pink nodes are a structuring device only and have no content All assumptions made in safety case are stated Recommendations Environmental hazards Limitations PHI Other hazard analyses OHHA, OSHA and ZHA Field experience Hazard log Identified mitigations are in place OME risk level category PHI Other hazard analyses OHHA, OSHA and ZHA Field experience Explicit claim for risk levels Safety case supports claim for tolerable accident rates Review Explicit ALARP claim System risks have been made ALARP Review Test evidence Other evidence Safety criteria Fully analysed hazards have tolerable accident rates Partly analysed hazards have reasonable occurrence rate estimates Development follows good practice Design controls causes of specific hazards Environmental testing Safety criteria Hazard log Disposal considerations Functional hazards OME good practice System provides defence System mitigates against foreseeable errors inherent hazards elsewhere in its operating environment OME design controls Identified good practice Example compliance matrix for STANAG4187 Example compliance matrix for STANAG4368 Example explosive qualification forms Inherent hazards Protection from external faults/errors 25

26 Safety have been identified and Guidance validated System description enables full understanding of how the system works Safety case supports current predicted use of system Safety case relevance ReadMe OTHER References OTHER Definitions OTHER System complies with prescriptive safety Compliance evidence System description Safety Test and assessment evidence has been generated and is adequate Predicted service environment for OME OME specific legislation, policy and standards Safety have been identified and validated System complies with prescriptive safety System description enables full understanding of how the system works **DELETE** Additional subclaims XYZ safety case supports current predicted use of system OTHER References Summary Current status of safety case is clearly defined OTHER Introduction to the safety case review Safety case relevance Current status of safety case is clearly defined Safety case maintenance Safety case status Emergency COSHH arrangements Hazard log contains COSHH relevant occupational assessment hazards Adequate hazard identification has been conducted and the hazard log contains relevant hazards Hazard log contains relevant hazards of functional failure Hazard log Adequate hazard identification has been conducted and the hazard log contains relevant hazards All assumptions made in safety case are stated All assumptions made in safety case are stated Recommendations Environmental hazards Limitations PHI Other hazard analyses OHHA, OSHA and ZHA Field experience Hazard log Identified mitigations are in place OME risk level category PHI Other hazard analyses OHHA, OSHA and ZHA Field experience Safety case supports Explicit claim for claim for tolerable accident Review risk levels rates Fully analysed hazards Partly analysed Safety criteria have tolerable accident hazards have rates reasonable occurrence rate estimates System risks have been made Explicit ALARP claim Review ALARP Design controls causes of specific hazards Development follows good practice Test evidence Environmental testing Other evidence Identified mitigations are in place Safety case supports claim for tolerable accident rates Safety criteria Hazard log Disposal considerations Functional hazards Example compliance matrix for STANAG4187 OME good practice Example compliance matrix for STANAG4368 System provides defence System mitigates against foreseeable errors inherent hazards elsewhere in its operating environment OME design controls Identified good practice Inherent hazards Protection from Example external faults/errors explosive qualification forms System risks have been made ALARP 26

27 Using the Template Read the Safety Case Review against each of the guidance nodes Write an assessment against the guidance in the linked red node When complete use 1-click export to produce a narrative report 27

28 Export path This gives ASCE the order node content is exported Start at the green node Follow the heavy black line over the structure Ending on the red node This produces a report which contains the guidance on each aspect followed by your report against the guidance 28

29 The Disk What s on it The SC Template JSP 520 report The SC Review Template What s not ALW Release Weapons Legislation Database It s not quite finished Please don t distribute further Final version will be available in the new year Contact me for a copy 29

30 The Team Adelard Fan Ye Jack Crawford George Cleland Tim Clement Luke Emmet DOSG Ian Barnes Martin Reglar Bryn Thomas Phil Morris Bristol Aerospace Tracy Hunt Phil Bryant MoD Damian James, IGMR PT Clive Walder, SAM PT Pete Burrell, SAM PT 30

31 The Future Extend the template for environmental Develop an Approved Code of Practice for the Weapons Operating Centre Support development of standardised risk metrics for the WOC 31

32 The end Questions? 32