Failure Mode and Effect Analysis (FMEA) for a DMLC Tracking System Amit Sawant 1, Sonja Dieterich 1, Michelle Svatos 2 and Paul Keall 1 1 Stanford University, Stanford, CA 2 Varian Medical Systems, Palo Alto, CA
Developing Quality Assurance Guidelines for DMLC Tracking In order to translate DMLC tracking into the clinic, we need to develop and recommend reliable QA guidelines for tracking-based delivery Our goal is to maximize system diagnosis while minimizing the additional QA burden on the clinical staff Therefore, instead of recommending prescriptive QA procedures (i.e., test everything), we will develop a QA protocol based on a failure mode and effect analysis (FMEA) of a DMLC tracking system
What Is FMEA? FMEA is a quality management technique for risk assessment and safety improvement of complex processes The FMEA methodology charts a process identifies potential modes and causes of failure anticipates the downstream effect of each failure FMEA is the recommended framework for evaluating the needs of modern radiotherapy QA in the forthcoming AAPM TG100
How is FMEA Performed? Each functional step in a process is mapped and a process tree is created Failure modes are identified - a failure is defined as an instance where an expected system response (e.g., beam hold) does not occur Each failure is assigned a risk probability number (RPN) which is defined as RPN = O S D, where O = probability of occurrence of the failure S = Severity of effect D = Detectibility of the failure during operation of the process
DMLC Tracking Process Tree
Only the steps within the shaded large blue region are tracking-specific. All other steps are common to any motion-managed radiotherapy e.g., gating We are only interested in failure modes of these tracking-specific steps Failure of other steps (i.e., those not in our box) should be assessed and rectified via pre-established QA procedures. Example 1. Position monitoring (PM) system is not accurate: Identified and rectified during the PM system QA. Example 2. One or more MLC leaves out of tolerance but beam-hold is not asserted: Identified and rectified during standard MLC QA.
Scale for O, S, D Values Value Probability of Occurrence [O] Severity of Effect [S] Source: National Cancer Institute Common Terminology Criteria for Adverse Events v4.0 (2009) Detectability [D] (Probability of Failure to Detect) 1 Very unlikely (< 0.01%) No adverse event (AE) Very unlikely (i.e., always detected) (< 0.01%) 2 3 Low probability (0.02% 0.04%) 4 5 Some probability (0.05% 0.4%) 6 7 Moderate probability (0.5% 1%) 8 9 High probability (2% 5%) 10 Certain failure (> 5%) Grade 1 Mild; asymptomatic or mild symptoms; clinical or diagnostic observations only; intervention not Indicated Grade 2 Moderate; minimal, local or noninvasive intervention indicated; limiting age-appropriate instrumental activities of daily living (ADL) 1 Grade 3 Severe or medically significant but not immediately life-threatening; hospitalization or prolongation of hospitalization indicated; disabling; limiting self-care ADL 2 Grade 4 Life threatening consequences. Urgent intervention indicated. Grade 5 Patient death related to AE Low probability (0.2% 0.5%) Some probability (1% 2%) Moderate probability (5% 10%) High probability (15% 20%) Certain failure (impossible to detect) (> 20%) Activities of Daily Living (ADL) 1 Instrumental ADL: preparing meals, shopping for groceries or clothes, using the telephone, managing money, etc. 2 Self-care ADL: bathing, dressing and undressing, feeding self, using the toilet, taking medications, and not bedridden.
Example O, S, D Scoring (non-tracking) Process Failure Mode Causes of Failure Effect of Failure Probability of Occurrence [O] Severity of Effect [S] Detectability [D] Risk Prob Number RPN = O S D 1. Patient localization Patient moves > 50mm from planned position during beam on but motion is not detected Absence/failure of real-time position monitoring system Dose error >15% 5 8 8 320 2. Dose delivery Beam does not turn off after a field is delivered Multiple backup systems for beam-off fail Dose error >60% 2 10 1 20
FMEA for MLC Tracking The next two slides show a table for anticipated failure modes. Table 1 is for conventionally fractionated treatments. Table 2 is for hypofractionated treatments (e.g., SBRT) In each case, failure is quantified in terms of estimated dosimetric error. Dose error is defined as underdose to target and/or overdose to normal tissue Please enter your best estimates for O, S and D according to the scale that was provided in Slide#7
FMEA Conventional Fractionation Please enter your estimates of O, S and D Process Step Failure Mode Possible causes of Failure Effect of Failure (%Dose Error) O S D 1. Position Monitoring (PM) system estimates real-time target position 2. Tracking system receives realtime target position and recalculates MLC leaf positions as a function of dose fraction and target position Target moves outside spatial tolerance set by PM system, but beam hold is not asserted i. communication with PM system is lost, i.e., target position is no longer current but beam hold not asserted ii. error in coordinate system conversion iii. optimal leaf-fitting not achieved Beam hold not sent by monitoring system to tracking system i. PM system failure ii. data transfer cable(s) physically disconnected i. system installation error ii. hardware / software >15% >30% >60% changes Complex motion + highly modulated field > 10% 3. Tracking system checks if fluence map is completely within field 4. MLC controller actuates leaf motion. If leaves are within tolerance, linac delivers dose, else beam is held off. Other Beam hold not asserted when fluence map is partially or completely under one or more jaws i. system latency outside expected range ii. too many beam holds. Efficiency drops below desired threshold i. Software crash or failure ii. jaws outside tolerance i. hardware and/or software changes ii. changes in network connection speed MLC leaves cannot keep up with target motion > 20% > 5% Efficiency < 70%
FMEA Hypofractionation Please enter your estimates of O, S and D Process Step Failure Mode Possible causes of Failure Effect of Failure (%Dose Error) O S D 1. Position Monitoring (PM) system estimates real-time target position 2. Tracking system receives realtime target position and recalculates MLC leaf positions as a function of dose fraction and target position Target moves outside spatial tolerance set by PM system, but beam hold is not asserted i. communication with PM system is lost, i.e., target position is no longer current but beam hold not asserted ii. error in coordinate system conversion iii. optimal leaf-fitting not achieved Beam hold not sent by monitoring system to tracking system i. PM system failure ii. data transfer cable(s) physically disconnected i. system installation error ii. hardware / software >15% >30% >60% changes Complex motion + highly modulated field > 10% 3. Tracking system checks if fluence map is completely within field 4. MLC controller actuates leaf motion. If leaves are within tolerance, linac delivers dose, else beam is held off. Other Beam hold not asserted when fluence map is partially or completely under one or more jaws i. system latency outside expected range ii. too many beam holds. Efficiency drops below desired threshold i. Software crash or failure ii. jaws outside tolerance i. hardware and/or software changes ii. changes in network connection speed MLC leaves cannot keep up with target motion > 20% > 5% Efficiency < 70%
Thank You!