Evaluation of Three New Fit Test Protocols for Use with the TSI PortaCount by Roy T. McKay, Ph.D. Jeff Bradley, RRT Occupational Pulmonary Services Department of Environmental Health University of Cincinnati Cincinnati, OH 45267 Roy@DrMcKay.com
Purpose: To investigate three new fit testing protocols with shorter exercise times specifically designed for use with the TSI PortaCount. Methods: Fit testing software was modified to permit data acquisition at ten (10) second intervals during mask sampling. This allowed 10, 20, & 30-second fit factors to be compared to the OSHA 40-second fit factors as if they were different fit tests with identical donnings.
TSI PortaCount Plus Sampling Sequence Ambient Purge 4 sec Ambient Sample 5 sec Mask Purge 11 sec OSHA Accepted Protocol Mask Sample 40 sec
OSHA Protocol Ambient Purge 4 sec Ambient Sample 5 sec Protocol 1 Protocol 2 & 3 Mask Purge 11 sec Mask Sample 10 s 20 s Mask Sample 40 sec Excluded 30 s 20 s
Advantages of this method: Avoids problems associated with comparison studies using two different fit test methods. For example; New method fit factors were compared to the reference method fit factors during the same donning. Influence of previous exercise maneuvers was, therefore, eliminated. Utilized same sample probe location, size, flow, etc.
OSHA Protocol Protocol #1 Protocol #2 Protocol #3 levels (half/full) Individual exercise time (seconds)* In facepiece sampling time Normal breathing Deep breathing Turn head side to side Move head up and down Talking Grimace (15 seconds) Bending over Normal breathing Total test time (minutes:seconds) 100/500 60 40 7:15 100/500 30 10 3:45 100/500 40 20 4:55 200/1000 40 20 - - 3:35 Protocol #3 eliminates the first two less challenging exercises, thus putting an emphasis on the more challenging exercises.
Human subject approval was obtained from the University of Cincinnati s Institutional Review Board. Medical clearance was obtained using an online web-based service available through the University of Cincinnati.
Results were compared to ANSI Z88.10-2001 Acceptance Criteria for Evaluating New Fit Test Methods Minimum number of paired tests Minimum number of test subjects ANSI Criterion 100 25 Study Results 140 30 Minimum number of reference fit factors less than the required fit factor Make and model facepieces 50 Variety 57 < fit factor of 100* 91 < fit factor of 500 24 * After excluding 3 tests with reference fit factors within ±1 SD (otherwise 60 reference fit factors were below 100.
Number of Fit Tests Half Masks Full Face All Masks Reference Fit Factors < 100 Reference Fit Factors > 100 Reference Fit Factors < 500 Reference Fit Factors > 500 Total Number of Fit Tests 69 71 140 60 80 91 49
Distribution of Reference Fit Factors 100000.0 Reference Fit Factors 10000.0 1000.0 100.0 10.0 1.0 1 51 101 Test number (n = 140)
Test Sensitivity ANSI Z88.10 Sensitivity Requirement 0.95 Test sensitivity: The probability that the new fit test method will correctly identify an inadequately (ie., poorly) fitting respirator. In essence, it answers the following question: Of the reference method fit tests with unacceptable fits, what fraction also failed the new fit test method? Answer should be 95%. Sensitivity is sometimes called the βeta error.
Sensitivity = C (A + C) Reference Method Failed ed New Method ed Failed A C B D
Since each fit testing protocol was compared to reference pass/fail criteria of 100 and 500, there are six (6) separate determinations made in this study Protocol Reference /Fail Criterion Exercise Time (sec) Mask Sample Time Sensitivity Result 1 100 30 10 2 100 40 20 3 100 40 20 1 500 30 10 2 500 40 20 3 500 40 20
20 s Mask Sample 40 s Exercise Time 8 Exercises: Total Test Time = 4:55 Fail Criterion: 100 Reference Method 100 New Method Failed Protocol #2 Table IVb Reference Method ed New Method Protocol #2 ed Failed 2 55 78 0 80 55 57 78 135 Sensitivity = 0.97
20 s Mask Sample 40 s Exercise Time 8 Exercises: Total Test Time = 4:55 Fail Criterion: 100 Reference Method 100 New Method Protocol #2 Table IVb ANSI Requirement Study Results or Fail Sensitivity 0.95 0.97 Predictive Value of a 0.95 0.98 Specificity > 0.50 1.00 Predictive Value of a Fail > 0.50 1.00 Kappa Statistic > 0.70 0.97
20 s Mask Sample 40 s Exercise Time 6 Exercises: Total Test Time = 3:35 Fail Criterion: 100 Reference Method 200 New Method Failed Protocol #3 Table IVc: Reference Method ed New Method Protocol #3 ed Failed 0 57 63 15 63 72 57 78 135 Sensitivity = 1.00
20 s Mask Sample 40 s Exercise Time 6 Exercises: Total Test Time = 3:35 Fail Criterion: 100 Reference Method 200 New Method Protocol #3 Table IVc: ANSI Requirement Study Results or Fail Sensitivity 0.95 1.00 Predictive Value of a 0.95 1.00 Specificity > 0.50 0.81 Predictive Value of a Fail > 0.50 0.79 Kappa Statistic > 0.70 0.78
20 s Mask Sample 40 s Exercise Time 6 Exercises: Total Test Time = 3:35 Fail Criterion: 500 Reference Method 1000 New Method Failed Protocol #3 Table Vc: Reference Method ed New Method Protocol #3 ed Failed 0 91 41 8 41 99 91 49 140 Sensitivity = 1.00
20 s Mask Sample 40 s Exercise Time 6 Exercises: Total Test Time = 3:35 Fail Criterion: 500 Reference Method 1000 New Method Protocol #3 Table Vc: ANSI Requirement Study Results or Fail Sensitivity 0.95 1.00 Predictive Value of a 0.95 1.00 Specificity > 0.50 0.84 Predictive Value of a Fail > 0.50 0.92 Kappa Statistic > 0.70 0.87
Summary of Test Sensitivity ANSI Z88.10 Sensitivity Requirement 0.95 Protocol Reference /Fail Criterion Exercise Time (sec) Mask Sample Time Sensitivity Result 1 100 30 10 0.91 Fail 2 100 40 20 0.97 3 100 40 20 1.00 1 500 30 10 0.97 2 500 40 20 0.97 3 500 40 20 1.00
All three protocols were highly correlated with the reference fit test method Protocol Number 1 2 3 Correlation Coefficient 0.956 0.988 0.982
Conclusions Reducing exercise time with the TSI PortaCount using these three protocols had little effect on the ability of this instrumentation to identify poorly fitting respirators. Protocol #2 exceeded all ANSI criteria. This protocol has a shorter mask sampling time (20 versus 40 seconds), with the same number of exercises.
Conclusions If you re a little uncomfortable with the shorter mask sampling time, you may want to consider protocol #3. Protocol #3 has the same mask sampling time as protocol #2, but eliminates the first two (less challenging) exercises from the OSHA protocol. In addition, it requires doubling the pass/fail criterion. This not only provides a significantly faster fit test protocol, but also provides a more conservative fit test.
Acknowledgements: This study was supported in part by TSI Inc., Shoreview, MN, USA. The authors exercised full control over the design and execution of the study as well as the interpretation and reporting of results.
For additional information, go to: McKay RT, Bradley J. Evaluation of Three New Fit Test Protocols for Use with the TSI PortaCount. Journal of the International Society of Respiratory Protection 22:112-121, 2005. or, e-mail Roy@DrMcKay.com
Questions? Evaluation of Three New Fit Test Protocols for Use with the TSI PortaCount by Roy T. McKay, Ph.D. Jeff Bradley, RRT Occupational Pulmonary Services Department of Environmental Health University of Cincinnati Cincinnati, OH 45267 Roy@DrMcKay.com