Developing a Validated Tool For Evaluating Whole Slide Images

Transcription

1 Developing a Validated Tool For Evaluating Whole Slide Images

2 Authors Russell Silowash, BS Evaluation Team Research Analyst Leslie Anthony, MA UPMC IMITS Telepathology Project Project Manager Robb Wilson, MA Evaluation Team Project Manager Dana Grzybicki, MD, PhD Evaluation Team Principal Investigator

3 Background Through appropriations in the defense-spending bills for 2002 and 2004, the University of Pittsburgh Medical Center (UPMC) and the United State Air Force Medical Service (AFMS) created a partnership called the Integrated Medical Information Technology System (IMITS) Program Telepathology is a branch of the IMITS program that implements and validates digital pathology practices

4 Introduction Whole Slide Imaging (WSI) systems are increasingly popular within pathology practices It is essential to acquire images that are diagnostically equivalent to glass slides There are no validated assessments for evaluating WSI cases The UPMC Digital Pathology Imaging Group is working on the validation of a unique evaluation tool

5 Participants 5 pathologists 2 pathology fellows 3 staff pathologists with training in GU pathology

6 Apparatus Prostate tissue viewed on Aperio ImageScope browser BX45 used to view glass slides Aperio T2 robotic scanner Aperio ImageScope browser used to view digital images of slides

7 Case Selection 30 difficult prostate biopsy foci Diagnosis Number of Cases Adenocarcinoma 12 Atypical 6 Atypical and High Grade PIN 1 Atypical PIN 1 High Grade PIN 2 Benign 8

8 Hypotheses Internal Validity There will be a positive correlation between the number of slides/images in a case and the time needed to complete the case External Validity Whole slide image quality will be positively correlated with glass slide quality

9 Hypotheses (continued) Construct Validity There will be a negative correlation between the diagnostic confidence of a participant and the case complexity rating Content Validity The assessment is asking the proper questions for the study at hand

10 Time Results TIME BREAKDOWN BY PHASE NUMBER OF CASES PATH 1 PATH 2 PATH 3 PATH 4 PATH < 15 MIN MIN MIN MISSING < 15 MIN MIN MIN MISSING WSI PHASE TIME VALUES GLASS Most cases took less than 15 minutes to complete for both WSI and glass phases

11 Internal Validity Results Glass Time Value WSI vs Glass Time Per Case Correlation y = x WSI Time Value Positive correlation exists between WSI time per case and glass time per case Statistically significant for only one participant (r 2 =0.327, p<0.01)

12 External Validity Results Slide Quality Rating Average Image vs Slide Quality Correlation y = x A positive correlation exists between WSI quality and glass slide quality Im age Quality Rating

13 External Validity Results (continued) WSI vs. Glass Quality Correlation Pathologist Spearman s Coefficient * * * *Statistically significant correlations (p<0.01) There were statistically significant positive correlations for 3 of the 5 subjects

14 Construct Validity Results Confidence In Diagnosis WSI Average Case Complexity vs Diagnostic Confidence y = x Negative correlation exists in WSI phase of study between confidence in diagnosis and case complexity Case Complexity

15 Construct Validity Results (continued) Confidence in Diagnosis Glass Average Case Complextiy vs Diagnostic Confidence y = x Negative correlation exists in the Glass phase between case complexity and diagnostic confidence Case Complexity

16 Construct Validity Results (continued) Case Complexity vs. Confidence In Diagnosis Pathologist WSI Phase Spearman s Coefficient Glass Phase Spearman s Coefficient * * * * * * * *Statistically significant correlations (p<0.01) 3 of 5 pathologists had statistically significant positive correlations in the WSI Phase 4 of 5 pathologists had statistically significant positive correlations in the Glass Phase

17 Content Validity Results Content validity has been obtained by gaining feedback from pathologists that are part of the Digital Pathology Imaging Group (DPIG)

18 Summary We were able to establish external and construct validity, however internal validity is rather low. Low internal validity could be due to time categories being too broad Slide quality correlates positively with image quality, which suggests good/bad slides causes good/bad images WSI may be used for more clinical applications

19 Limitations Only used 5 pathologist participants from UPMC who have had experience with WSI Only prostate tissue examined

20 Next Steps Change the categories for the time variable Time to Complete (Minus Interruptions) Less Than 15 Minutes Minutes Minutes Minutes Over 60 Minutes Time to Complete (Minus Interruptions) Less Than 30 Seconds 7 10 Minutes 30 Seconds 1 Minute Minutes 1 2 Minutes Minutes 2 5 Minutes Over 15 Minutes 5 7 Minutes Implement an automatic timing solution

21 Next Steps (continued) Recruit more pathologists to participate in our validation studies Other educational/medical institutions Air Force Expand across multiple tissue types

22 Acknowledgements We would like to acknowledge all of the members of the Digital Pathology Special Interest Group for their input and contributions towards this project, especially the Principal Investigator, Drazen Jukic,MD

23 Funding This work was supported by funding from the U.S. Air Force administered by the U.S. Army Medical Research Acquisition Activity, Fort Detrick, Maryland (Award No. W81XWH and Contract No. DAMD ). The content of the information does not imply U.S. Air Force or Government endorsement of factual accuracy or opinion