Interpretable Discovery in Large Image Data Sets

Transcription

1 Interpretable Discovery in Large Image Data Sets Kiri L. Wagstaff and Jake Lee Jet Propulsion Laboratory, California Institute of Technology December 7, 2017 NIPS Interpretable Machine Learning Symposium 2017, California Institute of Technology. Government sponsorship acknowledged. This work was performed in part at the Jet Propulsion Laboratory, California Institute of Technology, under a contract with NASA.

2 Discovery in Large Data Sets Scientific discoveries often come from outliers By Flickr user Klaus Wagstaff and Lee - NIPS

3 Discovery in Large Image Data Sets Surveillance Human faces Credit: Pixabay user Geralt Planetary Science HiRISE 1.4M images Credit: NASA/JPL-Caltech/Univ. of Arizona Challenges: Representation, Explanations Wagstaff and Lee - NIPS

4 Novelty Detection Methods Clustering Isolation Forest [Liu et al., 2008] Density-based (e.g., Local Outlier Factor [Breunig et al., 2000]) SVD DEMUD: SVD-based + explanations Explanation: SVD residual; information the model could not explain [Wagstaff et al., 2013] Wagstaff and Lee - NIPS

5 Why DEMUD? Incremental discovery using SVD model of selections Especially good for discovering rare classes Explanations justify selections Rare class discovery UCI glass data 6 Explanations UCI glass data Number of classes discovered DEMUD CLOVER NNDM 2 SEDER Interleave Static SVD Random Number of selected examples Wagstaff and Lee - NIPS

6 Why DEMUD? Incremental discovery using SVD model of selections Especially good for discovering rare classes Explanations justify selections Explanations help users classify items Intensity 20 x Explanations: ChemCam spectra Rhodochrosite: MnCO 3 +Mn Fe Fe Fe Fe Fe Fe Fe Mn Mn Mn Mn Mg Mg Mg ChemCam expert classification performance Cumulative accuracy DEMUD explanations No explanations Random Wavelength (nm) Selection number Wagstaff and Lee - NIPS

7 DEMUD for Images Representation Raw pixels SIFT [Lowe, 2004], HOG [Dalal & Triggs, 2005] CNN features [Razavian et al., 2014] Wagstaff and Lee - NIPS

8 DEMUD + CNN Representations Class probs Images [Krizhevsky et al., 2012] Wagstaff and Lee - NIPS

9 DEMUD + CNN Representations Class probs Images [Krizhevsky et al., 2012] Features DEMUD Wagstaff and Lee - NIPS

10 DEMUD Explanations with CNN Features Selection Features DEMUD Explanation? Invert residuals to get visual explanations Wagstaff and Lee - NIPS

11 DEMUD Explanations with CNN Features Selection Features DEMUD Explanation CNN Feature Inversion Methods Deep Goggle: Generate input that yields feature values (Mahendran & Vedaldi, 2015)? Invert residuals to get visual explanations Wagstaff and Lee - NIPS

12 DEMUD Explanations with CNN Features Selection Features DEMUD Explanation? Invert residuals to get visual explanations CNN Feature Inversion Methods Deep Goggle: Generate input that yields feature values (Mahendran & Vedaldi, 2015) Up-Conv: Predict original image with second NN (Dosovitskiy & Brox, 2016) Wagstaff and Lee - NIPS

13 Experiments ImageNet 1000 images 10 classes Evenly distributed Wagstaff and Lee - NIPS

14 Experiments ImageNet DEMUD-CNN SVD-CNN DEMUD-pixel SVD-pixel Random Wagstaff and Lee - NIPS

15 Explanations ImageNet Dial Food packet Dogsled Zucchini Up-Conv Deep Goggle Selection Bassoon Wagstaff and Lee - NIPS

16 Experiments MSL Rover images 6737 images 26 classes Uneven distribution Wagstaff and Lee - NIPS

17 Experiments MSL Rover images DEMUD-CNN SVD-CNN DEMUD-pixel SVD-pixel Random Wagstaff and Lee - NIPS

18 Explanations MSL Rover images REMS UV sensor MAHLI cal target Turret Ground Up-Conv Deep Goggle Selection Chemin inlet Wagstaff and Lee - NIPS

19 Summary DEMUD + CNN features + CNN feature inversion Fast discovery of novel images With visual explanations What will you find in your image data set? Thank you: NASA Planetary Data System (PDS) Imaging Node Wagstaff and Lee - NIPS