Computer-Aided Technologies for Food Risk Assessment

Computer-Aided Technologies for Food Risk Assessment Candidate: Francesco Rossi Tutor: Prof. Alfredo Benso XXX Cycle PhD in Computer and Control Eng. at Politecnico di Torino

Motivation of PhD in Computer Science Master s Degree Biomedical Engineering Virtual Colonoscopy Prostate Cancer Research #machinelearning #computervision #computeraided

Computer-Aided Technologies for Food Risk Assessment Unitentional Intentional Economical Gain FOOD QUALITY FOOD FRAUD Health Harm FOOD SAFETY FOOD DEFENSE

Outline Dairy Product Fish Traceability Species Substitution

Heuristic Molecular Dairy Farming Analysis From farm to Fork Food Traceability Qualitative and Quantitative certification STR-DNA Pool Analisys

Heuristic Molecular Dairy Farming Analysis From farm to Fork Get around the pool problem

Data 2 farms for 12 months 1. Sample Collection (DNA) 2. STRs selection (20) 3. Genotyping Process (STR) 4. Data extraction (RFU) Cow 2 alleles Pool vector of alleles

Assumption N i = 0 i On-site contamination & some possible loss Some loss during ripening

Correct

Correct Medium Forgery

Correct Medium Forgery High Forgery

DATA COWs HEURISTIC SIMULATION CMA-ES DATA POOL W Predicted POOL Every dairy product (i.e. 46 pools) has been simulated 24 times P1 & P2 SSE Heuristic Analysis 0-50-100 % forgery SCORE

DATA COWs POOL HEURISTIC SIMULATION CMA-ES W CMA-ES Covariance Matrix Adaptation Evolution Strategy Predicted POOL P1 & P2 SSE SCORE

DATA HEURISTIC SIMULATION COWs CMA-ES W POOL W N Predicted POOL P1 & P2 SSE i = 0 i w i SCORE

DATA COWs HEURISTIC SIMULATION CMA-ES W POOL W Predicted POOL lower boundary= 0.5 m upper boundary=max ( 3 m, 1 ) P1 & P2 SSE SCORE

DATA COWs POOL HEURISTIC SIMULATION CMA-ES W Predicted POOL 12000 10000 8000 6000 4000 2000 0 Predicted POOL CORRECT POOL P1 & P2 SCORE SSE 7000 ALTERATED POOL 6000 5000 4000 3000 2000 1000 0

DATA COWs POOL HEURISTIC SIMULATION CMA-ES W P1 - forgery Rate of alleles that are included in the pool s profile but not in the cows P1 & P2 Predicted POOL SSE P2 - loss by ripening Percentage of alleles within the cows profile but not detected in the pool SCORE

DATA COWs POOL HEURISTIC SIMULATION CMA-ES W SSE Sum of Squared Errors between original and predicted pool Predicted POOL P1 & P2 SSE SCORE = P1 P2 SSE SCORE

Results

Results grouped by pool s type

Final Results & Conclusion

Vision and Learning in Fish Species Identification OK il Pesce è Giusto Fish Species Substitution

Introduction

Introduction VISUAL FEATURES

Introduction VISUAL FEATURES FEATURE ENGINEERING

Introduction VISUAL FEATURES FEATURE ENGINEERING CLASSIFICATION MODEL Feature engineering turn your inputs into things the algorithm can understand

DATA Raw Image

DATA Segmentation

DATA 12 Key-Points

DATA INFORMATION 30 features

«Is really this the species declared on the label or not?» INFORMATION KNOWLEDGE

Dataset 339 fishes COMMON NAME LATIN NAME N European Anchovy Engraulis encrasicolus 125 European Pilchard Sardina pilchardus 107 Common Pandora Pagellus erythrinus 20 Atlantic Mackerel Scomber scombrus 18 Gilt-Head Bream Sparus aurata 22 European Hake Merluccius merluccius 19 Striped Red Mullet Mullus surmuletus 28 Classification Model ANNs: One-Class & Multi-Class Classifiers

Results Leave One Out Cross and in-field validation OCC 100% Acc. MCC 100% Acc (only with sardine and anchovy) Engraulis encrasicolus Sardina pilchardus Pagellus erythrinus Scomber scombrus Sparus aurata Merluccius merluccius Mullus surmuletus MCC OCC

RAI1 - Live Demo

Limitations New order/family/genre/species of fish may require new features Segmentation will fail if background is not uniform, and key-points and features too The two step for key-points should be improved Key-points interaction should be avoided

FEATURES CLASSIFIER OUTPUT

FEATURES CLASSIFIER OUTPUT??? OUTPUT

FEATURES CLASSIFIER OUTPUT DEEP LEARNING OUTPUT

F.I.S.HUB Fish Identification Software Hub Database Classifier Mobile App Validation Fillets Recognition

Database GUIDELINES 25 species > 7k pictures UK & Italy

Database ORDER FAMILY GENUS SPECIES Clupeiformes Clupeidae Clupea harengus Clupeiformes Clupeidae Sardina pilchardus Clupeiformes Clupeidae Sprattus sprattus Clupeiformes Engraulidae Engraulis encrasicolus Gadiformes Gadidae Gadus morhua Gadiformes Gadidae Melanogrammus aeglefinus Gadiformes Gadidae Merlangius merlangus Gadiformes Gadidae Pollachius virens Gadiformes Merluccidae Merluccius merluccius Perciformes Sparidae Dentex dentex Perciformes Sparidae Dentex gibbosus Perciformes Sparidae Diplodus annularis Perciformes Sparidae Pagellus acarne Perciformes Sparidae Pagellus bogaraveo Perciformes Sparidae Pagellus erythrinus Perciformes Sparidae Pagrus caeruleostictus Perciformes Sparidae Pagrus pagrus Pleuronectiformes Pleuronectidae Hippoglossus hippoglossus Pleuronectiformes Pleuronectidae Limanda limanda Pleuronectiformes Pleuronectidae Microstomus kitt Pleuronectiformes Pleuronectidae Pleuronectes platessa Pleuronectiformes Pleuronectidae Reinhardtius hippoglossoides Pleuronectiformes Scophthalmidae Psetta maxima Pleuronectiformes Scophthalmidae Scophthalmus rhombus Pleuronectiformes Soleidae Solea vulgaris

Database NORMAL FISH FLATFISH ORDER FAMILY GENUS SPECIES Clupeiformes Clupeidae Clupea harengus Clupeiformes Clupeidae Sardina pilchardus Clupeiformes Clupeidae Sprattus sprattus Clupeiformes Engraulidae Engraulis encrasicolus Gadiformes Gadidae Gadus morhua Gadiformes Gadidae Melanogrammus aeglefinus Gadiformes Gadidae Merlangius merlangus Gadiformes Gadidae Pollachius virens Gadiformes Merluccidae Merluccius merluccius Perciformes Sparidae Dentex dentex Perciformes Sparidae Dentex gibbosus Perciformes Sparidae Diplodus annularis Perciformes Sparidae Pagellus acarne Perciformes Sparidae Pagellus bogaraveo Perciformes Sparidae Pagellus erythrinus Perciformes Sparidae Pagrus caeruleostictus Perciformes Sparidae Pagrus pagrus Pleuronectiformes Pleuronectidae Hippoglossus hippoglossus Pleuronectiformes Pleuronectidae Limanda limanda Pleuronectiformes Pleuronectidae Microstomus kitt Pleuronectiformes Pleuronectidae Pleuronectes platessa Pleuronectiformes Pleuronectidae Reinhardtius hippoglossoides Pleuronectiformes Scophthalmidae Psetta maxima Pleuronectiformes Scophthalmidae Scophthalmus rhombus Pleuronectiformes Soleidae Solea vulgaris

Database MANY FEW ORDER FAMILY GENUS SPECIES Clupeiformes Clupeidae Clupea harengus Clupeiformes Clupeidae Sardina pilchardus Clupeiformes Clupeidae Sprattus sprattus Clupeiformes Engraulidae Engraulis encrasicolus Gadiformes Gadidae Gadus morhua Gadiformes Gadidae Melanogrammus aeglefinus Gadiformes Gadidae Merlangius merlangus Gadiformes Gadidae Pollachius virens Gadiformes Merluccidae Merluccius merluccius Perciformes Sparidae Dentex dentex Perciformes Sparidae Dentex gibbosus Perciformes Sparidae Diplodus annularis Perciformes Sparidae Pagellus acarne Perciformes Sparidae Pagellus bogaraveo Perciformes Sparidae Pagellus erythrinus Perciformes Sparidae Pagrus caeruleostictus Perciformes Sparidae Pagrus pagrus Pleuronectiformes Pleuronectidae Hippoglossus hippoglossus Pleuronectiformes Pleuronectidae Limanda limanda Pleuronectiformes Pleuronectidae Microstomus kitt Pleuronectiformes Pleuronectidae Pleuronectes platessa Pleuronectiformes Pleuronectidae Reinhardtius hippoglossoides Pleuronectiformes Scophthalmidae Psetta maxima Pleuronectiformes Scophthalmidae Scophthalmus rhombus Pleuronectiformes Soleidae Solea vulgaris

Classifier

Examples good m is the metric Melanogrammus aeglefinus m=0,052 Engraulius encrasicolus m= 0,023 Psetta maxima m=0,015

Examples wrong m is the metric Sardina pilchardus Sprattus sprattus m= 1,129 Hippoglossus hippoglossus Microstomus kitt m=1,612 Merlangius merlangus Pollachius virens m= 1,741

Classifier - Results t-sne plot for species clustering Features embedding in 10 dimension

t-sne plot for species clustering Features embedding in 10 dimension Classifier - Results KNN

Mobile Application & Live Demo Record

Validation FISHUB Database IN FIELD with DNA analysis Global Acc. 94% e.g. Dentex dentex Acc. 86% Diplodus annularis Acc. 96% Sardina pilchardus Acc. 96% The Species Acc. is higher when the number of picture per species are elevated and vice-versa 69 fish sampled with pictures and DNA barcode 9 fraud found by DNA 5 fraud discovered by App 62 correct identification by App

Fillets Recognition EXPLORATORY TASK Legend Normal image FILLET + RESULTS Microscope + Toluidine blue stain Molecular sensor

+ +

NIR - SCiO The Near-Infrared spectroscopic is a method that makes use of the electromagnetic spectrum from about 700 nm to 2500 nm. It can penetrate tissues and it is useful to probe bulk material with essentially no preparation Acquisition Visualization Data Model

For the POC we selected 10 fish fillets for each considered species: Solea solea, Pleuronectes platessa, Pangasianodon hypophthalmus. METHOD Details Global Accuracy % 1 normal img 59.8 2 microscope img 49.5 + + 3 normal img + staining 56.6 4 microscope img + staining 53.2 5 SCiO NIR 98.3 For methods 1-2-3-4 the pictures were collected and processed with image feature extraction techniques such as: Scale-Invariant Feature Transform (SIFT), Speeded Up Robust Feature (SURF) and Gray Level Co-Occurrence Matrix (GLCM). Eventually a SVM classifier was used.

Latest News 3 random scans per fillets 40 fillets per species RESULTS 98% Accuracy Gadus morhua Pleuronectes platessa Pollachius virens Epinephelus costae Synaptura cadenati Sebastes norvegicus Merluccius merluccius Scomber scombrus

Conclusion Bio* Data Interdisciplinarity Computer Science Model

PhD List of Publications 1 Journal Paper * 3 Conference Proceeding * 3 Abstract * first author

Computer-Aided Technologies for Food Risk Assessment Candidate: Francesco Rossi Tutor: Prof. Alfredo Benso XXX Cycle PhD in Computer and Control Eng. at Politecnico di Torino

N Name 15 STR 10 STR 1 AGLA29 ok ok 2 MB025 ok ok 3 Z27077 ok ok 4 BMS2142 5 MB071 ok ok 6 SRC276 7 MB064 8 BM1706 ok ok 9 HUJ625 ok 10 BMC1207 ok 11 AGLA232 ok 12 BM3507 13 BM4602 14 BMC6020 ok ok 15 INRA133 ok ok 16 BMS0607 ok ok 17 BMC4214 ok ok 18 BM720 ok 19 BMS4044 ok ok 20 BM0143 ok

GTX 580 600 W 1.5 GB 400 DGX-2 10 kw 512 GB 420 k

Processed Assumes Beer-Lambert model is valid, and transforms the measured signal to be linear with concentration by doing a log transform and adjusting the result for noise and deviations from the model. Normalized Performs normalization of the signal. This is meant to compensate for changing measurement conditions (e.g. varied scanning distances) that typically occur from sample to sample. Y axis still means reflectance but in normalized units instead of raw reflectance. Processed and Normalized First assumes Beer-Lambert model (Processed) and then normalizes the results to compensate for differences in the optical path between samples. This is useful, for example, when there is variation in the thickness of the samples. (log(r)) and Normalized Similar to Processed and Normalized, uses a more aggressive form of Processed. Adds more noise, but in some cases may be the only way to create a good model.