Agenda Item I.5.b Supplemental NOAA Presentation November 2012 This file contains the Powerpoint presentation with selected screen shots from the videos. Videos were not submitted electronically to the Council office. For information regarding the videos, please contact the presenter directly: Mr. Colby Brady National Marine Fisheries Service Telephone: 206-526-6117 Colby.Brady@noaa.gov
Computational Vision-based Monitoring (CVM) Northwest Region Current Agency Collaborative Research Efforts November 6, 2012
Alaska Regional Office-SFD efforts Pilot studies to test Archipelago Marine Research Electronic Monitoring (EM) data: Count halibut discard Obtain halibut lengths Studies showed: Successful using Archipelago EM data except- Not cost-effective Data not timely AK-SFD contracted Mamigo, Inc. to develop software: Count halibut discard Obtain halibut lengths Provide software analysis user interface U.S. Department of Commerce National Oceanic and Atmospheric Administration NOAA Fisheries Page 3
Graphic User Interface Jobs Panel Video Review Panel Video Files Panel Control Panel Detection Indicator Detection List U.S. Department of Commerce National Oceanic and Atmospheric Administration NOAA Fisheries Page 4
Video Tool Bar Location Indicator Timeline with Annotations Video Information Play/Pause Go to Begin/End Single Frame Step Play Speed Control Range Split Comment Tool Length Erase Tool Length Measurement Tool Manual Detection Tool U.S. Department of Commerce National Oceanic and Atmospheric Administration NOAA Fisheries Page 5
Extraction Result U.S. Department of Commerce National Oceanic and Atmospheric Administration NOAA Fisheries Page 6
Mamigo Recommendations Frame Rate Replicate frame & add to video stream, frame stuffing Video rated at 8 fps, but actual below 4 fps Eliminate analog/digital conversion capture cards Improve processor type U.S. Department of Commerce National Oceanic and Atmospheric Administration NOAA Fisheries Page 7
AFSC Research Overview: University of Washington Paul G. Allen School of Electrical Engineering and Computer Science Dr. Hwang, Meng-Che Chuang (graduate student) in collaboration with Alaska Fisheries Science Center Kresimir Williams and Craig Rose Funded by a National Cooperative Research Program Conservation Engineering grant. U.S. Department of Commerce National Oceanic and Atmospheric Administration NOAA Fisheries Page 8
Tracking, length, & bounding box algorithms (underwater). U.S. Department of Commerce National Oceanic and Atmospheric Administration NOAA Fisheries Page 9
Tracking, length, & bounding box algorithms (conveyor belt) U.S. Department of Commerce National Oceanic and Atmospheric Administration NOAA Fisheries Page 10
NWR-SFD Research Collaborators General Vision Inc. & CogniMem Technologies Inc Guy Paillet & Anne Menendez. University of Washington Paul G. Allen School of Electrical Engineering and Computer Science Dr. Hwang, Meng-Che Chuang. AFSC funded research from generosity of Craig Rose, Kresimir Williams. Dr. Shapiro, Lynn Yang. Pacific Seafood, (Warrenton, Oregon plant) Mike Okoniewski, Rick Harris, Mike Brown, Dominic Kohlasch. Oregon Department of Fish and Wildlife Dan Erikson, Dave Douglas, Liz Hanwacker. U.S. Department of Commerce National Oceanic and Atmospheric Administration NOAA Fisheries Page 11
CogniMem Technology General Vision Inc. Applied research arm of CogniMem image recognition chips. Assisted Agency with EM computer vision Discard Detection test from Archipelago EM data. Currently assisting Agency with development of proof of concept Computational Vision-based Monitoring units. Shoreside unit in development with web-based data transmission strategy. Vessel prototype in development. U.S. Department of Commerce National Oceanic and Atmospheric Administration NOAA Fisheries Page 12
CogniMem Technology Modern CPU processors Memory bottleneck High power consumption A multicore processor CM1K Image Recognition Chip Memory and processing logic combined in a same element Parallel architecture of identical elements Simple access to all elements connected in parallel The CM1K pattern recognition chip with 1024 identical cognitive memories in parallel U.S. Department of Commerce National Oceanic and Atmospheric Administration NOAA Fisheries Page 13
CogniMem s Image recognition Chip (CM1K) CPU/DSP CM1K 1 processor for many memory entries processor processor processor 1 processor for each memory entry processor processor processor Much faster than normal CPU processor U.S. Department of Commerce National Oceanic and Atmospheric Administration NOAA Fisheries Page 14
CogniMem deployment (~60 N. Atlantic herring processors) U.S. Department of Commerce National Oceanic and Atmospheric Administration NOAA Fisheries Page 16
Image Knowledge Builder (IKB ) showing no fishing event Region of interest Showing no event despite crew U.S. Department of Commerce National Oceanic and Atmospheric Administration NOAA Fisheries Page 17
Cod-end reaches ramp Region of interest cod end detected category U.S. Department of Commerce National Oceanic and Atmospheric Administration NOAA Fisheries Page 18
Discard Detection Test, General Vision Inc. U.S. Department of Commerce National Oceanic and Atmospheric Administration NOAA Fisheries Page 19
Discard Detection Test Lesson learned: Software able to be trained to detect discard event imagery. Improvements in EM Hardware needed for robust automatic detection of discard events: Improved Frame rates needed. Improved resolution needed. Use of digital image sensors needed. Road blocks in artificial intelligence practicality result from the lack of appropriate hardware architecture, Guy Paillet, General Vision co-founder. U.S. Department of Commerce National Oceanic and Atmospheric Administration NOAA Fisheries Page 20
Traditional EM, Archipelago Marine Research Inc. U.S. Department of Commerce National Oceanic and Atmospheric Administration NOAA Fisheries Page 21
General Vision Inc., Vessel Prototype Inside Bottom Inside Top Power supply 24 Volts input CogniBlox (2) Real time Cognitive Video Gigabit Ethernet 2 USB HS HDMI Output (test) Dual Core i.mx6xx ARM Running Android WiFi Module For Tablet connection 512 GB SSD (600 Mbytes/second) SATA 2 Outside IP68 Interface up to 8 cameras HD_SDI 1080p synchronized U.S. Department of Commerce National Oceanic and Atmospheric Administration NOAA Fisheries Page 22
General Vision Inc. & Pacific Seafoods Shoreside Prototype Development U.S. Department of Commerce National Oceanic and Atmospheric Administration NOAA Fisheries Page 23
Revised Installation Strategy: Intelli-Glass U.S. Department of Commerce National Oceanic and Atmospheric Administration NOAA Fisheries Page 24
NWR-SFD Research Goals Accurate Discard Event Data. Reduced review time. Low Monitoring Costs. Near Real Time Computational fisheries management. Real Time management becomes more of a tangible possibility. Vessel-Vessel on the grounds bycatch reduction. Speciation capabilities. Length and Volume 100% census of all catch and discards possible with continued development. U.S. Department of Commerce National Oceanic and Atmospheric Administration NOAA Fisheries Page 25