PCIe/104 - The "fishal" recognition revolution in fish counting

Fish-count surveys are vital in ensuring the conservation and management of fish stocks in our increasingly overfished seas. The accuracy of those surveys is critical in providing evidence of overfishing to drive restorative and preventative legislation. Traditional fish-counting is inherently manual, relying on fishing trawlers casting enormous nets into the ocean, pulling aboard as many fish as possible to manually count and develop the trawl survey.

Environmentally, the primary shortcoming of this approach is the number of fish killed during the course of this out of water survey. Naturally areas demanding the closest monitoring are those that are most threatened, thus such surveys can quickly become counter-productive to the aim of protecting their stocks. Trawling nets do not discriminate, so it’s not just fish that can get caught in the net. Additionally, often small species of fish are too small to be caught by the wide net mesh required to provide the strength to lift the catch out of the ocean.

Inaccuracies of the traditional methodologies also fail to support protection legislation. There are substantial error margins involved in extrapolating from a small surveyed area to the larger area likely seeking protection. The length of time taken to undertake the survey and natural free movement of fish stocks again must again be addressed via extrapolation. Finally, the relatively high cost of the equipment and resource often dictates that the ideal level of monitoring required is prohibited.

Seeking to revolutionize this historic practice, covering an area of ocean with particular commercial and scientific interest, the National Oceanic and Atmospheric Administration (NOAA) sought the answer in the power of embedded computing and specifically, in the flexibility of PCIe/104. In the latest iteration of this concept, PC/104 consortium members ADL Embedded Solutions, Inc. and RTD Inc. recently partnered to help bring this idea from an R&D exercise to reality.

NOAA’s National Marine Fisheries service (NMFS) manages nearly 3 million square miles of coastal waters off Alaska and the west coast of the United States. This region supports some of the most significant commercial fisheries on the planet, yet are also home to some of the largest marine mammal populations.

NMFS have been working tirelessly in the research and development phase of a camera based trawl technology (codenamed CAM-Trawl) for a number of years. The industry advances in underwater image capturing and facial recognition sought to address the many shortfalls of the traditional approach – loosely cleverly termed “Fishal Recognition.”

Early generations of the CAM-Trawl system worked in isolation, effectively just capturing myriad images to local storage, which necessitated analysis at some later date back at the laboratory. While addressing many of the environmental and accuracy shortfalls, the process remained laborious and relatively expensive.

CAM-Trawl (Figure 1) enables undersea fish-count surveys with little disturbance to physical fish stocks.

Figure 1: CAM-Trawl enables undersea fish-count surveys with little disturbance to physical fish stocks.

NMFS sought the assistance of ADL Embedded Solutions Inc via their Fish Management Acquisition (FMA) program to improve their proof-of-concept to the point it truly revolutionized the process. Employing a PCIe/104 stack with Quad-Core Intel Core-i7 the CAM-Trawl system gained the local performance necessary to facilitate real-time processing of image data, all whilst reducing the overall size to improve mounting flexibility and ease of storage and transportation. Typically residing in the wheelhouse, the Control Computer steadily monitors a wide range of external sensors monitoring geographical data, time, pressure, RFID tag readings and control of Vision Boxes derived from sensor input. Acting as a control sensor, it autonomously manages remote power on/off, start/stop image acquisition and clock syncing in a one-to-many relationship. The systems are housed in a stainless steel IP67 enclosure with waterproof I/O connectors to guarantee longevity in harsh marine environments.

Why PCIe/104? No other configuration offers flexible support for today’s PCIe functions of quad GbE and quad USB 3.0, while facilitating likely future expansion needs of a PCIe x16 GPU or 10GbE LAN. The mechanical methodology of the famed PC/104 stack is sufficiently rugged for the shock and vibration stress of a top-side environment on a fishing trawler while enabling a small-form-factor design for an IP67 marine environment.

The Image Acquisition Computer naturally drives image capture. The GeniCam-compliant vision platform uses the latest vision technology and supports up to 6x GbE ports for camera interfacing, critically adding real-time image processing of those incoming video streams and storing crucial metadata for date/time, fish identify, etc.

Figure 2: Left: ADL Control Computer, Right: ADL image acquisition computer.

The combination has revolutionized fish surveying and achieved all the stated goals for the project. Real-time fish count data analysis is now instantly available for NOAA marine researchers to aid in their conservation efforts. Still new technology, the results from experimental trawl surveys in 2016 has been very positive and work is underway to standardize on the key components utilised to facilitate mass adoption of “Fishal Recognition” technology.

Is the revolution complete? Not yet. The CAM-Trawl concept has indeed addressed the issues of traditional fish-surveying, then localized image processing has slashed the cost and processing time of data obtained via this means. So what’s next?

The next step will evolve how that image capture data is processed, likely employing the power of FPGAs in parallel to traditional computing where FPGAs are deployed in data-centers and undertaking pre-processing to achieve data-set reductions in majority of image applications “on land”. FPGAs are particularly performance and power efficient using run-length-encodes to get rid of “nothing” – in layman’s terms we mean handling any mathematics, at lightning speed. Such FPGA pre-processing would both increase data processing speed and slash the CPU performance requirements of the host computer.

While fish-counting may appear to be a somewhat niche need, the possibilities to revolutionize other applications throughout numerous industry sectors is unprecedented. The PC/104 consortium is collectively working tirelessly to realize the hardware necessary to facilitate these, with companies including Sundance, Euresys, EPIX, Sensoray, and AMP Ltd all heavily involved in image capture, processing, and pre-processing under PCIe/104.