Millisecond-scale behaviours of plankton quantified in situ and in vitro using the Event-based Vision Sensor (EVS)

Abstract

The Event-based Vision Sensor (EVS) is a bio-inspired sensor that captures detailed motions of objects, aiming to become the ‘eyes’ of machines like self-driving cars. Compared to conventional frame-based image sensors, EVS has an extremely fast motion capture equivalent to 10,000-fps even with standard optical settings plus high dynamic ranges for brightness and also lower consumption of memory and energy. Here, we developed 22 characteristic features for analysing the motions of aquatic particles from the EVS raw data, and tested the applicability in analysing plankton behaviour. Several particles exhibiting distinct cumulative trajectory with periodicities in their motion (up to 16 Hz) were identified from turbid water at the bottom of Lake Biwa, Japan, suggesting that they were living organisms with rhythmic behaviour. We also used EVS in the deep sea, observing particles with active motion and periodicities over 40 Hz. Laboratory cultures of six species of zooplankton and phytoplankton were also observed, confirming species-specific motion periodicities up to 41 Hz. We applied machine learning to automatically classify particles into four categories of zooplankton and passive particles, achieving an accuracy up to 86%. Our application of EVS, especially focusing on its millisecond-scale temporal resolution and wide dynamic range, provides a new avenue to investigate rapid and periodical motion and behaviour in small organisms. The EVS will likely be applicable in the near future for the automated monitoring of plankton behaviour by edge computing on autonomous floats, as well as quantifying rapid cellular-level activities under microscopy.