Susumu Takatsuka, Advanced Research Laboratory, Sony Group Corporation (Jointly appointed as a Visiting Researcher at the Japan Agency for Marine-Earth Science and Technology (JAMSTEC) since September 2021) gave an invited lecture on the development of marine particle quantification technology using event-based vision sensors (EVSs) at the Ocean Sciences Meeting (OSM) 2024 held in New Orleans, the U.S.A. The lecture was to present the results of a join research project with JAMSTEC researchers Norio Miyamoto and Shinsuke Kawagucci and an Advanced Research Laboratory researcher Hidehito Sato.
■ Ocean Sciences Meeting 2024
The Ocean Sciences Meeting (OSM) is one of the largest international conferences on ocean science that is held every other year and attended by ocean researchers from around the world.
This research project aims to build an observation system that monitors the ocean environment in real time using event-based vision sensors (EVSs) and generates big data on marine particles. Conventional ocean environment sensing methods typically involve microscopically observing samples of collected and filtered seawater. These conventional methods are incapable of real-time monitoring of the ocean environment because they cannot dynamically observe particles.
To solve this problem, we use EVSs for sensing, which enables dynamic monitoring of observation objects using a camera system installed in the ocean. Also, when compared to the traditional image sensor, the EVS is suitable for marine particle sensing because it supports high frame rates for accurate monitoring of particle movement, capability to detect low-luminance changes in the dark ocean, and low power consumption achieved by outputting only changes and reducing the amount of data, among other features. Through our research, we are currently analyzing 22 individual features in three aspects: silhouette, velocity, and periodicity.
Moreover, white-light illumination, used in conventional shooting techniques, causes phototaxis, a phenomenon in which organisms such as plankton are drawn to the light source. This makes it impossible to monitor the natural condition accurately. The EVS can detect even the slightest change in near-infrared light, which allows the use of 760-nm wavelength for illumination that affects almost no organisms, enabling non-invasive measurement. We classified 138 measurement results obtained with this system configuration using deep learning (results of measuring mixtures of five types of particles: copepods in the adult and larval stages, shells and starfish in the larval stage, and sediment). We checked the test data and obtained an accuracy rate of 81%. Furthermore, when the distinction between adult copepods and larval ones was removed, the accuracy rate increased to 86%. In the lecture, we also reported on the results of observing Lake Biwa in Japan as well as the hydrothermal deposits of the Myojin Seamount.
We look forward to receiving feedback on this presentation from experts. Our proposed EVS-based observation system is available for free lease. Please feel free to contact us.
Part of this research is being conducted in the project selected in the public call of the Ministry of Education, Culture, Sports, Science and Technology in 2021, “Program for the Development of Technology to Promote the Utilization of Ocean Resources – Advancement of Technologies for Utilizing Big Data of Marine Life.” Susumu Takatsuka serves as the leading researcher of the project.
We are contributing to several other ocean research projects as well. One of them is a grant-in-aid project led by Naomi Harada, which has been adopted under specially promoted research. The project is titled “Response of biogeochemical cycle and lower trophic ecosystem to the environmental change in the Southern Ocean, East Antarctica.” Naomi Harada is a leading expert on polar research and serves as a professor at the Center for International Collaboration of the Atmosphere and Ocean Research Institute of the University of Tokyo.
