Interactive Machine Learning Tool for Marine Image Analysis

Even though ocean exploration has exponentially increased in the last two decades due to the development of novel technologies, such as drop cameras, remotely operated vehicles (ROVs), automated underwater vehicles (AUVs), and time-lapse image/video camera systems, we currently lack the capacity to analyse these large visual data sets to their full potential.

The new open-source software tool, RootPainter, offers a solution. This powerful AI tool allows you to rapidly and accurately develop models to automatically identify species/objects from large image datasets (Smith et al. 2022; Clark et al. 2024). The tool extracts valuable information such as the count, surface area, diameter, perimeter, eccentricity, the x, and y coordinates of each discrete area, five to sixteen times faster than manual annotation, without needing to pre-process the images (Clark et al. 2024).

The advantages of RootPainter are: it is open-source, you can run it from any laptop (it can run on Googles GPUs), it is user-friendly, you can create robust models with relatively few images (100s rather than 1000s), it gives you accuracy metrics (so you can report on the performance and know when to stop training) and you can easily re-apply model on new data or share it collaboratively with others. It is also fun to use!

RootPainter Manual and tutorial

Want to try it yourself? Follow the steps in our manual, which you can download here. Our manual explains how to install the software, and how to train, apply and share a model.

Projects & workshops

Publications

Nozères, C. et al. Image annotations for biodiversity with benthic landers in the Gully MPA and Scotian Shelf from 2021-2023. Can. Manuscr. Rep. Fish. Aquat. Sci. 3290, iv + 99 p. (2024).

Clark, H. P., Smith, A. G., McKay Fletcher, D., Larsson, A. I., Jaspars, M., & De Clippele, L. H. (2024). New interactive machine learning tool for marine image analysis. Royal Society Open Science, 11(5), 231678.

Clark, H. P., Smith, A. G., & De Clippele, L. (2024). Marine Image Analysis Handbook for RootPainter. Zenodo. 10.5281/zenodo.7984564.

Smith, A.G., Han, E., Petersen, J., Olsen, N.A.F., Giese, C., Athmann, M., Dresbøll, D.B. and Thorup‐Kristensen, K., 2022. RootPainter: deep learning segmentation of biological images with corrective annotation. New Phytologist, 236(2), pp.774-791.

Model zoo

Available models (A); Submitted as part of a publication, and coming soon (CS); Models in preparation (IP)

Image data acknowledgements

Clark, H.P., Smith, A.G., McKay Fletcher, D., Larsson, A.I., Jaspars, M. and De Clippele, L.H. (2023). M. lingua, D. pertusum and laser scale models. Image data collected in 2021 during AmpLOHPHELIA project (funded by ASSEMBLE plus) in the cold-water coral Tisler reef (Norway). The research vessel Nereus, stationed at the Tjärnö Marine Laboratory was used to deploy the Ocean Modules ROV.

Kohler O., De Clippele L.H., Maier S. (in prep): Bolocera sp. and round sponge models. Image data collected during the FATE cruise 2023 (funded by Dansk Center for Havforskning). Copyright of images is with AWI (Alfred Wegener Institute) and GCRC (Greenland Climate Research Centre).

De Clippele L.H., Nozeres C., Lirette C. Kenchington E. (in prep). Redfish, Silver Hake, White Hake, Rainbow squat lobster, Shrimp models. Image data collected with time-lapse image camera attached to landers, which were deployed from 2021-2022 in Sambro Bank (Canada) during the Hudson2021-048 mission with Fisheries and Oceans Canada