Coverage estimation of benthic habitat features by semantic segmentation of underwater imagery from South-eastern Baltic reefs using deep learning models

Šiaulys, Andrius; Vaičiukynas, Evaldas; Medelytė, Saulė; Buškus, Kazimieras

doi:10.1016/j.oceano.2023.12.004

Artykuł - szczegóły

Tytuł artykułu

Coverage estimation of benthic habitat features by semantic segmentation of underwater imagery from South-eastern Baltic reefs using deep learning models

Autorzy

Šiaulys Andrius , Vaičiukynas Evaldas , Medelytė Saulė , Buškus Kazimieras

Wybrane pełne teksty z tego czasopisma

Identyfikatory

DOI

10.1016/j.oceano.2023.12.004

Warianty tytułu

Języki publikacji

Abstrakty

Underwater imagery (UI) is an important and sometimes the only tool for mapping hard-bottom habitats. With the development of new camera systems, from hand-held or simple “drop-down” cameras to ROV/AUV-mounted video systems, video data collection has increased considerably. However, the processing and analysing of vast amounts of imagery can become very labour-intensive, thus making it ineffective both time-wise and financially. This task could be simplified if the processes or their intermediate steps could be done automatically. Luckily, the rise of AI applications for automatic image analysis tasks in the last decade has empowered researchers with robust and effective tools. In this study, two ways to make UI analysis more efficient were tested with eight dominant visual features of the Southeastern Baltic reefs: 1) the simplification of video processing and expert annotation efforts by skipping the video mosaicking step and reducing the number of frames analysed; 2) the application of semantic segmentation of UI using deep learning models. The results showed that the annotation of individual frames provides similar results compared to 2D mosaics; moreover, the reduction of frames by 2–3 times resulted in only minor differences from the baseline. Semantic segmentation using the PSPNet model as the deep learning architecture was extensively evaluated, applying three variants of validation. The accuracy of segmentation, as measured by the intersection-over-union, was mediocre; however, estimates of visual coverage percentages were fair: the difference between the expert annotations and model-predicted segmentation was less than 6–8%, which could be considered an encouraging result.

Słowa kluczowe

underwater video epibenthos mosaicking segmentation computer vision PSPNet ResNet Baltic Sea

Wydawca

Polish Academy of Sciences, Institute of Oceanology
Elsevier

Czasopismo

Oceanologia

Rocznik

2024

Tom

Vol. 66 (2)

Strony

286--298

Opis fizyczny

Bibliogr. 38 poz., fot., map., rys., tab., wykr.

Twórcy

autor

Šiaulys Andrius

andrius.siaulys@ku.lt

Marine Research Institute, Klaipeda University, Klaipeda, Lithuania

autor

Vaičiukynas Evaldas

Faculty of Informatics, Kaunas University of Technology, Kaunas, Lithuania

autor

Medelytė Saulė

Marine Research Institute, Klaipeda University, Klaipeda, Lithuania

autor

Buškus Kazimieras

Faculty of Informatics, Kaunas University of Technology, Kaunas, Lithuania

Bibliografia

1. Alonso, I., Yuval, M., Eyal, G., Treibitz, T., Murillo, A.C., 2019. CoralSeg: Learning coral segmentation from sparse annotations. J. Field Robot. 36 (8), 1456-1477. https://doi.org/10.1002/rob.21915
2. Beijbom, O., Edmunds, P.J., Roelfsema, C., Smith, J., Kline, D.I., Neal, B.P., Dunlap, M.J., Moriarty, V., Fan, T.Y., Tan, C.J., Chan, S., 2015. Towards automated annotation of benthic survey images: Variability of human experts and operational modes of automation. PLOS One 10 (7), e0130312. https://doi.org/10.1371/journal.pone.0130312
3. Bučas, M., Daunys, D., Olenin, S., 2007. Overgrowth patterns of the red algae Furcellaria lumbricalis at an exposed Baltic Sea coast: the results of a remote underwater video data analysis. Estuar. Coast. Shelf S. 75 (3), 308-316. https://doi.org/10.1016/j.ecss.2007.04.038
4. Buhl-Mortensen, L., Buhl-Mortensen, P., Dolan, M.F., Holte, B., 2015. The MAREANO programme—A full coverage mapping of the Norwegian off-shore benthic environment and fauna. Mar. Biol. Res. 11 (1), 4-17. https://doi.org/10.1080/17451000.2014.952312
5. Buškus, K., Vaiˇciukynas, E., Verikas, A., Medelyt˙e, S., Šiaulys, A., Šaškov, A., 2021. Automated quantification of brittle stars in seabed imagery using computer vision techniques. Sensors 21 (22), 7598. https://doi.org/10.3390/s21227598
6. Casoli, E., Ventura, D., Mancini, G., Pace, D.S., Belluscio, A., Ardizzone, G., 2021. High spatial resolution photo mosaicking for the monitoring of coralligenous reefs. Coral Reefs 40 (4), 1267-1280. https://doi.org/10.1007/s00338- 021- 02136- 4
7. Chao, L., Wang, M., 2010. Removal of water scattering. In: International conference on computer engineering and technology IEEE, 2, 35. https://doi.org/10.1109/ICCET.2010.5485339
8. Dabuleviciene, T., Kozlov, I.E., Vaiciute, D., Dailidiene, I., 2018. Remote sensing of coastal upwelling in the south-eastern Baltic Sea: Statistical properties and implications for the coastal environment. Remote Sens. 10 (11), 1752. https://doi.org/10.3390/rs10111752
9. Eelbode, T., Bertels, J., Berman, M., Vandermeulen, D., Maes, F., Bisschops, R., Blaschko, M.B., 2020. Optimization for medical image segmentation: theory and practice when evaluating with dice score or Jaccard index. IEEE T. Med. Imaging 39 (11), 3679-3690. https://doi.org/10.1109/TMI.2020.3002417
10. Elhamifar, E., Sapiro, G., Vidal, R., 2012. See all by looking at a few: Sparse modeling for finding representative objects. In: Proc. CVPR IEEE, 1600-1607. https://doi.org/10.1109/CVPR.2012.6247852
11. Ferreira, T., Rasband, W., 2012. ImageJ user guide — IJ 1.46. URL: https://imagej.net/docs/guide/ (accessed on 2 August 2023).
12. Gracias, N., Garcia, R., Campos, R., Hurtos, N., Prados, R., Shihavuddin, A.S.M., Nicosevici, T., Elibol, A., Neumann, L., Escartin, J., 2017. Application challenges of underwater vision. In: López, A.M., Imiya, A., Pajdla, T., Álvarez, J.M. (Eds.), Computer Vision in Vehicle Technology: Land, Sea & Air. Wiley, Chichester, 133-160. https://doi.org/10.1002/9781118868065.ch7
13. He, K., Zhang, X., Ren, S., Sun, J., 2016. Deep Residual Learning for Image Recognition. Proc. CVPR IEEE, 770-778. https://doi.org/10.1109/CVPR.2016.90
14. Islam, M.J., Edge, C., Xiao, Y., Luo, P., Mehtaz, M., Morse, C., Enan, S.S., Sattar, J., 2020. Semantic segmentation of underwater imagery: Dataset and benchmark. IEEE Int. C. Int. Robot. 1769-1776. https://doi.org/10.1109/iros45743.2020.9340821
15. Kohler, K.E., Gill, S.M., 2006. Coral Point Count with Excel extensions (CPCe): A Visual Basic program for the determination of coral and substrate coverage using random point count methodology. Comput. Geosci. 32 (9), 1259-1269. https://doi.org/10.1016/j.cageo.2005.11.009
16. Labelbox. URL: https://labelbox.com (accessed 2 August 2023).
17. Langenkämper, D., Zurowietz, M., Schoening, T., Nattkemper, T.W., 2017. Biigle 2.0-browsing and annotating large marine image collections. Front. Mar. Sci. 4, 83. https://doi.org/10.3389/fmars.2017.00083
18. Li, Y., Randall, C.J., van Woesik, R., Ribeiro, E., 2019. Underwater video mosaicing using topology and superpixel-based pairwise stitching. Expert Syst. Appl. 119, 171-183. https://doi.org/10.1016/j.eswa.2018.10.041
19. Liu, F., Fang, M., 2020. Semantic segmentation of underwater images based on improved Deeplab. J. Mar. Sci. Eng. 8 (3), 188. https://doi.org/10.3390/jmse8030188
20. Martin-Abadal, M., Guerrero-Font, E., Bonin-Font, F., Gonzalez-Cid, Y., 2018. Deep semantic segmentation in an AUV for on-line Posidonia oceanica meadows identification. IEEE Access 6, 60956-60967. https://doi.org/10.1109/ACCESS.2018.2875412
21. Medelyte, S., Šiaulys, A., Daunys, D., Włodarska-Kowalczuk, M., W˛esławski, J.M., Olenin, S., 2022a. Application of underwater imagery for the description of upper sublittoral benthic communities in glaciated and ice-free Arctic fjords. Polar Biol. 45 (12), 1655-1671. https://doi.org/10.1007/s00300- 022- 03096- 3
22. Medelyte, S., Šiaulys, A., Vaiciukynas, E., Buškus, K., Šaškov, A., Olenin, S., 2022b. A fully-annotated imagery dataset of sublittoral benthic species in South Eastern Baltic Sea reefs. Mendeley Data, V1. https://doi.org/10.17632/wsd42v8mk5.1
23. Moghaddam, S.H.A., Mokhtarzade, M., Beirami, B.A., 2020. A feature extraction method based on spectral segmentation and integration of hyperspectral images. Int. J. Appl. Earth Obs. 89, 102097. https://doi.org/10.1016/j.jag.2020.102097
24. Piechaud, N., Hunt, C., Culverhouse, P.F., Foster, N.L., Howell, K.L., 2019. Automated identification of benthic epifauna with computer vision. Mar. Ecol. Prog. Ser. 615, 15-30. https://doi.org/10.3354/meps12925
25. Reeves, B.R., Dowty, P.R., Wyllie-Echeverria, S., Berry, H.D., 2007. Classifying the seagrass Zostera marina L. from underwater video: an assessment of sampling variation. J. Mar. Environ. Eng. 9 (1), 1-15.
26. Rimavičius, T., Gelžinis, A., Verikas, A., Vaiˇciukynas, E., Bačauskien˙e, M., Šaškov, A., 2018. Automatic benthic imagery recognition using a hierarchical two-stage approach. Signal Image Video P 12 (6), 1107-1114. https://doi.org/10.1007/s11760- 018- 1262- 4
27. Rzhanov, Y., Mayer, L., 2004. Deep-sea image processing, 647-652, Oceans ’04 MTS/IEEE Techno-Ocean ’04 (IEEE Cat. No.04CH37600). https://doi.org/10.1109/OCEANS.2004.1405498
28. Šaškov, A., Dahlgren, T.G., Rzhanov, Y., Schläppy, M.L., 2015. Comparison of manual and semi-automatic underwater imagery analyses for monitoring of benthic hard-bottom organisms at offshore renewable energy installations. Hydrobiologia 756, 139-153. https://doi.org/10.1007/s10750- 014- 2072- 5
29. Šiaulys, A., Vaiˇciukynas, E., Medelyt˙e, S., Olenin, S., Šaškov, A., Buškus, K., Verikas, A., 2021. A fully-annotated imagery dataset of sublittoral benthic species in Svalbard. Arctic. Data Br. 35, 106823. https://doi.org/10.1016/j.dib.2021.106823
30. Smith Menandro, P., Cardoso Bastos, A., 2020. Seabed mapping: A brief history from meaningful words. Geosciences 10 (7), 273. https://doi.org/10.3390/geosciences10070273
31. Tan, M., Le, Q., 2019. EfficientNet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, 6105-6114. https://doi.org/10.48550/arXiv.1905.11946
32. Tomar, S., 2006. Converting video formats with FFmpeg. Linux J. 2006 (146), 10. https://www.linuxjournal.com/article/8517
33. Trygonis, V., Sini, M., 2012. photoQuad: a dedicated seabed image processing software, and a comparative error analysis of four photoquadrat methods. J. Exp. Mar. Biol. Ecol. 424 (425), 99-108. https://doi.org/10.1016/j.jembe.2012.04.018
34. Urra, J., Palomino, D., Lozano, P., González-García, E., Farias, C., Mateo-Ramírez, Á., Fernández-Salas, L.M., López-González, N., Vila, Y., Orejas, C., Puerta, P., 2021. Deep-sea habitat characterization using acoustic data and underwater imagery in Gazul mud volcano (Gulf of Cádiz, NE Atlantic). Deep-Sea Res. Pt. I 169, 103458. https://doi.org/10.1016/j.dsr.2020.103458
35. Vaičiūt˙e, D., Bucas, M., Bresciani, M., 2012. Validation of MERIS bio-optical products with in situ data in the turbid Lithuanian Baltic Sea coastal waters. J. Appl. Remote Sens. 6 (1), 063568. https://doi.org/10.1117/1.JRS.6.063568
36. Wentworth, C.K., 1922. A scale of grade and class terms for clastic sediments. J. Geol. 30 (5), 377-392. https://doi.org/10.1086/622910
37. Yakubovskiy, P., 2019. Segmentation Models. GitHub repository.URL: https://github.com/qubvel/segmentation_models (accessed on 2 August 2023).
38. Zhao, H., Shi, J., Qi, X., Wang, X., Jia, J., 2017. Pyramid Scene Parsing Network. Proc. CVPR IEEE 6230-6239. https://doi.org/10.1109/CVPR.2017.660

Typ dokumentu

Bibliografia

Identyfikator YADDA

bwmeta1.element.baztech-01a0b4d5-8349-479e-a077-bfd8dbb161fe