Volcanic disaster scene classification of remote sensing image based on deep multi-instance network

• Autorzy: Li Chengfan , Han Jingxin , Wu Chengzhi , Liu Lan , Liu Xuefeng , Zhao Junjuan

Identyfikatory

DOI

10.1007/s11600-024-01394-4

• Języki publikacji: EN

Abstrakty

EN

Due to the varieties, random distributions, and rich visual characteristics of the volcanic disaster scene, traditional methods fail to fully express the complex features of volcanic disaster scenes in remote sensing images. To tackle this problem, a new multi-instance network framework with the Shift Windows Transformer (i.e., Swin-T) and attention mechanism is used to classify the volcanic disaster scene from remote sensing images (MI-STA). Firstly, via aggregating the global contextual information of remote sensing image features, the Swin-T extracts the multi-scale hierarchical features of volcano disaster scenes from remote sensing images. Secondly, the channel attention module and spatial attention module fuse to extract the features of volcanic disaster scene to enhance the description and representation for the local details and global informa- tion in volcanic disaster scenes. Last, the importance weight of different example characteristics is scored to calculate the attributive probabilities of each instance. This study elaborates an experiment on the xBD dataset and gives comparisons with the commonly used deep network models. The results show that the overall classification accuracy of the proposed method achieves 92.46% and has good performance on the test dataset. Then, we further utilize our model to classify the volcanic disaster scenes of the specific Hunga Tonga-Hunga Ha’apai on January 15, 2022, and the classification images have good consistency with the existing literature. It provides a new approach for volcanic disaster monitoring by means of remote sensing image and has broad application prospects.

Słowa kluczowe

EN

volcanic disaster; remote sensing image; deep neural network; multi-instance learning; scene classification

PL

katastrofa wulkaniczna; obraz teledetekcyjny; głęboka sieć neuronowa; uczenie wieloinstancyjne

• Wydawca: Instytut Geofizyki PAN ; Springer
• Czasopismo: Acta Geophysica
• Rocznik: 2025
• Tom: Vol. 73, no. 1
• Strony: 25--41
• Opis fizyczny: Bibliogr. 47 poz.

Twórcy

autor

Li Chengfan

• School of Computer Engineering and Science, Shanghai University, Shanghai 200444, China

autor

Han Jingxin

• School of Computer Engineering and Science, Shanghai University, Shanghai 200444, China

autor

Wu Chengzhi

• Changbai Mountain Tianchi Volcano Observatory, Antu 133613, China

autor

Liu Lan

• School of Electronic and Electrical Engineering, Shanghai University of Engineering Science, Shanghai 201620, China

autor

Liu Xuefeng

• School of Communication and Information Engineering, Shanghai University, Shanghai 200444, China

autor

Zhao Junjuan

• School of Computer Engineering and Science, Shanghai University, Shanghai 200444, China

Bibliografia

• 1. Adam D (2022) Tonga volcano eruption created puzzling ripples in Earth’s atmosphere. Nature 601(7894):497. https://doi.org/10. 1038/d41586-022-00127-1
• 2. Alcaras E, Amoroso PP, Falchi U (2022) Using bi-temporal Senti- nel-2 images to detect the effects of Hunga Tonga-Hunga Ha’apai eruption. In: Proceedings of the IEEE International Workshop on Metrology for the Sea Learning to Measure Sea Health Parameters (MetroSea), 3-5 October 2022, Milazzo, pp 127-132. https://doi. org/10.1109/METROSEA55331.2022.9950808
• 3. Amores J (2013) Multiple instance classification: review, taxonomy and comparative study. Artif Intell 201(4):81-105
• 4. Bohnenstiehl DWR, Dziak RP, Matsumoto H, Lau TKA (2013) Underwater acoustic records from the March 2009 eruption of Hunga Haa’pai-Hunga Tonga Volcano in the Kingdom of Tonga.
• 5. J Volcanol Geoth Res 249(1):12-24. https://doi.Org/10.1016/j. jvolgeores.2012.08.014
• 6. Cao H, Wang YY, Chen J, Jiang DS, Zhang XP, Tian Q, Wang MN (2022) Swin-Unet: Unet-like pure transformer for medical image segmentation. In: Proceedings of the 2022 European Conference on Computer Vision, Tel-Aviv, Springer, 23-27 October 2022, pp 205-218. https://doi.org/10.48550/arXiv.2105.05537
• 7. Chaib S, Liu H, Gu YF, Yao HX (2017) Deep feature fusion for VHR remote sensing scene classification. IEEE T Geosci Remote 55(8):4775-4784. https://doi.org/10.1109/TGRS.2017.2700322
• 8. Cheng MM, Liu Y, Lin WY, Zhang ZM, Rosin PL, Torr PHS (2019) BING: binarized normed gradients for objectness estimation at 300fps. Comput Visual Media 5(1):3286-3293
• 9. Dai C, Cao NW, Yang SB (2019) Analysis of the effects on low-level atmosphere by volcanic disasters in Hawaii based on multi-source data. J Nat Disasters 28(5):160-168
• 10. Dietterich TG, Lathrop RH, Lozano-Perez T (1997) Solving the multiple instance problem with axisparallel rectangles. Artif Intell 89(1-2):31-71
• 11. Duan Z, Zhang X, Chen R, Yu MG, Chu PL, Hong WT, Cao MX (2022) Advances on volcano monitoring and its implications for volcanic hazards monitoring and early warning in China. E China Geol 43(4):391-414
• 12. Franklin SE, Hall RJ, Moskal LM, Maudie AJ, Lavigne MB (2000) Incorporating texture into classification of forest species com- position from airborne multispectral images. Int J Remote Sens 21(1):61-79. https://doi.org/10.1080/014311600210993
• 13. Garvin JB, Slayback DA, Ferrini V, Frawley J, Giguere C, Asrar GR, Andersen K (2018) Monitoring and modeling the rapid evolution of earth’s newest volcanic island: Hunga Tonga Hunga Ha’apai (Tonga) using high spatial resolution satellite observations. Geo- phys Res Lett 45(8):3445-3452. https://doi.org/10.1002/2017g l076621
• 14. Gupta R, Hosfelt R, Sajeev S, Patel N, Goodman B, Doshi J, Heim E, Choset H, Gaston M (2019) Creating xBD: a dataset for assessing building damage from satellite imagery. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition Workshops. Long Beach: IEEE, pp 10-17. https://doi.org/10. 48550/arXiv.1911.09296
• 15. He KL, Shi YH, Gao Y, Huo J, Wang D, Zhang Y (2017) A prototype learning based on multi-instance convolutional neural network. Chin J Comput 40(6):1265-1274
• 16. He XY, Xu WM, Pan KX, Wang J, Li ZW (2024) Classification of high-resolution remote sensing image based on swin trans- former and convolutional neural network. Laser Optoelectron P 61(16):1628002. https://doi.org/10.3788/LOP232003
• 17. Hu YF, Li ZH, Wang L, Chen B, Zhu W, Zhang SC, Du JT, Zhang XS, Yang J, Zhou ML, Liu ZJ, Wang SS, Miao C, Zhang LZ, Peng JB (2022) Rapid interpretation and analysis of the 2022 eruption of Hunga Tonga-Hunga Ha’apai Volcano with integrated remote sensing techniques. Geomat Inform Sci Wuhan Univ 47(2):242-251
• 18. Khan N, Chaudhuri U, Banerjee B, Chaudhuri S (2019) Graph convolutional network for multi-label VHR remote sensing scene recognition. Neurocomputing 357(10):3646. https://doi.org/10. 1016/j.neucom.2019.05.024
• 19. Lee SC, Kang NH, Park ML, Hwang J, Yeon Y (2018) A review on volcanic gas compositions related to volcanic activities and non- volcanological effects. Geosci J 22(1):183-197
• 20. Li ZL, Xu K, Xie JF, Bi Q, Qin K (2020) Deep multiple instance convolutional neural networks for learning robust scene representations. IEEE T Geosci Remote 58(5):3685-3702. https://doi.org/10.1109/ TGRS.2019.2960889
• 21. Liu L, Sun XK (2020) Volcanic ash cloud diffusion from remote sensing image using LSTM-CA method. IEEE ACCESS 8(1):54681- 54690. https://doi.org/10.1109/ACCESS.2020.2981368
• 22. Liu JQ, Wang SS (1982) Formation era of Changbaishan volcano and Tianchi lake. Chin Sci Bull 27(21):1312-1315. https://doi.org/10. 1360/csb1982-27-21-1312
• 23. Liu XN, Zhou Y, Zhao JQ, Yao R, Bing L, Zheng Y (2019) Siamese convolutional neural networks for remote sensing scene classification. IEEE Geosci Remote S 16(8):1200-1204. https://doi.org/10. 1109/LGRS.2019.2894399
• 24. Liu Z, Lin YT, Cao Y, Hu H, Wei YX, Zhang Z, Lin S, Guo BN (2021) Swin transformer: hierarchical vision transformer using shifted windows. In: Proceedings of the IEEE International Conference on Computer Vision, 11-17 October, Virtually, pp 10012-10022. https://doi.org/10.48550/arXiv.2103.14030
• 25. Mallapaty S (2022) How the Tonga eruption is helping space scientists understand Mars. Nature. https://doi. org/10. 1038/ d41586-022-00137-z
• 26. Menghani G (2023) Efficient deep learning: a survey on making deep learning models smaller, faster, and better. ACM Comput Surv 55(12):259
• 27. Nam K, Wang F, Yan K, Zhu GL (2023) Characteristics and time-series monitoring by GOES-17 of volcano flume on 15 January 2022 from Tonga submarine volcanic eruption. Geoenviron Disasters 10:2. https://doi.org/10.1186/s40677-023-00232-x
• 28. Pandey PC (2022) Highlighting the role of earth observation Sentinel5P TROPOMI in monitoring volcanic eruptions: a report on Hunga Tonga, a Submarine Volcano. Remote Sens Lett 13(9):912-923. https://doi.org/10.1080/2150704X.2022.2106799
• 29. Poli P, Shapiro NM (2022) Rapid characterization of large volcanic eruptions: measuring the impulse of the Hunga Tonga Ha’apai explosion from teleseismic waves. Geophys Res Lett. https://doi. org/10.1029/2022GL098123
• 30. Self S (2006) The effects and consequences of very large explosive volcanic eruptions. Philos T R Soc A 364(1845):2073-2097. https:// doi.org/10.1098/rsta.2006.1814
• 31. Sparks RSJ (2003) Forecasting volcanic eruptions. Earth Planet Sc Lett 210(1-2):1-15. https://doi.org/10.1016/S0012-821X(03)00124-9
• 32. Sun KX, Liu B, Su SG (2023) Medical mcroscopic image segmentation model based on CNN structure and Swin transformer. Comput Sci 50(11):230200119
• 33. Valentijn T, Margutti J, Homberg MVD, Laaksonen J (2020) Multi- hazard and spatial transferability of a CNN for automated building damage assessment. Remote Sens-Basel 12(17):2839. https://doi. org/10.3390/rs12172839
• 34. Vaughan RG, Webley PW (2010) Satellite observations of a surtseyan eruption: Hunga Ha’apai. Tonga J Volcanol Geoth Res 198(1- 2):177-186. https://doi.org/10.1016/j.jvolgeores.2010.08.017
• 35. Wang XG, Yan YL, Tang P, Bai X, Liu WY (2018) Revisiting multiple instance neural networks. Pattern Recog 74:15-24. https://doi.org/ 10.1016/j.patcog.2017.08.026
• 36. Wei SQ, Ji GL, Xu Z, Liu YJ (2022) Attention mechanism based multiple instance learning video anomaly detection. J Chin Comput Syst 43(12):2575-2579
• 37. Wicaksono W, Isa SM (2022) Predicting the extent of Sidoarjo mud flow using remote sensing. J ICT Res Appl 16(1):56-69
• 38. Witze A (2022) Why the Tongan eruption will go down in the history of volcanology. Nature 602(7897):376-378. https://doi.org/10. 1038/d41586-022-00394-y
• 39. Xu YG, Guo ZF, Liu JQ (2020) Studies on Volcanology and chemistry of the Earth’s interior in China: progresses and perspectives (2011-2020). Bull Miner Petrol Geochem 39(4):683-696
• 40. Xu JD, Wan Y, Wang XR, Pan B, Yu HM, Zhao B, Yang WJ (2022) Review on the development of volcanic hazard zonation in China. Geol Resour 31(3):426-433
• 41. Xuan JY, Lu J, Yan Z, Zhang G (2019) Bayesian deep reinforcement learning via deep kernel learning. Int J Comput Int Sys 12(1):164- 171. https://doi.org/10.2991/ijcis.2018.25905189
• 42. Zhang LP, Huang X, Huang B, Li PX (2006) A pixel shape index coupled with spectral information for classification of high spatial resolution remotely sensed imagery. IEEE T Geosci Remote 44(10):2950-2961. https://doi.org/10.1109/TGRS.2006.876704
• 43. Zhang SY, Tan XL, Zhu ML, Yang WL (2023) Landslide segmentation algorithm based on improved Swin Transformer. Foreign Electron Meas Technol 42(11):49-56
• 44. Zhang Y, Liu H, Hu Q (2021) TransFuse: fusing transformers and CNNs for medical image segmentation. In: Proceedings of the International Conference on Medical Image Computing and Computer-Assisted Intervention, Springer, Strasbourg, September 27- October 1, pp 14-24. https://doi.org/10.48550/arXiv.2102.08005
• 45. Zhao WB, Sun CQ, Guo ZF (2022) Reawaking of Tonga volcano.
• 46. Innov 3(2):100218. https://doi.org/10.1016/j.xinn.2022.100218
• 47. Zheng TT, Qiu Q, Lin J (2023) Advanced observation of the 2022 Tonga volcanic eruption and the associated tsunamis. Sci Technol Rev 41(2):44-50

Uwagi

Opracowanie rekordu ze środków MNiSW, umowa nr POPUL/SP/0154/2024/02 w ramach programu "Społeczna odpowiedzialność nauki II" - moduł: Popularyzacja nauki (2025).