Restoration of Remote Satellite Sensing Images using Machine and Deep Learning : a Survey

• Autorzy: Abdellaoui Meriem , Benabdelkader Souad , Assas Ouarda

Identyfikatory

DOI

10.22630/MGV.2023.32.2.8

• Języki publikacji: EN

Abstrakty

EN

Remote sensing satellite images are affected by different types of degradation, which poses an obstacle for remote sensing researchers to ensure a continuous and trouble-free observation of our space. This degradation can reduce the quality of information and its effect on the reliability of remote sensing research. To overcome this phenomenon, the methods of detecting and eliminating this degradation are used, which are the subject of our study. The original aim of this paper is that it proposes a state of art of recent decade (2012-2022) on advances in remote sensing image restoration using machine and deep learning, identified by this survey, including the databases used, the different categories of degradation, as well as the corresponding methods. Machine learning and deep learning based strategies for remote sensing satellite image restoration are recommended to achieve satisfactory improvements.

Słowa kluczowe

EN

image restoration; remote sensing images; artificial intelligence; AI; machine learning; ML; deep learning; DL; convolutional neural network; CNN

• Wydawca: Institute of Information Technology of the Warsaw University of Life Sciences – SGGW
• Czasopismo: Machine Graphics and Vision
• Rocznik: 2023
• Tom: Vol. 32, No. 2
• Strony: 147--167
• Opis fizyczny: Bibliogr. 57 poz., tab.

Twórcy

autor

Abdellaoui Meriem

• Electronics Department (LEA Laboratory), Faculty of Technology University of Batna 2 (Mostafa Benboulaid), Batna, Algeria

autor

Benabdelkader Souad

• Electronics Department (LEA Laboratory), Faculty of Technology University of Batna 2 (Mostafa Benboulaid), Batna, Algeria

autor

Assas Ouarda

• Electronics Department (LEA Laboratory), Faculty of Technology University of Batna 2 (Mostafa Benboulaid), Batna, Algeria

• https://orcid.org/0000-0002-3511-9488

Bibliografia

• [1] Number of satellites in space world 2022. Statista. https://www.statista.com/statistics/264472/number-of-satellites-in-orbit-by-operating-country/ (accessed Mar 11, 2022).
• [2] In 2022, space development will remain at the heart of China’s priorities. Opinion, Jan 05, 2022. https://www.Opinion.com/international/in2022 (accessed Mar 11, 2022).
• [3] NASA. GOES Satellite Network. https://science.nasa.gov/mission/goes/ (accessed Apr 09, 2022).
• [4] A. Taravat, S. Proud, S. Peronaci, F. Del Frate, and N. Oppelt. Multilayer perceptron neural networks model for meteosat second generation SEVIRI daytime cloud masking. Remote Sensing, 7(2):1529-1539, 2015. doi:10.3390/rs70201529.
• [5] M. Reguiegue and F. Chouireb. Automatic day time cloud detection over land and sea from MSG SEVIRI images using three features and two artificial intelligence approaches. Signal, Image and Video Processing, 12(1):189-196, 2018. doi:10.1007/s11760-017-1145-0.
• [6] NASA. http://www.nasa.gov (accessed Apr 09, 2022).
• [7] R. S. Williams Jr. and W. D. Carter. ERTS-1, a new window on our planet. U.S. Geological Survey, Professional Paper 929, USGS 1976. doi:10.3133/pp929.
• [8] M. J. Hughes and D. J. Hayes. Automated detection of cloud and cloud shadow in single-date Landsat imagery using neural networks and spatial post-processing. Remote Sensing, 6(6):4907-4926, 2014. doi:10.3390/rs6064907.
• [9] M. Qin, F. Xie, W. Li, Z. Shi, and H. Zhang. (2018). Dehazing for multispectral remote sensing images based on a convolutional neural network with the residual architecture. IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing, 11(5):1645-1655, 2018. doi:10.1109/JSTARS.2018.2812726.
• [10] S. Mohajerani, T. A. Krammer, and P. Saeedi. Cloud detection algorithm for remote sensing images using fully convolutional neural networks. arXiv, preprint arXiv:1810.05782, 2018. doi:10.48550/arXiv.1810.05782.
• [11] H. Jiang and N. Lu. Multi-scale residual convolutional neural network for haze removal of remote sensing images. Remote Sensing, 10(6):945, 2018. doi:10.3390/rs10060945.
• [12] Z. Shao, Y. Pan, C. Diao, and J. Cai. Cloud detection in remote sensing images based on multiscale features-convolutional neural network. IEEE Transactions on Geoscience and Remote Sensing, 57(6):4062-4076, 2019. doi:10.1109/TGRS.2018.2889677.
• [13] A. Francis, P. Sidiropoulos, and J.-P. Muller. CloudFCN: Accurate and robust cloud detection for satellite imagery with deep learning. Remote Sensing, 11(19):2312, 2019. doi:10.3390/rs11192312.
• [14] S. Ji, P. Dai, M. Lu, and Y. Zhang. Simultaneous cloud detection and removal from bitemporal remote sensing images using cascade convolutional neural networks. IEEE Transactions on Geoscience and Remote Sensing, 59(1):732-748, 2020. doi:10.1109/TGRS.2020.2994349.
• [15] Y. Zi, F. Xie, N. Zhang, Z. Jiang, W. Zhu, and H. Zhang. Thin cloud removal for multispectral remote sensing images using convolutional neural networks combined with an imaging model. IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing, 14:3811-3823, 2021. doi:10.1109/JSTARS.2021.3068166.
• [16] Q. Zhang, Q. Yuan, C. Zeng, X. Li, and Y. Wei. Missing data reconstruction in remote sensing image with a unified spatial-temporal-spectral deep convolutional neural network. IEEE Transactions on Geoscience and Remote Sensing, 56(8):4274-4288, 2018. doi:10.1109/TGRS.2018.2810208.
• [17] S. R. Surya and P. Simon. Automatic cloud detection using spectral rationing and fuzzy clustering. Proc. 2013 2nd International Conference on Advanced Computing, Networking and Security, Mangalore, India, 2013, pp. 90-95. doi:10.1109/ADCONS.2013.44.
• [18] J. Yang, J. Guo, H. Yue, Z. Liu, H. Hu, and K. Li. CDnet: CNN-based cloud detection for remote sensing imagery. IEEE Transactions on Geoscience and Remote Sensing, 57(8):6195-6211, 2019. doi:10.1109/TGRS.2019.2904868.
• [19] D. Chai, S. Newsam, H. K. Zhang, Y. Qiu, and J. Huang. Cloud and cloud shadow detection in Landsat imagery based on deep convolutional neural networks. Remote sensing of environment, 225):307-316, 2019. doi:10.1016/j.rse.2019.03.007.
• [20] J. H. Jeppesen, R. H. Jacobsen, F. Inceoglu, and T. S. Toftegaard. A cloud detection algorithm for satellite imagery based on deep learning. Remote sensing of environment, 229:247-259, 2019. doi:https://doi.org/10.1016/j.rse.2019.03.039.
• [21] M. Shi, F. Xie, Y. Zi, and J. Yin. Cloud detection of remote sensing images by deep learning. Proc. 2016 IEEE International Geoscience and Remote Sensing Symposium (IGARSS), Beijing, China, 2016, pp. 701-704. doi:10.1109/IGARSS.2016.7729176.
• [22] Z. Wang, Z. Zhang, L. Dong, and G. Xu. Jitter detection and image restoration based on generative adversarial networks in satellite images. Sensors, 21(14):4693, 2021. doi:10.3390/s21144693.
• [23] F. Xie, M. Shi, Z. Shi, J. Yin, and D. Zhao. Multilevel cloud detection in remote sensing images based on deep learning. IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing, 10(8):3631-3640, 2017. doi:10.1109/JSTARS.2017.2686488.
• [24] CNES - The National Center for Space Studies. https://cnes.fr/en/ (accessed Apr 09, 2022).
• [25] COPERNICUS Programme. Sentinel Online. https://sentinels.copernicus.eu/web/sentinel/ (accessed Apr 09, 2022).
• [26] EUMETSAT - European Organisation for the Exploitation of Meteorological Satellites. Meteosat series. https://www.eumetsat.int/our-satellites/meteosat-series (accessed Apr 09, 2022).
• [27] M. Le Goff, J.-Y. Tourneret, H. Wendt, M. Ortner, and M. Spigai. Deep learning for cloud detection. Proc. 8th International Conference of Pattern Recognition Systems (ICPRS 2017), Madrid, 2017, pp. 1-6, 2017. doi:10.1049/cp.2017.0139.
• [28] N. Champion. Automatic cloud detection from multi-temporal satellite images: Towards the use of pléiades time series. International Archives of the Photogrammetry, Remote Sensing and Spatial Information Sciences, 24(B3:559-564, 2012. doi:10.5194/isprsarchives-XXXIX-B3-559-2012.
• [29] Q. Zhang, Q. Yuan, J. Li, Z. Li, H. Shen, and L. Zhang. Thick cloud and cloud shadow removal in multitemporal imagery using progressively spatio-temporal patch group deep learning. ISPRS Journal of Photogrammetry and Remote Sensing, 162:148-160, 2020. doi:10.1016/j.isprsjprs.2020.02.008.
• [30] L. Wang, Y. Chen, L. Tang, R. Fan, and Y. Yao. Object-based convolutional neural networks for cloud and snow detection in high-resolution multispectral imagers. Water, 10(11):1666, 2018. doi:10.3390/w10111666.
• [31] P. Ebel, Y. Xu, M. Schmitt, and X. Zhu. SEN12MS-CR-TS: A remote sensing data set for multi-modal multi-temporal cloud removal. IEEE Transactions on Geoscience and Remote Sensing, 60:5222414, 2022. doi:10.1109/TGRS.2022.3146246.
• [32] K. Kasturirangan and V. Jayaraman. Indian Remote Sensing Satellite, IRS-1A. A forerunner for operational era. https://www.isro.gov.in/Indian_Remote_Sensing_Satellite_1A.html (accessed Apr 09, 2022).
• [33] Y. Chen, R. Fan, M. Bilal, X. Yang, J. Wang, and W. Li. Multilevel cloud detection for high-resolution remote sensing imagery using multiple convolutional neural networks. ISPRS International Journal of Geo-Information, 7(5):181, 2018. doi:10.3390/ijgi7050181.
• [34] Z. Yan et al. Cloud and cloud shadow detection using multilevel feature fused segmentation network. IEEE Geoscience and Remote Sensing Letters, 15(10):1600-1604, 2018. doi:10.1109/LGRS.2018.2846802.
• [35] T. Bai, D. Li, K. Sun, Y. Chen, and W. Li. Cloud detection for high-resolution satellite imagery using machine learning and multi-feature fusion. Remote Sensing, 8(9):715, 2016. doi:10.3390/rs8090715.
• [36] B. Li et al. Stripe noise detection of high-resolution remote sensing images using deep learning method. Remote Sensing, 14(4):873, 2022. doi:10.3390/rs14040873.
• [37] Y. Chen et al. Cloud and cloud shadow detection based on multiscale 3D-CNN for high resolution multispectral imagery. IEEE Access, 8:16505-16516, 2020. doi:10.1109/ACCESS.2020.2967590.
• [38] Y. Chen, L. Tang, X. Yang, R. Fan, M. Bilal, and Q. Li. Thick clouds removal from multitemporal ZY-3 satellite images using deep learning. IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing, 13:143-153, 2019. doi:10.1109/JSTARS.2019.2954130.
• [39] B. Jiang et al. Deep Dehazing Network for Remote Sensing Image with Non-Uniform Haze. Remote Sensing, 13(21):4443, 2021. doi:10.3390/rs13214443.
• [40] T. Julliand, V. Nozick, and H. Talbot. Image noise and digital image forensics. Proc. International Workshop on Digital-Forensics and Watermarking, Tokyo, Japan, Oct 7-10, 2015. Lecture Notes in Computer Science, vol. 9569, pp. 3-17. doi:10.1007/978-3-319-31960-5 1.
• [41] K.-Y. Lee and J.-Y. Sim. (2019). Cloud removal of satellite images using convolutional neural network with reliable cloudy image synthesis model. Proc. 2019 IEEE International Conference on Image Processing (ICIP), Taipei, Taiwan, 2019, pp. 3581-3585. doi:10.1109/ICIP.2019.8803666.
• [42] S. Shao, Y. Guo, Z. Zhang, and H. Yuan. Single remote sensing multispectral image dehazing based on a learning framework. Mathematical Problems in Engineering, 2019:4131378, 2019. doi:10.1155/2019/4131378.
• [43] Y. Chang, L. Yan, H. Fang, S. Zhong, and W. Liao. HSI-DeNet: Hyperspectral image restoration via convolutional neural network. IEEE Transactions on Geoscience and Remote Sensing, 57(2):667-682, 2019. doi:10.1109/TGRS.2018.2859203.
• [44] H. Zeng, X. Xie, H. Cui, Y. Zhao, and J. Ning. Hyperspectral image restoration via CNN denoiser prior regularized low-rank tensor recovery. Computer Vision and Image Understanding, 197-198:103004, 2020. doi:10.1016/j.cviu.2020.103004.
• [45] S. H. Khan, M. Bennamoun, F. Sohel, and R. Togneri. Automatic shadow detection and removal from a single image. IEEE Transactions on Pattern Analysis and Machine Intelligence, 38(3):431-446, 2015. doi:10.1109/TPAMI.2015.2462355.
• [46] J. Tang et al.. RestoreNet-Plus: Image restoration via deep learning in optical synthetic aperture imaging system. Optics and Lasers in Engineering, 146:106707, 2021. doi:10.1016/j.optlaseng.2021.106707.
• [47] A. Tafsast, M. L. Hadjili, H. Hafdaoui, A. Bouakaz and N. Benoudjit. Automatic Gaussian mixture model (GMM) for segmenting 18F-FDG-PET images based on Akaike information criteria. Proc. 2015 4th International Conference on Electrical Engineering (ICEE), Boumerdes, Algeria, 2015, pp. 1-4, 2015. doi:10.1109/INTEE.2015.7416845.
• [48] S. Mahajan and B. Fataniya. Cloud detection methodologies: Variants and development-A review. Complex & Intelligent Systems, 6:251-261, 2020. doi:10.1007/s40747-019-00128-0.
• [49] IBM. What is AI? https://www.ibm.com/topics/artificial-intelligence (accessed Apr 09, 2022).
• [50] I. El Naqa, and M. J. Murphy. What Are Machine and Deep Learning?. In: El Naqa, I., Murphy, M. J. (eds.) Machine and Deep Learning in Oncology, Medical Physics and Radiology. Springer, Cham, 2022, pp. 3-15. doi:10.1007/978-3-030-83047-2 1.
• [51] J. Yang, Q. Min, W. Lu, Y. Ma, W. Yao, T. Lu, J. Du, and G. Liu. A total sky cloud detection method using real clear sky background. Atmospheric Measurement Techniques, 9(2):587-597, 2016. doi:10.5194/amt-9-587-2016.
• [52] B. Zhong, W. Chen, S. Wu, L. Hu, X. Luo and Q. Liu. A cloud detection method based on relationship between objects of cloud and cloud-shadow for Chinese moderate to high resolution satellite imagery. IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing, 10(11):4898-4908, 2017. doi:10.1109/JSTARS.2017.2734912.
• [53] J. Yang, W. Lu, Y. Ma, and W. Yao. An automated cirrus cloud detection method for a ground-based cloud image. Journal of Atmospheric and Oceanic Technology, 29:527-537, 2012. doi:10.1175/JTECH-D-11-00002.1.
• [54] H. Zhai, H. Zhang, L. Zhang, and P. Li. Cloud/shadow detection based on spectral indices for multi/hyperspectral optical remote sensing imagery. ISPRS Journal of Photogrammetry and Remote Sensing, 144:235-253, 2018. doi:10.1016/j.isprsjprs.2018.07.006.
• [55] H. Shang, H. Letu, Z. Peng, and Z. Wang. Development of a daytime cloud and aerosol loadings detection algorithm for himawari-8 satellite measurements over desert. Proc. ISPRS Workshop on Remote Sensing and Synergic Analysis on Atmospheric Environment, 7-8 Nov 2018, Guangzhou, China. The International Archives of the Photogrammetry, Remote Sensing and Spatial Information Sciences, Vol. XLII-3/W5, pp. 61-66, 2018. doi:10.5194/isprs-archives-XLII-3-W5-61-2018.
• [56] D. Frantz, E. Ha, A. Uhl, J. Stofels, and J. Hill. Improvement of the Fmask algorithm for Sentinel-2 images: separating clouds from bright surfaces based on parallax effects. Remote Sensing of Environment, 215:471-481, 2018. doi:10.1016/j.rse.2018.04.046.
• [57] P. Wang, H. Zhang and V. M. Patel. SAR image despeckling using a Convolutional Neural Network. IEEE Signal Processing Letters, 24(12):1763-1767, 2017. doi:10.1109/LSP.2017.2758203.

Uwagi

Opracowanie rekordu ze środków MNiSW, umowa nr SONP/SP/546092/2022 w ramach programu "Społeczna odpowiedzialność nauki" - moduł: Popularyzacja nauki i promocja sportu (2024).