An AI & ML based detection & identification in remote imagery : state-of-the-art

• Autorzy: Hashmi Hina , Dwivedi Rakesh , Kumar Anil

Identyfikatory

DOI

10.14313/JAMRIS/4-2021/22

• Języki publikacji: EN

Abstrakty

EN

Remotely sensed images and their allied areas of application have been the charm for a long time among researchers. Remote imagery has a vast area in which it is serving and achieving milestones. From the past, after the advent of AL, ML, and DL-based computing, remote imagery is related techniques for processing and analyzing are continuously growing and offering countless services like traffic surveillance, earth observation, land surveying, and other agricultural areas. As Artificial intelligence has become the charm of researchers, machine learning and deep learning have been proven as the most commonly used and highly effective techniques for object detection. AI & ML-based object segmentation & detection makes this area hot and fond to the researchers again with the opportunities of enhanced accuracy in the same. Several researchers have been proposed their works in the form of research papers to highlight the effectiveness of using remotely sensed imagery for commercial purposes. In this article, we have discussed the concept of remote imagery with some preprocessing techniques to extract hidden and fruitful information from them. Deep learning techniques applied by various researchers along with object detection, object recognition are also discussed here. This literature survey is also included a chronological review of work done related to detection and recognition using deep learning techniques.

Słowa kluczowe

EN

convolutional neural network; remote sensed imagery; object detection; artificial intelligence; feature extraction; deep learning; machine learning

• Wydawca: Łukasiewicz Industrial Research Institute for Automation and Measurements PIAP
• Czasopismo: Journal of Automation Mobile Robotics and Intelligent Systems
• Rocznik: 2021
• Tom: Vol. 15, No. 4
• Strony: 3--17
• Opis fizyczny: Bibliogr. 69 poz., rys.

Twórcy

autor

Hashmi Hina

• CCSIT, Teerthanker Mahaveer University, Moradabad, India

autor

Dwivedi Rakesh

• CCSIT, Teerthanker Mahaveer University, Moradabad, India

autor

Kumar Anil

• Indian Institute of Remote Sensing, Dehradun, India

Bibliografia

• [1] Q. Yao, X. Hu and H. Lei, “Multiscale Convolutional Neural Networks for Geospatial Object Detection in VHR Satellite Images”, IEEE Geoscience and Remote Sensing Letters, vol. 18, no. 1, 2021, 23–27, 10.1109/LGRS.2020.2967819.
• [2] G. Mountrakis, J. Im and C. Ogole, “Support vector machines in remote sensing: A review”, ISPRS Journal of Photogrammetry and Remote Sensing, vol. 66, no. 3, 2011, 247–259, 10.1016/j.isprsjprs.2010.11.001.
• [3] X. Yao, X. Feng, J. Han, G. Cheng and L. Guo, “Automatic Weakly Supervised Object Detection From High Spatial Resolution Remote Sensing mages via Dynamic Curriculum Learning”, IEEE Transactions on Geoscience and Remote Sensing, vol. 59, no. 1, 2021, 675–685, 10.1109/TGRS.2020.2991407.
• [4] L. Li, Z. Zhou, B. Wang, L. Miao and H. Zong, “ANovel CNN-Based Method for Accurate Ship Detection in HR Optical Remote Sensing Images via Rotated Bounding Box”, IEEE Transactions on Geoscience and Remote Sensing, vol. 59, no. 1, 2021, 686–699, 10.1109/TGRS.2020.2995477.
• [5] T. Matsuyama, “Knowledge-Based Aerial Image Understanding Systems and Expert Systems for Image Processing”, IEEE Transactions on Geoscience and Remote Sensing, vol. GE-25, no. 3, 1987, 305–316, 10.1109/TGRS.1987.289802.
• [6] P. Garnesson, G. Giraudon and P. Montesinos, “An image analysis, application for aerial imagery interpretation”. In: Proc. 10th International Conference on Pattern Recognition, vol. 1, 1990, 210–212, 10.1109/ICPR.1990.118094.
• [7] D. Ionescu and G. Geling, “Automatic detection of large object features from SAR data”. In: Proc. IGARSS ‘93 - IEEE International Geoscience and Remote Sensing Symposium, 1993, 1225–1227, 10.1109/IGARSS.1993.322663.
• [8] P. R. Coppin and M. E. Bauer, “Processing of multitemporal Landsat TM imagery to optimize extraction of forest cover change features”, IEEE Transactions on Geoscience and Remote Sensing, vol. 32, no. 4, 1994, 918–927, 0.1109/36.298020.
• [9] D. P. Mandal, C. A. Murthy and S. K. Pal, “Analysis of IRS imagery for detecting man-made objects with a multivalued recognition system”, IEEE Transactions on Systems, Man, and Cybernetics -Part A: Systems and Humans, vol. 26, no. 2, 1996, 241–247, 10.1109/3468.485750.
• [10] Hsuan Ren and Chein-I Chang, “A computer--aided detection and classification method for concealed targets in hyperspectral imagery”. In: IGARSS ‘98. Sensing and Managing the Environment. 1998 IEEE International Geoscience and Remote Sensing. Symposium Proceedings. 1998, 1016–1018, 10.1109/IGARSS.1998.699658.
• [11] J. A. Shufelt, “Performance evaluation and analysis of monocular building extraction from aerial imagery”, IEEE Transactions on Pattern Analysis and Machine Intelligence, vol. 21, no. 4, 1999, 311–326, 10.1109/34.761262.
• [12] J. G. Shanks and B. V. Shetler, “Confronting clouds: detection, remediation and simulation approaches for hyperspectral remote sensing systems”. In: Proc. 29th Applied Imagery PatternRecognition Workshop, 2000, 25–31, 10.1109/AIPRW.2000.953599.
• [13] T. L. Haithcoat, W. Song and J. D. Hipple, “Building footprint extraction and 3-D reconstruction from LIDAR data”. In: IEEE/ISPRS Joint Workshop on Remote Sensing and Data Fusion over Urban Areas, 2001, 74–78, 10.1109/DFUA.2001.985730.
• [14] K. Chen and R. Blong, “Extracting building features from high resolution aerial imagery for natural hazards risk assessment”. In: IEEE International Geoscience and Remote Sensing Symposium, vol. 4, 2002, 2039–2041, 10.1109/IGARSS.2002.1026437.
• [15] J. Duan, V. Prinet and H. Lu, “Building extraction in urban areas from satellite images using GIS data as prior information”. In: Proc. IEEE International Geoscience and Remote Sensing Symposium, 2004. IGARSS ‘04., vol. 7, 2004, 4762–4764, 10.1109/IGARSS.2004.1370223.
• [16] Yan Dongmei, Zhao Zhongming and Chen Zhong, “A fused road detection approach in high resolution multi-spectrum remote sensing imagery”. In: Proc. 2005 IEEE International Geoscience and Remote Sensing Symposium, 2005. IGARSS ‘05., vol. 3, 2005, 1557–1560, 10.1109/IGARSS.2005.1526290.
• [17] J. Secord and A. Zakhor, “Tree Detection in Urban Regions Using Aerial Lidar and Image Data”, IEEE Geoscience and Remote Sensing Letters, vol. 4, no. 2, 2007, 196–200, 10.1109/LGRS.2006.888107.
• [18] D. Chaudhuri and A. Samal, “An Automatic Bridge Detection Technique for Multispectral Images”, IEEE Transactions on Geoscience and Remote Sensing, vol. 46, no. 9, 2008, 2720–2727, 10.1109/TGRS.2008.923631.
• [19] R. L. Paes, J. A. Lorenzzetti and D. F. M. Gherardi, “Ship Detection Using TerraSAR-X Images in the Campos Basin (Brazil)”, IEEE Geoscience and Remote Sensing Letters, vol. 7, no. 3, 2010, 545–548, 10.1109/LGRS.2010.2041322.
• [20] C. S. Grant, T. K. Moon, J. H. Gunther, M. R. Stites and G. P. Williams, “Detection of Amorphously Shaped Objects Using Spatial Information Detection Enhancement (SIDE)”, IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing, vol. 5, no. 2, 2012, 478–487, 10.1109/JSTARS.2012.2186284.
• [21] I. Sevo and A. Avramovic, “Convolutional Neural Network Based Automatic Object Detection on Aerial Images”, IEEE Geoscience and Remote Sensing Letters, vol. 13, no. 5, 2016, 740–744, 10.1109/LGRS.2016.2542358.
• [22] Z. Deng, L. Lei, H. Sun, H. Zou, S. Zhou and J. Zhao, “An enhanced deep convolutional neural network for densely packed objects detection in remote sensing images”. In: 2017 International Workshop on Remote Sensing with Intelligent Processing (RSIP), 2017, 1–4, 10.1109/RSIP.2017.7958800.
• [23] A. Farooq, J. Hu and X. Jia, “Efficient object proposals extraction for target detection in VHR remote sensing images”. In: 2017 IEEE International Geoscience and Remote Sensing Symposium (IGARSS), 2017, 3337–3340, 10.1109/IGARSS.2017.8127712.
• [24] Y. Li, L. Jiao, X. Tang, X. Zhang, W. Zhang and L. Gao, “Weak Moving Object Detection In Optical Remote Sensing Video With Motion-Drive usion Network”. In: IGARSS 2019 - 2019 IEEE International Geoscience and Remote Sensing Symposium, 2019, 5476–5479, 10.1109/IGARSS.2019.8900412.
• [25] B. Sui, M. Xu and F. Gao, “Patch-Based Three-Stage Aggregation Network for Object Detection inHigh Resolution Remote Sensing Images”, IEEE Access, vol. 8, 2020, 184934–184944, 10.1109/ACCESS.2020.3027044.
• [26] J. Guo, J. Yang, H. Yue, H. Tan, C. Hou and K. Li,“CDnetV2: CNN-Based Cloud Detection for Remote Sensing Imagery With Cloud-Snow Coexistence”, IEEE Transactions on Geoscience andRemote Sensing, vol. 59, no. 1, 2021, 700–713, 10.1109/TGRS.2020.2991398.
• [27] Y. Han, S. Ma, Y. Xu, L. He, S. Li and M. Zhu, “Effective Complex Airport Object Detection in Remote Sensing Images Based on Improved End--to-End Convolutional Neural Network”, IEEE Access, vol. 8, 2020, 172652–172663, 10.1109/ACCESS.2020.3021895.
• [28] J. Yang, J. Guo, H. Yue, Z. Liu, H. Hu and K. Li, “CDnet: CNN-Based Cloud Detection for Remote Sensing Imagery”, IEEE Transactions on Geo-science and Remote Sensing, vol. 57, no. 8, 2019, 6195–6211, 10.1109/TGRS.2019.2904868.
• [29] Z. Rao, M. He and Y. Dai, “Class Attention Network for Semantic Segmentation of Remote Sensing Images”, 2020 Asia-Pacific Signal and Information Processing Association Annual Summit and Conference (APSIPA ASC), 2020, 150–155.
• [30] P. Sharma, N. Ding, S. Goodman and R. Soricut, “Conceptual Captions: A Cleaned, Hypernymed, Image Alt-text Dataset for Automatic Image Captioning”. In: Proceedings of the 56th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), 2018, 2556–2565, 10.18653/v1/P18-1238.
• [31] A. Sisodia, Swati and H. Hashmi, “Incorporation of Non-Fictional Applications in Wireless Sensor Networks”, International Journal of Innovative Technology and Exploring Engineering, vol. 9, no. 11, 2020, 42–49, 10.35940/ijitee.K7673.0991120.
• [32] H. Hashmi, R. K. Dwivedi and A. Kumar, “Identification of Objects using AI & ML Approaches: State-of-the-Art”. In: 2021 10th International Conference on System Modeling & Advancement in Research Trends (SMART), 2021, 1–5, 10.1109/SMART52563.2021.9676273.
• [33] A. Kumar, H. Hashmi, S. A. Khan and S. Kazim Naqvi, “SSE: A Smart Framework for Live Video Streaming based Alerting System”. In: 2021 10th International Conference on System Modeling & Advancement in Research Trends (SMART), 2021, 193–197, 10.1109/SMART52563.2021.9675306.
• [34] S. Belongie, J. Malik and J. Puzicha, “Shape matching and object recognition using shape contexts”, IEEE Transactions on Pattern Analysis and Machine Intelligence, vol. 24, no. 4, 2002, 509–522, 10.1109/34.993558.
• [35] D. Yu, H. Guo, Q. Xu, J. Lu, C. Zhao and Y. Lin, “Hierarchical Attention and Bilinear Fusion for Remote Sensing Image Scene Classification”, IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing, vol. 13, 2020, 6372–6383, 10.1109/JSTARS.2020.3030257.
• [36] D. Xue, T. Lei, X. Jia, X. Wang, T. Chen and A. K. Nandi, “Unsupervised Change Detection Using Multiscale and Multiresolution Gaussian-Mixture--Model Guided by Saliency Enhancement”, IEEE ournal of Selected Topics in Applied Earth Observations and Remote Sensing, vol. 14, 2021, 1796–1809, 10.1109/JSTARS.2020.3046838.
• [37] P. Peer and B. Batagelj, “Computer vision in contemporary art (introduction to the special session)”, 2008 50th International Symposium ELMAR, vol. 2, 2008, 471–474.
• [38] R. M. Haralick and G. L. Kelly, “Pattern recognition with measurement space and spatial clustering for multiple images”, Proc. of the IEEE, vol. 57, no. 4, 1969, 654–665, 10.1109/PROC.1969.7020.
• [39] P. E. Anuta, “Spatial Registration of Multispectral and Multitemporal Digital Imagery Using Fast Fourier Transform Techniques”, IEEE Transactions on Geoscience Electronics, vol. 8, no. 4, 1970, 353–368, 10.1109/TGE.1970.271435.
• [40] W. P. Waite and H. C. MacDonald, ““Vegetation Penetration” with K-Band Imaging Radars”, IEEE Transactions on Geoscience Electronics, vol. 9, no. 3, 1971, 147–155, 10.1109/TGE.1971.271487.
• [41] G. Nagy, “Digital image-processing activities in remote sensing for earth resources”, Proc. of the IEEE, vol. 60, no. 10, 1972, 1177–1200, 10.1109/PROC.1972.8879.
• [42] T. Roska, T. Boros, P. Thiran and L. O. Chua, “Detecting simple motion using cellular neural networks”. In: IEEE International Workshop on Cellular Neural Networks and their Applications, 1990, 127–138, 10.1109/CNNA.1990.207516.
• [43] Xiou-Ping Yang, Tao Yang and Lin-Bao Yang, “Extracting focused object from defocused background using cellular neural networks”. In: Proc. of the Third IEEE International Workshop on Cellular Neural Networks and Their Applications (CNNA-94), 1994, 451–455, 10.1109/CNNA.1994.381633.
• [44] O. Tolluoglu, O. M. Ucan, S. Kent and S. Kargin, “Edge detection in noise mounted picture”. In: Proc. of the IEEE 12th Signal Processing and Communications Applications Conference,, 2004, 335–338, 10.1109/SIU.2004.1338328.
• [45] G. Grassi, E. Di Sciascio, A. L. Grieco and P. Vecchio, “A New Object-Oriented Segmentation Algorithm based on CNNs - Part I: Edge Extraction”. In: 2005 9th International Workshop on Cellular Neural Networks and Their Applications,005, 158–161, 10.1109/CNNA.2005.1543185.
• [46] K. Slot, P. Korbel, M. Gozdzik and H. Kim, “Pattern detection in spectrograms by means of Cellular Neural Networks”. In: 2006 10th International Workshop on Cellular Neural Networks and Their Applications, 2006, 1–6, 10.1109/CNNA.2006.341632.
• [47] F. Takarli, A. Aghagolzadeh and H. Seyedarabi, “Robust pedestrian detection using low level and high level features”. In: 2013 21st Iranian Conference on Electrical Engineering (ICEE), 2013, 1–6, 10.1109/IranianCEE.2013.6599574.
• [48] W. Thubsaeng, A. Kawewong and K. Patanukhom, “Vehicle logo detection using convolutional neural network and pyramid of histogram of oriented gradients”. In: 2014 11th International Joint Conference on Computer Science and Software Engineering (JCSSE), 2014, 34–39, 10.1109/JCSSE.2014.6841838.
• [49] Z. Li, Q. Wu, B. Cheng, L. Cao and H. Yang, “Remote Sensing Image Scene Classification Based on Object Relationship Reasoning CNN”, IEEE Geoscience and Remote Sensing Letters, vol. 19, 2022, 1–5, 10.1109/LGRS.2020.3017542.
• [50] Y. LeCun, L. Bottou, Y. Bengio and P. Haffner, “Gradient-Based Learning Applied to Document Recognition”, Proc. of the IEEE, vol. 86, 1998, 2278–2324, 10.1109/5.726791.
• [51] A. Femin and K. S. Biju, “Accurate Detection of Buildings from Satellite Images using CNN”. In: 2020 International Conference on Electrical, Communication, and Computer Engineering (ICECCE), 2020, 1-5, 10.1109/ICECCE49384.2020.9179232.
• [52] D. Marmanis, J. D. Wegner, S. Galliani, K. Schindler, M. Datcu and U. Stilla, “Semantic Segmentation of Aerial Images With an Ensemble of CNNs”, ISPRS Annals of Photogrammetry, Remote Sensing and Spatial Information Sciences, vol. III-3, 2016, 473–480, 10.5194/isprsannals--III-3-473-2016.
• [53] L. Cheng, K. E. Trenberth, M. D. Palmer, J. Zhu and J. P. Abraham, “Observed and simulated full--depth ocean heat-content changes for 1970–2005”, Ocean Science, vol. 12, no. 4, 2016, 925–935, 10.5194/os-12-925-2016.
• [54] N. Kussul, M. Lavreniuk, S. Skakun and A. Shelestov, “Deep Learning Classification of Land Cover and Crop Types Using Remote Sensing Data”, IEEE Geoscience and Remote Sensing Letters, vol. 14, no. 5, 2017, 778–782, 10.1109/LGRS.2017.2681128.
• [55] M. I. Elbakary and K. M. Iftekharuddin, “Shadow Detection of Man-Made Buildings in High-Resolution Panchromatic Satellite Images”, IEEE Transactions on Geoscience and Remote Sensing, vol. 52, no. 9, 2014, 5374–5386, 10.1109/TGRS.2013.2288500.
• [56] X. Zhang, S. Noda, R. Himeno and H. Liu, “Cardiovascular disease-induced thermal responses during passive heat stress: an integrated computational study: An integrated bioheat transfer model”, International Journal for Numerical Methods in Biomedical Engineering, vol. 32, no. 11, 2016, 5374–5386, 10.1002/cnm.2768.
• [57] L. Liu, W. Ouyang, X. Wang, P. Fieguth, J. Chen, X. Liu and M. Pietikäinen, “Deep Learning for Generic Object Detection: A Survey”, International Journal of Computer Vision, vol. 128, no. 2, 2020, 261–318, 10.1007/s11263-019-01247-4.
• [58] W. S. McCulloch and W. Pitts, “A Logical Calculus of the Ideas Immanent in Nervous Activity”, The Bulletin of Mathematical Biophysics, vol. 5, no. 4, 1943, 115–133, 10.1007/BF02478259.
• [59] Y.-C. Hsieh, C.-L. Chin, C.-S. Wei, I.-M. Chen, P.-Y. Yeh and R.-J. Tseng, “Combining VGG16, Mask R-CNN and Inception V3 to identify the benign and malignant of breast microcalcification clusters”. In: 2020 International Conference on Fuzzy Theory and Its Applications (iFUZZY), 2020, 2022–01-04, 10.1109/iFUZZY50310.2020.9297809.
• [60] G. E. Hinton, S. Osindero and Y.-W. Teh, “A Fast Learning Algorithm for Deep Belief Nets”, Neural Computation, vol. 18, no. 7, 2006, 1527–1554, 0.1162/neco.2006.18.7.1527.
• [61] V. Dumoulin, I. Goodfellow, A. Courville and Y. Bengio, “On the Challenges of Physical Implementations of RBMs”, Proceedings of the AAAI Conference on Artificial Intelligence, vol. 28, no. 1, 2014, 10.1609/aaai.v28i1.8924.
• [62] O. Russakovsky, J. Deng, H. Su, J. Krause, S. Satheesh, S. Ma, Z. Huang, A. Karpathy, A. Khosla, M. Bernstein, A. C. Berg and L. Fei-Fei, “ImageNet Large Scale Visual Recognition Challenge”, International Journal of Computer Vision, vol. 115,no. 3, 2015, 211–252, 10.1007/s11263-015-0816-y.
• [63] G. M. Castelluccio and D. L. McDowell, “Microstructure and mesh sensitivities of mesoscale surrogate driving force measures for transgranular fatigue cracks in polycrystals”, Materials Science and Engineering: A, vol. 639, 2015, 626–639, 10.1016/j.msea.2015.05.048.
• [64] A. Krizhevsky, I. Sutskever and G. E. Hinton, “ImageNet classification with deep convolutional neural networks”, Communications of the ACM,vol. 60, no. 6, 2017, 84–90, 10.1145/3065386.
• [65] L. Tian and A. Noore, “Software Reliability Prediction Using Recurrent Neural Network with Bayesian Regularization”, International Journal of Neural Systems, vol. 14, no. 3, 2004, 165–174, 10.1142/S0129065704001966.
• [66] A. Benamira, B. Devillers, E. Lesot, A. K. Ray, M. Saadi and F. D. Malliaros, “Semi-Supervised Learning and Graph Neural Networks for Fake News Detection”. In: Proceedings of the 2019 IEEE/ACM International Conference on Advances in Social Networks Analysis and Mining, 2019, 568–569, 10.1145/3341161.3342958.
• [67] X. Zhou, M. W. Koch and M. W. Roberts, “A Selective Attention Neural Network for Invariant Recognition of Distorted Objects”. In: IJCNN--91-Seattle International Joint Conference n Neural Networks, vol. ii, 1991, 10.1109/IJCNN.1991.155543.
• [68] L. Fu, D. Zhang and Q. Ye, “Recurrent Thrifty Attention Network for Remote Sensing Scene Recognition”, IEEE Transactions on Geoscience and Remote Sensing, vol. 59, no. 10, 2021, 8257–8268, 10.1109/TGRS.2020.3042507.
• [69] R. Larionov, V. Khryashchev and V. Pavlov, “Separation of Closely Located Buildings on Aerial Images Using U-Net Neural Network”. In: 2020 26th Conference of Open Innovations Association (FRUCT), 2020, 256–261, 10.23919/FRUCT48808.2020.9087365.