Artificial Intelligence You Only Look Once – Based Unmanned Aerial System for Remote Sensing in Security Surveillance

• Autorzy: Gutt, Jakub , Kaczor, Michał , Paleta, Grzegorz , Polec, Krzysztof , Stiborski, Kacper , Strzępek, Filip , Czyba, Roman , Czekalski, Piotr , Domin, Jarosław , Gebeyehu, Nebiyat
• Języki publikacji: EN

Abstrakty

EN

Unmanned aerial vehicles are a synergistic technology that complements other new technologies and is in constant development. The paper focuses on using artificial intelligence (AI) in security surveillance. This article aims to develop an unmanned aerial system to monitor border areas and detect human silhouettes in challenging environmental conditions. For this purpose, thermal imaging technology was used for remote sensing in combination with artificial intelligence, particularly Yolo algorithms. After testing various Yolo versions, the target algorithm was implemented on an NVIDIA Jetson Xavier edge device. Prototyping of the AI-based thermal detection system was carried out on the DJI S900 multi-rotor aircraft. The final solution was implemented on a vertical take-off and landing aircraft. A summary containing observations and conclusions, as well as perspectives for the development of future work, are included at the end of the paper.

Słowa kluczowe

EN

drone; UAV; artificial intelligence; artificial neural network; machine learning; control system; thermal imaging; edge computing

• Czasopismo: Advances in Science and Technology. Research Journal
• Rocznik: 2024
• Tom: Vol. 18, no 7
• Strony: 364--378
• Opis fizyczny: Bibliogr. 31 poz., fig., tab.

Twórcy

autor

Gutt, Jakub

• Faculty of Automatic Control, Electronics and Computer Science, Silesian University of Technology, ul. Akademicka 16, 44-100 Gliwice, Poland

autor

Kaczor, Michał

• Faculty of Transport and Aviation Engineering, Silesian University of Technology, ul. Zygmunta Krasińskiego 8, 40-019 Katowice, Poland

autor

Paleta, Grzegorz

• Faculty of Automatic Control, Electronics and Computer Science, Silesian University of Technology, ul. Akademicka 16, 44-100 Gliwice, Poland

autor

Polec, Krzysztof

• Faculty of Automatic Control, Electronics and Computer Science, Silesian University of Technology, ul. Akademicka 16, 44-100 Gliwice, Poland

autor

Stiborski, Kacper

• Faculty of Automatic Control, Electronics and Computer Science, Silesian University of Technology, ul. Akademicka 16, 44-100 Gliwice, Poland

autor

Strzępek, Filip

• Faculty of Automatic Control, Electronics and Computer Science, Silesian University of Technology, ul. Akademicka 16, 44-100 Gliwice, Poland

autor

Czyba, Roman

• Faculty of Automatic Control, Electronics and Computer Science, Silesian University of Technology, ul. Akademicka 16, 44-100 Gliwice, Poland,

autor

Czekalski, Piotr

• Faculty of Automatic Control, Electronics and Computer Science, Silesian University of Technology, ul. Akademicka 16, 44-100 Gliwice, Poland

autor

Domin, Jarosław

• Faculty of Electrical Engineering, Silesian University of Technology, ul. Akademicka 10a, 44-100 Gliwice, Poland

autor

Gebeyehu, Nebiyat

• Faculty of Automatic Control, Electronics and Computer Science, Silesian University of Technology, ul. Akademicka 16, 44-100 Gliwice, Poland

Bibliografia

• 1. Chui M., Manyika J., Miremadi M., Henke N., Chung R., Nel P., Malhotra S. Notes from the AI frontier insights from hundreds of use cases. Discussion paper, 2, McKinsey Global Institute, McKinsey&Company, April 2018.
• 2. Wieczorek S., Ludwig S., Büttner L., Bauer M.R., Markovič P. Forecast of changes in business administration driven by digitalization. open journal of business and management, July 2021; 9(4), https://doi.org/10.4236/ojbm.2021.94092.
• 3. Conejero J.M., Brito I.S., Moreira A., Cunha J., Araújo J. Modeling the Impact of UAVs in Sustainability. In: Proceedings of the 24th International Requirements Engineering Conference Workshops (REW), Beijing, China, 12–16 September 2016. https://doi.org/10.1109/REW.2016.044.
• 4. Cao Z., Kooistra L., Wang W., Guo L., Valente, J. Real-time object detection based on UAV remote sensing: A Systematic Literature Review. Drones 2023; 7(10), 620. https://doi.org/10.3390/drones7100620.
• 5. Sandino J., Vanegas F., Maire F., Caccetta P. Sanderson C.; Gonzalez, F. UAV framework for autonomous onboard navigation and people/object detection in cluttered indoor environments. Remote Sens. 2020; 12, 3386. https://doi.org/10.3390/rs12203386.
• 6. Cheng J., Zhang S. TBFNT3D: Two-Branch Fusion Network With Transformer for Multimodal Indoor 3D Object Detection. IEEE Robotics and Automation Letters 2023; 8(10), 6523–6530. https://doi.org/10.1109/LRA.2023.3309133.
• 7. Samani E.U., Yang X. Banerjee A.G. Visual Object Recognition in Indoor Environments Using Topologically Persistent Features. IEEE Robotics and Automation Letters 2021; 6(4), 2377–3766. https://doi.org/10.1109/LRA.2021.3099460.
• 8. Saif A.F.M.S., Prabuwono A.S., Mahayuddin Z.R. Real time vision based object detection from uav aerial images: a conceptual framework. In: Omar, K., et al. Intelligent Robotics Systems: Inspiring the NEXT. Communications in Computer and Information Science, 376. Springer, Berlin, Heidelberg. https://doi.org/10.1007/978-3-642-40409-2_23.
• 9. Zhang Z., Liu Y., Liu T., Lin Z., Wang S. DAGN: a real-time uav remote sensing image vehicle detection framework. IEEE Geoscience and Remote Sensing Letters, 2019; 17(11), 1884–1888. https://doi.org/10.1109/LGRS.2019.2956513.
• 10. Parker A., Gonzales F., Trotter P. Live detection of foreign object debris on runways detection using drones and AI. In: Proceedings of the 2022 IEEE Aerospace Conference (AERO), Big Sky, MT, USA, 05–12 March 2022. https://doi.org/10.1109/AERO53065.2022.9843697.
• 11. Truong H. M., Clavel M. Identifying temporary water bodies from drone images at real-time using deep-learning techniques, In: Proceedings of the 2022 International Conference on Advanced Computing and Analytics (ACOMPA), Ho Chi Minh City, Vietnam, 21–23 November 2022, https://doi.org/10.1109/ACOMPA57018.2022.00009.
• 12. Zhang Z., Uchiya T., Takumi I., Speed of autonomous drones in locating survivors after a disaster. In: Proceedings of the IEEE 10th Global Conference on Consumer Electronics (GCCE), Kyoto, Japan, 12–15 October 2021. https://doi.org/10.1109/GCCE53005.2021.9621953.
• 13. Hoshino W., Seo J., Yamazaki Y. A study for detecting disaster victims using multi-copter drone with a thermographic camera and image object recognition by SSD. In: Proceedings of the 2021 IEEE/ASME International Conference on Advanced Intelligent Mechatronics (AIM), Delft, Netherlands, 12–16 July 202. https://doi.org/10.1109/AIM46487.2021.9517524.
• 14. Geng X., Peng R. Li M., Liu W., Jiang G., Jiang H., Luo J. A lightweight approach for passive human localization using an infrared thermal camera. IEEE Internet of Things Journal, 2022; 9(24), 24800–24811. https://doi.org/10.1109/JIOT.2022.3194714.
• 15. Zhao X., Xiang M., He J., Huang Ch. Fire detection method in infrared image based on improved YOLO network, In: Proceedings of the 2021China Automation Congress (CAC), Beijing, China, 22–24 October 2021. https://doi.org/10.1109/CAC53003.2021.9728278.
• 16. Munshi A.A. Fire detection methods based on various color spaces and gaussian mixture models. Advances in Science and Technology Research Journal 2021; 15(3), 197–214. https://doi.org/10.12913/22998624/138924.
• 17. Shao Z., Liang Y., Tian F., Song S., Deng R. Constructing 3-D Land Surface Temperature Model of Local Coal Fires Using UAV Thermal Images. IEEE Transactions on Geoscience and Remote Sensing, 2022; 60, 5002309. https://doi.org/10.1109/TGRS.2022.3176854.
• 18. Redmon J., Divvala S., Girshick R., Farhadi A. You Only Look Once: unified, real-time object detection. In: Proceedings of the 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), Las Vegas, USA, 27–30 June 2016. https://doi.org/10.1109/CVPR.2016.91.
• 19. Liu K, Tang H., He S., Yu Q., Xiong Y., Wang N. Performance validation of yolo variants for object detection. In: Proceedings of the 2021 International Conference on Bioinformatics and Intelligent Computing (BIC ‘21). New York, USA, January 2021. https://doi.org/10.1145/3448748.344878.
• 20. Wu H.-H., Zhou Z., Feng M., Yan Y, Xu H, Qian L. Real-time single object detection on the UAV. In: Proceedings of the 2019 International Conference on Unmanned Aircraft Systems (ICUAS), Atlanta, GA, USA, 11–14 June 2019. https://doi.org/10.1109/ICUAS.2019.8797866.
• 21. Tijtgat N., Ranst W. V., Goedeme T., Volckaert B. De Turck F. Embedded real-time object detection for a UAV warning system. In: Proceedings of the 2017 IEEE International Conference on Computer Vision (ICCV), Venice, Italy, 22–29 October 2017. https://doi.org/10.1109/ICCVW.2017.247.
• 22. Subrahmanyam V, Iowa State University ProQuest Dissertations & Theses, 2019. 13896596. https://www.proquest.com/openview/52ebcc2818e1a95e8ee890197e97762b/1.
• 23. Wu Q., Zhou Y., Real-time object detection based on unmanned aerial vehicle. In: Proceedings of the 2019 IEEE 8th Data Driven Control and Learning Systems Conference (DDCLS), Dali, China, 24–27 May 2019. https://doi.org/10.1109/DDCLS.2019.8908984.
• 24. Kim H.J., Shin M.C., Han M.W., Hong Ch., Lee H.W. An Efficient Scheme to Obtain Background Image in Video for YOLO-Based Static Object Recognition. Journal of Web Engineering, 2022; 21(5), 1691–1706. https://doi.org/10.13052/jwe1540-9589.21513.
• 25. Zhang Q., Hu X. MSFFA-YOLO Network: Multiclass Object Detection for Traffic Investigations in Foggy Weather. IEEE Transactions on Instrumentation and Measurement, 2023; 72, 2528712. https://doi.org/10.1109/TIM.2023.3318671.
• 26. Chen X., Peng D. L., Gu Y. Real-time object detection for UAV images based on improved YOLOv5s. Opto-Electron Eng, 2022; 49(3), 210372. https://doi.org/10.12086/oee.2022.210372.
• 27. Carranza-García M., Torres-Mateo J., Lara-Benítez P., García-Gutiérrez J. On the performance of one-stage and two-stage object detectors in autonomous vehicles using camera data. Remote Sensing. 2021; 13(1), 89. https://doi.org/10.3390/rs13010089.
• 28. J. Lee, J. Wang, D. Crandall, S. Šabanović and G. Fox, Real-time, cloud-based object detection for unmanned aerial vehicles. In: Proceedings of the 2017 First IEEE International Conference on Robotic Computing (IRC), Taichung, Taiwan, 10–12 April 2017. https://doi.org/10.1109/IRC.2017.77.
• 29. Thamm F.-P., Brieger N., Neitzke K.-P., Meyer M., Jansen R., Mönninghof M. Songbird – an innovative UAS combining the advantages of fixed wing and multi rotor UAS, In: Proceedings of the International Conference on Unmanned Aerial Vehicles in Geomatics, Toronto, Canada, 30 Aug–02 Sep 2015. https://doi.org/10.5194/isprsarchives-XL-1-W4-345-2015, 2015.
• 30. Boon M. A., Drijfhout A. P., Tesfamichael, S. Comparison of a fixed-wing and multi-rotor UAV for environmental mapping applications: a case study, In: Proceedings of the International Conference on Unmanned Aerial Vehicles in Geomatics, Bonn, Germany, 4–7 September 2017. https://doi.org/10.5194/isprs-archives-XLII-2-W6-47-2017, 2017.
• 31. Skóra J. The use of unmanned aerial vehicles in the context of ensuring security in the state. Aviation and Security, 2022; 1. https://doi.org/10.55676/asi.v1i1.7.

Identyfikatory

DOI

10.12913/22998624/193368