Violence prediction in surveillance videos

• Autorzy: Mahareek Esraa A. , Fathy Doaa R. , Elsayed Eman K. , Eldesouky Nahed M. , Eldahshan Kamal A.

Identyfikatory

DOI

10.35784/acs-2024-25

• Języki publikacji: EN

Abstrakty

EN

Forecasting violence has become a critical obstacle in the field of video monitoring to guarantee public safety. Lately, YOLO (You Only Look Once) has become a popular and effective method for detecting weapons. However, identifying and forecasting violence remains a challenging endeavor. Additionally, the classification results had to be enhanced with semantic information. This study suggests a method for forecasting violent incidents by utilizing Yolov9 and ontology. The authors employed Yolov9 to identify and categorize weapons and individuals carrying them. Ontology is utilized for semantic prediction to assist in predicting violence. Semantic prediction happens through the application of a SPARQL query to the identified frame label. The authors developed a Threat Events Ontology (TEO) to gain semantic significance. The system was tested with a fresh dataset obtained from a variety of security cameras and websites. The VP Dataset comprises 8739 images categorized into 9 classes. The authors examined the outcomes of using Yolov9 in conjunction with ontology in comparison to using Yolov9 alone. The findings show that by combining Yolov9 with ontology, the violence prediction system's semantics and dependability are enhanced. The suggested system achieved a mean Average Precision (mAP) of 83.7 %, 88% for precision, and 76.4% for recall. However, the mAP of Yolov9 without TEO ontology achieved a score of 80.4%. It suggests that this method has a lot of potential for enhancing public safety. The authors finished all training and testing processes on Google Colab's GPU. That reduced the average duration by approximately 90.9%. The result of this work is a next level of object detectors that utilize ontology to improve the semantic significance for real-time end-to-end object detection.

Słowa kluczowe

EN

violence prediction system; YOLO v8; ontology; surveillance cameras; anomaly prediction

• Wydawca: Polskie Towarzystwo Promocji Wiedzy ; Lublin University of Technology
• Czasopismo: Applied Computer Science
• Rocznik: 2024
• Tom: Vol. 20, no. 3
• Strony: 1--16
• Opis fizyczny: Bibliogr. 37 poz., fig., tab.

Twórcy

autor

Mahareek Esraa A.

• Al-Azhar University, Faculty of science, Mathematics Department, Egypt

• https://orcid.org/0000-0002-9042-248X

autor

Fathy Doaa R.

• Al-Azhar University, Faculty of science, Mathematics Department, Egypt

• https://orcid.org/0000-0002-5625-0282

autor

Elsayed Eman K.

• Canadian International College, Dean of School of Computer Science, Egypt

• https://orcid.org/0000-0001-7870-927X

autor

Eldesouky Nahed M.

• Al-Azhar University, Faculty of science, Mathematics Department, Egypt

• https://orcid.org/0009-0008-4547-3051

autor

Eldahshan Kamal A.

• Al-Azhar University, Faculty of science, Mathematics Department, Egypt

• https://orcid.org/0000-0002-9953-5480

Bibliografia

• [1] Arslan, A. N., Hempelmann, C. F., Attardo, S., Blount, G. P., & Sirakov, N. M. (2015). Threat assessment using visual hierarchy and conceptual firearms ontology. Optical Engineering, 54(5), 053109. https://doi.org/10.1117/1.oe.54.5.053109
• [2] Arslan, A. N., Sirakov, N. M., & Attardo, S. (2012). Weapon ontology annotation using boundary describing sequences. 2012 IEEE Southwest Symposium on Image Analysis and Interpretation (pp. 101-104). https://doi.org/10.1109/SSIAI.2012.6202463
• [3] Ashraf, A. H., Imran, M., Qahtani, A. M., Alsufyani, A., Almutiry, O., Mahmood, A., Attique, M., & Habib, M. (2022). Weapons detection for security and video surveillance using CNN and YOLO-V5s. Computers, Materials and Continua, 70(2), 2761–2775. https://doi.org/10.32604/cmc.2022.018785
• [4] Benjumea, A., Teeti, I., Cuzzolin, F., & Bradley, A. (2021). YOLO-Z: Improving small object detection in YOLOv5 for autonomous vehicles. ArXiv, abs/2112.11798. https://doi.org/10.48550/arXiv.2112.11798
• [5] Bisong, E. (2019). Building Machine Learning and Deep Learning models on Google Cloud platform: A Comprehensive Guide for Beginners. Apress Berkeley.
• [6] Bochkovskiy, A., Wang, C.-Y., & Liao, H.-Y. M. (2020). YOLOv4: Optimal speed and accuracy of object detection. ArXiv, abs/2004.10934. https://doi.org/10.48550/arXiv.2004.10934
• [7] Dugyala, R., Vishnu Vardhan Reddy, M., Tharun Reddy, C., & Vijendar, G. (2023). Weapon detection in surveillance videos using YOLOV8 and PELSF-DCNN. 4th International Conference on Design and Manufacturing Aspects for Sustainable Energy (ICMED-ICMPC 2023) (pp. 01071). E3S Web of Conferences. https://doi.org/10.1051/e3sconf/202339101071
• [8] Elsayed, E. K., & Fathy, D. R. (2020a). Semantic Deep Learning to translate dynamic sign language. International Journal of Intelligent Engineering and Systems, 14(1), 316-325. https://doi.org/10.22266/IJIES2021.0228.30
• [9] Elsayed, E. K., & Fathy, D. R. (2020b). Sign language semantic translation system using ontology and Deep Learning. International Journal of Advanced Computer Science and Applications, 11(1), 141-147. https://doi.org/10.14569/IJACSA.2020.0110118
• [10] Glenn, J. (2022, November 22). Yolov5 release v7.0. https://github.com/ultralytics/yolov5/tree/v7.0
• [11] Han, J., Liu, Y., Li, Z., Liu, Y., & Zhan, B. (2023). Safety helmet detection based on YOLOv5 driven by super-resolution reconstruction. Sensors, 23(4), 1822. https://doi.org/10.3390/s23041822
• [12] Khalid, S., Waqar, A., Ain Tahir, H. U., Edo, O. C., & Tenebe, I. T. (2023). Weapon detection system for surveillance and security. 2023 International Conference on IT Innovation and Knowledge Discovery (ITIKD 2023) (pp. 1-7). IEEE. https://doi.org/10.1109/ITIKD56332.2023.10099733
• [13] Lai, J., & Maples, S. (2017). Developing a real-time gun detection classifier. Stanford University.
• [14] Li, C., Li, L., Geng, Y., Jiang, H., Cheng, M., Zhang, B., Ke, Z., Xu, X., & Chu, X. (2023). YOLOv6 v3.0: A full-scale reloading. ArXiv, abs/2301.05586. https://doi.org/10.48550/arXiv.2301.05586
• [15] Li, C., Li, L., Jiang, H., Weng, K., Geng, Y., Li, L., Ke, Z., Li, Q., Cheng, M., Nie, W., Li, Y., Zhang, B., Liang, Y., Zhou, L., Xu, X., Chu, X., Wei, X., & Wei, X. (2022). YOLOv6: A single-stage object detection framework for industrial applications. ArXiv, abs/2209.02976. https://doi.org/10.48550/arXiv.2209.02976
• [16] Li, X., Wang, W., Wu, L., Chen, S., Hu, X., Li, J., Tang, J., & Yang, J. (2020). Generalized focal loss: Learning qualified and distributed bounding boxes for dense object detection. ArXiv, abs/2006.04388. https://doi.org/10.48550/arXiv.2006.04388
• [17] Lin, T. Y., Goyal, P., Girshick, R., He, K., & Dollar, P. (2017). Focal loss for dense object detection. 2017 IEEE International Conference on Computer Vision (ICCV) (pp. 2999–3007). IEEE. https://doi.org/10.1109/ICCV.2017.324
• [18] Lou, H., Duan, X., Guo, J., Liu, H., Gu, J., Bi, L., & Chen, H. (2023). DC-YOLOv8: Small-size object detection algorithm based on camera sensor. Electronics, 12(10), 2323. https://doi.org/10.3390/electronics12102323
• [19] Mahareek, E. A. (2024). VP Dataset. https://Universe.Roboflow.Com/al-Azhar-Unversity/Violence-Prediction-in-Surveillance-Videos.
• [20] Mahareek, E. A., Elsayed, E. K., Eldesouky, N. M., & Eldahshan, K. A. (2024). Detecting anomalies in security cameras with 3D-convolutional neural network and convolutional long short-term memory. International Journal of Electrical and Computer Engineering, 14(1), 993–1004. https://doi.org/10.11591/ijece.v14i1.pp993-1004
• [21] Redmon, J., & Farhadi, A. (2017). YOLO9000: Better, faster, stronger. Proceedings. 30th IEEE Conference on Computer Vision and Pattern Recognition (CVPR 2017), (pp. 6517-6525). IEEE. https://doi.org/10.1109/CVPR.2017.690
• [22] Redmon, J., & Farhadi, A. (2018). YOLOv3: An incremental improvement. ArXiv, abs/1804.02767. https://doi.org/10.48550/arXiv.1804.02767
• [23] Redmon, J. (2016). Darknet: Open source neural networks in c. http://pjreddie.com/darknet/
• [24] Redmon, J., Divvala, S., Girshick, R., & Farhadi, A. (2016). You only look once: Unified, real-time object detection. 2016 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (pp. 779-788). IEEE. https://doi.org/10.1109/CVPR.2016.91
• [25] Reis, D., Kupec, J., Hong, J., & Daoudi, A. (2023). Real-Time flying object detection with YOLOv8. ArXiv, abs/2305.09972. https://doi.org/10.48550/arXiv.2305.09972
• [26] Solawetz, J. F. (2023, January 11). What is YOLOv8? The Ultimate Guide. https://blog.roboflow.com/whats-new-in-yolov8/
• [27] Songire, S. B., Chandrakant Patkar, U., Chate, P. J., Patil, M. A., Wani, L. K., Pathak, A. S., Bhardwaj Shrivas, S., & Patil, U. (2023). Using Yolo V7 development of complete vids solution based on latest requirements to provide highway traffic and incident real time info to the atms control room using Artificial Intelligence. Journal of Survey in Fisheries Sciences, 10(4S), 3444-3456.
• [28] Tian, Z., Shen, C., Chen, H., & He, T. (2022). FCOS: A simple and strong anchor-free object detector. IEEE Transactions on Pattern Analysis and Machine Intelligence, 44(4), 1922–1933. https://doi.org/10.1109/TPAMI.2020.3032166
• [29] Verma, R., & Jayant, S. (2022). Cyber crime prediction using Machine Learning. In M. Singh, V. Tyagi, P. K. Gupta, J. Flusser, & T. Ören (Eds.), Advances in Computing and Data Sciences (Vol. 1614, pp. 160–172). Springer International Publishing. https://doi.org/10.1007/978-3-031-12641-3_14
• [30] Wang, C. Y., Mark Liao, H. Y., Wu, Y. H., Chen, P. Y., Hsieh, J. W., & Yeh, I. H. (2020). CSPNet: A new backbone that can enhance learning capability of CNN. IEEE Computer Society Conference on Computer Vision and Pattern Recognition Workshops (pp. 1571-1580). IEEE. https://doi.org/10.1109/CVPRW50498.2020.00203
• [31] Wang, C., He, W., Nie, Y., Guo, J., Liu, C., Han, K., & Wang, Y. (2023). Gold-YOLO: Efficient object detector via Gather-and-Distribute mechanism. ArXiv, abs/2309.11331. https://doi.org/10.48550/arXiv.2309.11331
• [32] Wang, C.-Y., Bochkovskiy, A., & Liao, H.-Y. M. (2022). YOLOv7: Trainable bag-of-freebies sets new state-of-the-art for real-time object detectors. ArXiv, abs/2207.02696. https://doi.org/10.48550/arXiv.2207.02696
• [33] Wang, C.-Y., Bochkovskiy, A., & Liao, H.-Y. M. (2023). YOLOv7: Trainable bag-of-freebies sets new state-of-the-art for real-time object detectors. 2023 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) (pp. 7464–7475). IEEE. https://doi.org/10.1109/cvpr52729.2023.00721
• [34] Wang, C.-Y., Yeh, I.-H., & Liao, H.-Y. M. (2024). YOLOv9: Learning what you want to learn using programmable gradient information. ArXiv, abs/2402.13616. https://doi.org/10.48550/arXiv.2402.13616
• [35] Zhang, X., Fang, S., Shen, Y., Yuan, X., & Lu, Z. (2024). Hierarchical velocity optimization for connected automated vehicles with cellular vehicle-to-everything communication at continuous signalized intersections. IEEE Transactions on Intelligent Transportation Systems, 25(3), 2944–2955. https://doi.org/10.1109/TITS.2023.3274580
• [36] Zhang, S., Chi, C., Yao, Y., Lei, Z., & Li, S. Z. (2019). Bridging the gap between anchor-based and anchor-free detection via adaptive training sample selection. ArXiv, abs/1912.02424. https://doi.org/10.48550/arXiv.1912.02424
• [37] Zheng, Z., Wang, P., Liu, W., Li, J., Ye, R., & Ren, D. (2020). Distance-IoU loss: Faster and better learning for bounding box regression. 34th AAAI Conference on Artificial Intelligence (AAAI 2020) (pp. 12993-13000). https://doi.org/10.1609/aaai.v34i07.6999

Uwagi

Opracowanie rekordu ze środków MNiSW, umowa nr POPUL/SP/0154/2024/02 w ramach programu "Społeczna odpowiedzialność nauki II" - moduł: Popularyzacja nauki (2025).