AI-driven cognitive surveillance framework for suspicious activity detection in academia

• Autorzy: Soomro Asad Hameed , Arain Rafaqat Hussain , Shaikh Riaz Ahmed

Identyfikatory

DOI

10.12913/22998624/207850

• Języki publikacji: EN

Abstrakty

EN

Early identification of human suspicious activities, especially in academia, is very crucial for enhancing security and safety, while existing surveillance systems remain ineffective at contextual behavior analysis. This study proposes the HYDPL Algorithm that incorporates a novel pseudo-labeling approach and innovative image analysis to perform human suspicious activity detection. The HYDPL algorithm includes several stages of data processing and feature extraction that aim to enhance the detection of activities by increasing the model’s accuracy. To evaluate the effectiveness of the algorithm, two datasets were utilized: CampusWatch (Dataset-I), which consists of real-world scenarios specifically collected for this study from academia, targeting nine specific behaviors: Kicking, Punching, Running, Pushing, Smoking, Throwing, Jumping, Falling, and Talking, providing realistic representation of human activities, and HMDB51, named as Dataset-II, which is a collection of movies or scripted videos. The evaluation results of the model were outstanding for both datasets, as the model was successful in delivering 98% accuracy for Dataset-I and 97% for Dataset-II, highlighting the model’s ability to accurately detect real-world conditions. However, the study is limited to academia and only nine categories out of ten, with the tenth category being normal behavior. Future work will expand the dataset, explore advanced deep learning architectures, and implement real-time processing to enhance the model's applicability across various environments.

Słowa kluczowe

EN

computer vision; deep learning; human activity detection; academia; HYDPL; campus watch; multi-modal large model

• Wydawca: Lublin University of Technology ; Polish Society of Ecological Engineering (PTIE), Branch of PTIE in Lublin
• Czasopismo: Advances in Science and Technology. Research Journal
• Rocznik: 2025
• Tom: Vol. 19, no. 9
• Strony: 333--352
• Opis fizyczny: Bibligr. 54 poz., fig., tab.

Twórcy

autor

Soomro Asad Hameed

• Institute of Computer Sciences, Shah Abdul Latif University, Khairpur Mirs, Sindh, Pakistan
• The Benazir Bhutto Shaheed University of Technology and Skill Development, Khairpur Mirs, Sindh, Pakistan

• https://orcid.org/0009-0004-4064-8343

autor

Arain Rafaqat Hussain

• Institute of Computer Sciences, Shah Abdul Latif University, Khairpur Mirs, Sindh, Pakistan

• https://orcid.org/0009-0008-7303-4849

autor

Shaikh Riaz Ahmed

• Institute of Computer Sciences, Shah Abdul Latif University, Khairpur Mirs, Sindh, Pakistan

• https://orcid.org/0009-0004-6766-4992

Bibliografia

• 1. Kumar P, Chauhan S, Awasthi LK. Human activity recognition (HAR) using deep learning: review, methodologies, progress and future research directions. Arch Comput Methods Eng 2024;31:179–219. https://doi.org/10.1007/S11831-023-09986-X/METRICS.
• 2. Qin Z, Liu Y, Ji P, Kim D, Wang L, McKay RI, et al. Fusing higher-order features in graph neural networks for skeleton-based action recognition. IEEE Trans Neural Networks Learn Syst 2024;35:4783–97. https://doi.org/10.1109/TNNLS.2022.3201518.
• 3. Zhu N, Diethe T, Camplani M, Tao L, Burrows A, Twomey N, et al. Bridging e-health and the internet of things: The SPHERE Project. IEEE Intell Syst 2015;30:39–46. https://doi.org/10.1109/MIS.2015.57.
• 4. Cheoi KJ. Temporal saliency-based suspicious behavior pattern detection. Appl Sci 2020;10:1020. https://doi.org/10.3390/APP10031020.
• 5. Rajpurkar OM, Kamble SS, Nandagiri JP, Nimkar A V. Alert Generation on Detection of Suspicious Activity Using Transfer Learning. 2020 11th Int. Conf. Comput. Commun. Netw. Technol. ICCCNT 2020, 2020. https://doi.org/10.1109/ICCCNT49239.2020.9225263.
• 6. Cabanillas-Carbonell M, Rivera JS, Muñoz JS. Artificial intelligence in video surveillance systems for suspicious activity detection and incident response: A systematic literature review. Adv Sci Technol Res J 2025;19:389–405. https://doi.org/10.12913/22998624/196795.
• 7. Concha DT, Maia HDA, Pedrini H, Tacon H, Brito ADS, Chaves HDL, et al. Multi-stream Convolutional Neural Networks for Action Recognition in Video Sequences Based on Adaptive Visual Rhythms. Proc - 17th IEEE Int Conf Mach Learn Appl ICMLA 2018 2018:473–80. https://doi.org/10.1109/ICMLA.2018.00077.
• 8. Chen L, Hoey J, Nugent CD, Cook DJ, Yu Z. Sensor-based activity recognition. IEEE Trans Syst Man Cybern Part C Appl Rev 2012;42:790–808. https://doi.org/10.1109/TSMCC.2012.2198883.
• 9. Lloret J, Canovas A, Sendra S, Parra L. A smart communication architecture for ambient assisted living. IEEE Commun Mag 2015;53:26–33. https://doi.org/10.1109/MCOM.2015.7010512.
• 10. Marsden M, McGuinness K, Little S, O’Connor NE. ResnetCrowd: A residual deep learning architecture for crowd counting, violent behaviour detection and crowd density level classification. 2017 14th IEEE Int. Conf. Adv. Video Signal Based Surveillance, AVSS 2017, Institute of Electrical and Electronics Engineers Inc.; 2017. https://doi.org/10.1109/AVSS.2017.8078482.
• 11. Gorave A, Misra S, Padir O, Patil A, Ladole K. Suspicious activity detection using live video analysis, 2020;203–14. https://doi.org/10.1007/978-981-15-0790-8_21.
• 12. Dhulekar PA, Gandhe ST, Sawale N, Shinde V, Khute S. Surveillance System for Detection of Suspicious Human Activities at War Field. 2018 Int. Conf. Adv. Commun. Comput. Technol. ICACCT 2018, 2018;357–60. https://doi.org/10.1109/ICACCT.2018.8529632.
• 13. Pappula S, Nadendla T, Lomadugu NB, Revanth Nalla S. Detection and Classification of Pneumonia Using Deep Learning by the Dense Net-121 Model. 2023 9th Int Conf Adv Comput Commun Syst ICACCS 2023 2023:1671–5. https://doi.org/10.1109/ICACCS57279.2023.10113110.
• 14. Khan RU, Haq AU, Hussain SM, Ullah S, Almakdi S, Kumar R, et al. Analyzing and Battling the Emerging Variants of Covid-19 Using Artificial Neural Network and Blockchain. 2021 18th Int Comput Conf Wavelet Act Media Technol Inf Process ICCWAMTIP 2021 2021:101–5. https://doi.org/10.1109/ICCWAMTIP53232.2021.9674142.
• 15. Al-qaness MAA, Dahou A, Abd Elaziz M, Helmi AM. Human activity recognition and fall detection using convolutional neural network and transformer-based architecture. Biomed Signal Process Control 2024;95:106412. https://doi.org/10.1016/J.BSPC.2024.106412.
• 16. Kaya Y, Topuz EK. Human activity recognition from multiple sensors data using deep CNNs. Multimed Tools Appl 2024;83:10815–38. https://doi.org/10.1007/S11042-023-15830-Y/METRICS.
• 17. Islam MM, Nooruddin S, Karray F, Muhammad G. Multi-level feature fusion for multimodal human activity recognition in Internet of Healthcare Things. Inf Fusion 2023;94:17–31. https://doi.org/10.1016/J.INFFUS.2023.01.015.
• 18. Kshirsagar AP, Azath H. YOLOv3-based human detection and heuristically modified-LSTM for abnormal human activities detection in ATM machine. J Vis Commun Image Represent 2023;95:103901. https://doi.org/10.1016/J.JVCIR.2023.103901.
• 19. Sudharson D, Srinithi J, Akshara S, Abhirami K, Sriharshitha P, Priyanka K. Proactive headcount and suspicious activity detection using YOLOv8. Procedia Comput Sci 2023;230:61–9. https://doi.org/10.1016/J.PROCS.2023.12.061.
• 20. Zhou Y. A YOLO-NL object detector for real-time detection. Expert Syst Appl 2024;238:122256. https://doi.org/10.1016/J.ESWA.2023.122256.
• 21. Zhang Y, Ye M, Zhu G, Liu Y, Guo P, Yan J. FFCA-YOLO for small object detection in remote sensing images. IEEE Trans Geosci Remote Sens 2024;62:1–15. https://doi.org/10.1109/TGRS.2024.3363057.
• 22. Kuehne H, Jhuang H, Stiefelhagen R, Serre T. HMDB51: A large video database for human motion recognition. High Perform Comput Sci Eng 2012 - Trans High Perform Comput Center, Stuttgart, HLRS 2012 2013:571–82. https://doi.org/10.1007/978-3-642-33374-3_41.
• 23. HMDB_Human_Activity_Recognition n.d. https://www.kaggle.com/datasets/avigoen/hmdb-human-activity-recognition (accessed November 14, 2024).
• 24. Gianluigi C, Raimondo S. An innovative algorithm for key frame extraction in video summarization. J Real-Time Image Process 2006;1:69–88. https://doi.org/10.1007/s11554-006-0001-1.
• 25. García GB. Learning Image Processing with OpenCV: exploit the amazing features of OpenCV to create powerful image processing applications through easy-to-follow examples. Packt Publishing; 2015.
• 26. Jiang Z, Zhao L, Li S, Jia Y, Liquan Z. Real-time object detection method for embedded devices. n.d.
• 27. Choi W, Heo S. Deep learning approaches to automated video classification of upper limb tension test. Healthc 2021;9. https://doi.org/10.3390/healthcare9111579.
• 28. Malla PP, Sahu S, Alutaibi AI. Classification of Tumor in Brain MR Images Using Deep Convolutional Neural Network and Global Average Pooling. Processes 2023;11. https://doi.org/10.3390/pr11030679.
• 29. Shdefat AY, Mostafa N, Al-Arnaout Z, Kotb Y, Alabed S. Optimizing HAR Systems: Comparative Analysis of Enhanced SVM and k-NN Classifiers. Int J Comput Intell Syst 2024;17:1–32. https://doi.org/10.1007/S44196-024-00554-0/FIGURES/9.
• 30. Culjak I, Abram D, Pribanic T, Dzapo H, Cifrek M. A brief introduction to OpenCV | IEEE Conference Publication | IEEE Xplore 2012:1725–30. https://ieeexplore.ieee.org/abstract/document/6240859 (accessed August 25, 2024).
• 31. Huang G, Liu Z, Van Der Maaten L, Weinberger KQ. Densely connected convolutional networks. Proc - 30th IEEE Conf Comput Vis Pattern Recognition, CVPR 2017;2017-January:2261–9. https://doi.org/10.1109/CVPR.2017.243.
• 32. Elnazer M, Elmamoon A, Mustapha AA. A comparative study of deep learning models for human activity recognition. Cloud Comput Data Sci 2025;6:79–93. https://doi.org/10.37256/CCDS.6120256264.
• 33. Rachmatullah MN, Sutarno S, Isnanto RF. Video annomaly classification using convolutional neural network. Comput Eng Appl J 2024;13:74–82. https://doi.org/10.18495/COMENGAPP.V13I1.468.
• 34. Nayak S, Panigrahi CR, Pati B, Nanda S, Hsieh MY. Comparative analysis of HAR datasets using classification algorithms. Comput Sci Inf Syst 2022;19:47–63. https://doi.org/10.2298/CSIS201221043N.
• 35. Dewi C, Chen RC. Human activity recognition based on evolution of features selection and random forest. Conf Proc - IEEE Int Conf Syst Man Cybern 2019;2019-October:2496–501. https://doi.org/10.1109/SMC.2019.8913868.
• 36. Bouchabou D, Nguyen SM, Lohr C, Leduc B, Kanellos I. A survey of human activity recognition in smart homes based on IoT sensors algorithms: taxonomies, challenges, and opportunities with deep learning. Sensors 2021;21:6037. https://doi.org/10.3390/S21186037.
• 37. Kumar P. Human activity recognition with deep learning: overview, Challenges & Possibilities 2021. https://doi.org/10.20944/PREPRINTS202102.0349.V1.
• 38. Fasciglione A, Leotta M, Verri A. Improving activity recognition while reducing misclassification of unknown activities. Proc Work Enabling Technol Infrastruct Collab Enterp WETICE 2021;2021-October:153–8. https://doi.org/10.1109/WETICE53228.2021.00039.
• 39. Lateef R, Abbas A. Tuning the hyperparameters of the 1D CNN Model to improve the performance of human activity recognition. Eng Technol J 2022;40:547–54. https://doi.org/10.30684/ETJ.V40I4.2054.
• 40. Ahmad I, Basheri M, Iqbal MJ, Rahim A. Performance comparison of support vector machine, random forest, and extreme learning machine for intrusion detection. IEEE Access 2018;6:33789–95. https://doi.org/10.1109/ACCESS.2018.2841987.
• 41. Janiesch C, Zschech P, Heinrich K. Machine learning and deep learning. Electron Mark 2021;31:685–95. https://doi.org/10.1007/S12525-021-00475-2/TABLES/2.
• 42. Zhang C, Li G, Qi Y, Wang S, Qing L, Huang Q, et al. Exploiting completeness and uncertainty of pseudo labels for weakly supervised video anomaly detection 2023:16271–80.
• 43. Qi Z, Zhu R, Fu Z, Chai W, Kindratenko V. Weakly supervised two-stage training scheme for deep video fight detection model. Proc - Int Conf Tools with Artif Intell ICTAI 2022;2022-October:677–85. https://doi.org/10.1109/ICTAI56018.2022.00105.
• 44. Liu Y, Zhuang C, Lu F. Unsupervised Two-Stage Anomaly Detection 2021.
• 45. Saleem MH, Velayudhan KK, Potgieter J, Arif KM. Weed identification by single-stage and two-stage neural networks: a study on the impact of image resizers and weights optimization algorithms. Front Plant Sci 2022;13:850666. https://doi.org/10.3389/FPLS.2022.850666/BIBTEX.
• 46. Dudukcu HV, Taskiran M, Gulru Cam Taskiran Z, Kahraman N. Human Activity Recognition with Ensemble Learning. 2024 26th Int Conf Digit Signal Process Its Appl DSPA 2024 2024. https://doi.org/10.1109/DSPA60853.2024.10510025.
• 47. Ghalan M, Aggarwal RK. Novel Human Activity Recognition by graph engineered ensemble deep learning model. IFAC J Syst Control 2024;27:100253. https://doi.org/10.1016/J.IFACSC.2024.100253.
• 48. Haresamudram H, Essa I, Plötz T. Assessing the state of self-supervised human activity recognition using wearables. Proc ACM Interactive, Mobile, Wearable Ubiquitous Technol 2022;6. https://doi.org/10.1145/3550299.
• 49. Khan MA, Javed K, Khan SA, Saba T, Habib U, Khan JA, et al. Human action recognition using fusion of multiview and deep features: an application to video surveillance. Multimed Tools Appl 2024;83:14885–911. https://doi.org/10.1007/S11042-020-08806-9/TABLES/7.
• 50. Dastbaravardeh E, Askarpour S, Saberi Anari M, Rezaee K. Channel attention-based approach with autoencoder network for human action recognition in low-resolution frames. Int J Intell Syst 2024;2024:1052344. https://doi.org/10.1155/2024/1052344.
• 51. Liu M, Li B, Yu Y. OmniCLIP: Adapting CLIP for Video Recognition with Spatial-Temporal Omni-Scale Feature Learning 2024.
• 52. Wu G, Wen C, Jiang H. Wushu Movement Recognition System Based on DTW Attitude Matching Algorithm. Entertain Comput 2025;52:100877. https://doi.org/10.1016/J.ENTCOM.2024.100877.
• 53. Uddin MA, Talukder MA, Uzzaman MS, Debnath C, Chanda M, Paul S, et al. Deep learning-based human activity recognition using CNN, ConvLSTM, and LRCN. Int J Cogn Comput Eng 2024;5:259–68. https://doi.org/10.1016/J.IJCCE.2024.06.004.
• 54. Butt UM, Ullah HA, Letchmunan S, Tariq I, Hassan FH, Koh TW. Leveraging transfer learning for spatio-temporal human activity recognition from video sequences. Comput Mater Contin 2022;74:5017–33. https://doi.org/10.32604/CMC.2023.035512.