A data association model for analysis of crowd structure

• Autorzy: Zitouni M. Sami , Śluzek Andrzej

Identyfikatory

DOI

10.34768/amcs-2022-0007

• Języki publikacji: EN

Abstrakty

EN

The paper discusses a non-deterministic model for data association tasks in visual surveillance of crowds. Using detection and tracking of crowd components (i.e., individuals and groups) as baseline tools, we propose a simple algebraic framework for maintaining data association (continuity of labels assigned to crowd components) between subsequent video-frames in spite of possible disruptions and inaccuracies in tracking/detection algorithms. Formally, two alternative schemes (which, in practice, can be jointly used) are introduced, depending on whether individuals or groups can be prospectively better tracked in the current scenario. In the first scheme, only individuals are tracked, and the continuity of group labels is inferred without explicitly tracking the groups. In the second scheme, only group tracking is performed, and associations between individuals are inferred from group tracking. The associations are built upon non-deterministic estimates of memberships (individuals in groups) and estimates obtained directly from the baseline detection and tracking algorithms. The framework can incorporate any detectors and trackers (both classical or DL-based) as long as they can provide some geometric outlines (e.g., bounding boxes) of the crowd components. The formal analysis is supported by experiments in sample scenarios, where the framework provides meaningful performance improvements in various crowd analysis tasks.

Słowa kluczowe

EN

data association; visual surveillance; crowd analysis; algebraic model

PL

powiązanie danych; nadzór wizyjny; analiza tłumu; tryb algebraiczny

• Wydawca: Oficyna Wydawnicza Uniwersytetu Zielonogórskiego
• Czasopismo: International Journal of Applied Mathematics and Computer Science
• Rocznik: 2022
• Tom: Vol. 32, no. 1
• Strony: 81--94
• Opis fizyczny: Bibliogr. 43 poz., rys., tab., wykr.

Twórcy

autor

Zitouni M. Sami

• Department of Biomedical Engineering Khalifa University of Science and Technology PO Box 127788, Abu Dhabi, United Arab Emirates

autor

Śluzek Andrzej

• Institute of Information Technology Warsaw University of Life Sciences (SGGW) ul. Nowoursynowska 166, 02-787 Warsaw, Poland

Bibliografia

• [1] Bazzani, L., Cristani, M. and Murino, V. (2012). Decentralized particle filter for joint individual-group tracking, IEEE Conference on Computer Vision and Pattern Recognition, Providence, USA, pp. 1886–1893, DOI: 10.1109/CVPR.2012.6247888.
• [2] Benfold, B. and Reid, I. (2011). Stable multi-target tracking in real-time surveillance video, Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Colorado Springs, USA, pp. 3457–3464, DOI: 10.1109/CVPR.2011.5995667.
• [3] Berclaz, J., Fleuret, F., Turetken, E. and Fua, P. (2011). Multiple object tracking using k-shortest paths optimization, IEEE Transactions on Pattern Analysis and Machine Intelligence 33(9): 1806–1819, DOI: 10.1109/TPAMI.2011.21.
• [4] Bochinski, E., Senst, T. and Sikora, T. (2018). Extending IOU based multi-object tracking by visual information, 15th IEEE International Conference on Advanced Video and Signal Based Surveillance (AVSS), Auckland, New Zealand, pp. 1–6, DOI: 10.1109/AVSS.2018.8639144.
• [5] Ciaparrone, G., Luque Sanchez, F., Tabik, S., Troiano, L., Tagliaferri, R. and Herrera, F. (2020). Deep learning in video multi-object tracking: A survey, Neurocomputing 381: 61–88, DOI: 10.1016/j.neucom.2019.11.023.
• [6] Dalal, N. and Triggs, B. (2005). Histograms of oriented gradients for human detection, IEEE Conference on Computer Vision and Pattern Recognition, San Diego, USA, Vol. 1, pp. 886–893, DOI: 10.1109/CVPR.2005.177.
• [7] Dehghan, A., Modiri Assari, S. and Shah, M. (2015). GMMCP tracker: Globally optimal generalized maximum multi clique problem for multiple object tracking, Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 4091–4099, DOI: 10.1109/CVPR.2015.7299036.
• [8] Dollár, P., Appel, R., Belongie, S. and Perona, P. (2014). Fast feature pyramids for object detection, IEEE Transactions on Pattern Analysis and Machine Intelligence 36(8): 1532–1545, DOI: 10.1109/TPAMI.2014.2300479.
• [9] Edman, V., Andersson, M., Granström, K. and Gustafsson, F. (2013). Pedestrian group tracking using the GM-PHD filter, European Signal Processing Conference (EUSIPCO), Marrakech, Morocco, pp. 1–5.
• [10] Ferryman, J. and Shahrokni, A. (2009). PETS2009: Dataset and challenge, 12th IEEE International Workshop on Performance Evaluation of Tracking and Surveillance, Snowbird, USA, DOI: 10.1109/PETS-WINTER.2009.5399556.
• [11] Garcia-Martin, A., Sanchez-Matilla, R. and Martinez, J.M. (2017). Hierarchical detection of persons in groups, Signal, Image and Video Processing 11(7): 1181–1188, DOI: 10.1007/s11760-017-1073-z.
• [12] Ge, W., Collins, R.T. and Ruback, R.B. (2012). Vision-based analysis of small groups in pedestrian crowds, IEEE Transactions on Pattern Analysis and Machine Intelligence 34(5): 1003–1016, DOI: 10.1109/TPAMI.2011.176.
• [13] Gong, S., Han, H., Shan, S. and Chen, X. (2016). Actions recognition in crowd based on coarse-to-fine multi-object tracking, Asian Conference on Computer Vision, Taipei, Taiwan, pp. 478–490, DOI: 10.1007/978-3-319-54526-4 35.
• [14] Heili, A. and Odobez, J.-M. (2013). Parameter estimation and contextual adaptation for a multi-object tracking CRF model, IEEE International Workshop on Performance Evaluation of Tracking and Surveillance (PETS), Clearwater, USA, pp. 14–21, DOI: 10.1109/PETS.2013.6523790.
• [15] Hofmann, M., Haag, M. and Rigoll, G. (2013). Unified hierarchical multi-object tracking using global data association, IEEE International Workshop on Performance Evaluation of Tracking and Surveillance (PETS), Clearwater, USA, pp. 22–28, DOI: 10.1109/PETS.2013.6523791.
• [16] Jacques, J.C.S., Braun, A., Soldera, J., Musse, S.R. and Jung, C.R. (2007). Understanding people motion in video sequences using Voronoi diagrams, Pattern Analysis and Applications 10(4): 321–332, DOI: 10.1007/s10044-007-0070-1.
• [17] Kasprzak, W., Wilkowski, A. and Czapnik, K. (2012). Hand gesture recognition based on free-form contours and probabilistic inference, International Journal of Applied Mathematics and Computer Science 22(2): 437–448, DOI: 10.2478/v10006-012-0033-6.
• [18] Li, D., Zhu, J., Xu, B., Lu, M. and Li, M. (2018). An ant-based filtering random-finite-set approach to simultaneous localization and mapping, International Journal of Applied Mathematics and Computer Science 28(3): 505–519, DOI: 10.2478/amcs-2018-0039.
• [19] Mazzon, R., Poiesi, F. and Cavallaro, A. (2013). Detection and tracking of groups in crowd, IEEE International Conference on Advanced Video and Signal Based Surveillance (AVSS), Krakow, Poland, pp. 202–207, DOI: 10.1109/AVSS.2013.6636640.
• [20] Milan, A., Roth, S. and Schindler, K. (2014). Continuous energy minimization for multitarget tracking, IEEE Transactions on Pattern Analysis and Machine Intelligence 36(1): 58–72, DOI: 10.1109/TPAMI.2013.103.
• [21] Park, M.-W. and Brilakis, I. (2016). Continuous localization of construction workers via integration of detection and tracking, Automation in Construction 72(Part 2): 129–142, DOI: 10.1016/j.autcon.2016.08.039.
• [22] Raj, K.S. and Poovendran, R. (2014). Pedestrian detection and tracking through hierarchical clustering, International Conference on Information Communication and Embedded Systems, Chennai, India, pp. 1–4, DOI: 10.1109/ICICES.2014.7033991.
• [23] Ren, W., Kang, D., Tang, Y. and Chan, A.B. (2018). Fusing crowd density maps and visual object trackers for people tracking in crowd scenes, IEEE Conference on Computer Vision and Pattern Recognition, Salt Lake City, USA, pp. 5353–5362, DOI: 10.1109/CVPR.2018.00561.
• [24] Rezatofighi, S.H., Milan, A., Zhang, Z., Shi, Q., Dick, A. and Reid, I. (2015). Joint probabilistic data association revisited, IEEE International Conference on Computer Vision (ICCV), Santiago, Chile, pp. 3047–3055, DOI: 10.1109/ICCV.2015.349.
• [25] Rodriguez, M., Sivic, J., Laptev, I. and Audibert, J.-Y. (2011). Data-driven crowd analysis in videos, 2011 International Conference on Computer Vision, Barcelona, Spain, pp. 1235–1242, DOI: 10.1109/ICCV.2011.6126374.
• [26] Shao, J., Change Loy, C. and Wang, X. (2014). Scene-independent group profiling in crowd, Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Columbus, USA, pp. 2219–2226, DOI: 10.1109/CVPR.2014.285.
• [27] Tang, S., Andriluka, M., Milan, A., Schindler, K., Roth, S. and Schiele, B. (2013). Learning people detectors for tracking in crowded scenes, Proceedings of the IEEE International Conference on Computer Vision, Sydney, Australia, pp. 1049–1056, DOI: 10.1109/ICCV.2013.134.
• [28] Wang, Q., Chen, M. and Li, X. (2017). Quantifying and detecting collective motion by manifold learning, AAAI Conference on Artificial Intelligence, San Francisco, USA, pp. 4292–4298, DOI: 10.5555/3298023.3298190.
• [29] Wang, Q., Chen, M., Nie, F. and Li, X. (2020a). Detecting coherent groups in crowd scenes by multiview clustering, IEEE Transactions on Pattern Analysis and Machine Intelligence 42(1): 46–58, DOI: 10.1109/TPAMI.2018.2875002.
• [30] Wang, Q., Gao, J., Lin, W. and Li, X. (2020b). NWPU-crowd: A large-scale benchmark for crowd counting and localization, IEEE Transactions on Pattern Analysis and Machine Intelligence 43(6): 2141–2149, DOI: 10.1109/TPAMI.2020.3013269.
• [31] Wang, Q., Gao, J., Lin, W. and Yuan, Y. (2020c). Pixel-wise crowd understanding via synthetic data, International Journal of Computer Vision 129(1): 225–245, DOI: 10.1007/s11263-020-01365-4.
• [32] Wen, L., Lei, Z., Lyu, S., Li, S. Z. and Yang, M.-H. (2016). Exploiting hierarchical dense structures on hypergraphs for multi-object tracking, IEEE Transactions on Pattern Analysis and Machine Intelligence 38(10): 1983–1996, DOI: 10.1109/TPAMI.2015.2509979.
• [33] Yang, B. and Nevatia, R. (2014). Multi-target tracking by online learning a CRF model of appearance and motion patterns, International Journal of Computer Vision 107(2): 203–217, DOI: 10.1007/s11263-013-0666-4.
• [34] Yu, H., Zhou, Y., Simmons, J., Przybyla, C.P., Lin, Y., Fan, X., Mi, Y. and Wang, S. (2016). Groupwise tracking of crowded similar-appearance targets from low-continuity image sequences, IEEE Conference on Computer Vision and Pattern Recognition, Las Vegas, USA, pp. 952–960, DOI: 10.1109/CVPR.2016.109.
• [35] Zhang, L., He, Z., Gu, M. and Yu, H. (2018). Crowd segmentation method based on trajectory tracking and prior knowledge learning, Arabian Journal for Science and Engineering 43(12): 7143–7152, DOI: 10.1007/s13369-017-2995-z.
• [36] Zhang, S., Wang, J., Wang, Z., Gong, Y. and Liu, Y. (2015). Multi-target tracking by learning local-to-global trajectory models, Pattern Recognition 48(2): 580–590, DOI: 10.1016/j.patcog.2014.08.013.
• [37] Zhou, X., Zhuo, J. and Krahenbuhl, P. (2019). Bottom-up object detection by grouping extreme and center points, 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), Long Beach, USA, pp. 850–859, DOI: 10.1109/CVPR.2019.00094.
• [38] Zhu, F., Wang, X. and Yu, N. (2014). Crowd tracking with dynamic evolution of group structures, European Conference on Computer Vision, Zurich, Switzerland, pp. 139–154, DOI: 10.1007/978-3-319-10599-4 10.
• [39] Zhu, F., Wang, X. and Yu, N. (2018). Crowd tracking by group structure evolution, IEEE Transactions on Circuits and Systems for Video Technology 28(3): 772–786, DOI: 10.1109/TCSVT.2016.2615460.
• [40] Zitouni, M.S., Bhaskar, H. and Al-Mualla, M.E. (2016). Robust background modeling and foreground detection using dynamic textures, International Conference on Computer Vision Theory and Applications (VISIGRAPP’16), Rome, Italy, pp. 403–410, DOI: 10.5220/0005724204030410.
• [41] Zitouni, M.S., Sluzek, A. and Bhaskar, H. (2019a). CNN-based analysis of crowd structure using automatically annotated training data, IEEE International Conference on Advanced Video and Signal Based Surveillance (AVSS), Taipei, Taiwan, pp. 1–8, DOI: 10.1109/AVSS.2019.8909846.
• [42] Zitouni, M.S., Sluzek, A. and Bhaskar, H. (2019b). Visual analysis of socio-cognitive crowd behaviors for surveillance: A survey and categorization of trends and methods, Engineering Applications of Artificial Intelligence 82: 294–312, DOI: 10.1016/j.engappai.2019.04.012.
• [43] Zitouni, M.S., Sluzek, A. and Bhaskar, H. (2020). Towards understanding socio-cognitive behaviors of crowds from visual surveillance data, Multimedia Tools and Applications 79(3): 1781–1799, DOI: 10.1007/s11042-019-08201-z.