STAFGCN: a spatial-temporal attention-based fusion graph convolution network for pedestrian trajectory prediction

• Autorzy: Lui Guihong , Pan Chenying , Zhang Xiaoyan , Leng Qiangkui

Identyfikatory

DOI

10.24425/bpasts.2024.151960

• Języki publikacji: EN

Abstrakty

EN

Pedestrian trajectory prediction provides crucial data support for the development of smart cities. Existing pedestrian trajectory prediction methods often overlook the different types of pedestrian interactions and the micro-level spatial-temporal relationships when handling the interaction information in spatial and temporal dimensions. The model employs a spatial-temporal attention-based fusion graph convolutional framework to predict future pedestrian trajectories. For the different types of local and global relationships between pedestrians, it first employs spatial-temporal attention mechanisms to capture dependencies in pedestrian sequence data, obtaining the social interactions of pedestrians in spatial contexts and the movement trends of pedestrians over time. Subsequently, a fusion graph convolutional module merges the temporal weight matrix and the spatial weight matrix into a spatial-temporal fusion feature map. Finally, a decoder section utilizes time-stacked convolutional neural networks to predict future trajectories. The final validation on the ETH and UCY datasets yielded experimental results with an average displacement error (ADE) of 0.34 and an final displacement error (FDE) of 0.55. The visualization results further demonstrated the rationality of the model.

Słowa kluczowe

EN

pedestrian trajectory prediction; micro-level spatial-temporal relationship; spatial-temporal attention; fusion graph convolution; time-stacked convolutional neural network

PL

przewidywanie trajektorii pieszych; mikropoziomowa relacja przestrzenno-czasowa; przestrzenno-czasowa uwaga; splot grafów fuzyjnych; splotowa sieć neuronowa

• Wydawca: Polska Akademia Nauk, Wydział IV Nauk Technicznych
• Czasopismo: Bulletin of the Polish Academy of Sciences. Technical Sciences
• Rocznik: 2025
• Tom: Vol. 73, nr 1
• Strony: art. no. e151960
• Opis fizyczny: Bibliogr. 38 poz., rys., tab.

Twórcy

autor

Lui Guihong

• School of Electronic and Information Engineering, Liaoning Technical University, Huludao, Liaoning, China

autor

Pan Chenying

• School of Electronic and Information Engineering, Liaoning Technical University, Huludao, Liaoning, China

• https://orcid.org/0009-0006-8415-2838

autor

Zhang Xiaoyan

• School of Electronic and Information Engineering, Liaoning Technical University, Huludao, Liaoning, China

autor

Leng Qiangkui

• School of Electronic and Information Engineering, Liaoning Technical University, Huludao, Liaoning, China

Bibliografia

• [1] J. Hong, B. Sapp, and J. Philbin, “Rules of the road: Predicting driving behavior with a convolutional model of semantic inter-actions,” in Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2019, pp. 8454–8462, doi: 10.1109/cvpr.2019.00865.
• [2] M. Golchoubian, M. Ghafurian, K. Dautenhahn, and N.L. Azad, “Pedestrian trajectory prediction in pedestrian-vehicle mixed environments: A systematic review,” IEEE Trans. Intell. Transp. Syst., vol. 24, no. 11, pp. 11 544–11 567, 2023, doi: 10.1109/tits.2023.3291196.
• [3] H. Xue, D. Huynh, and M. Reynolds, “Location-velocity attention for pedestrian trajectory prediction,” in 2019 IEEE Winter Conference on Applications of Computer Vision (WACV), 2019, pp. 2038–2047, doi: 10.1109/wacv.2019.00221.
• [4] C.K. Williams and C.E. Rasmussen, Gaussian processes for machine learning. MIT press Cambridge, MA, 2006, vol. 2, no. 3, doi: 10.7551/mitpress/3206.001.0001.
• [5] S. Lefèvre, C. Laugier, and J. Ibañez-Guzmán, “Exploiting map information for driver intention estimation at road intersections,” in 2011 IEEE Intelligent Vehicles Symposium (IV), 2011, pp. 583–588, doi: 10.1109/ivs.2011.5940452.
• [6] R. Toledo-Moreo and M.A. Zamora-Izquierdo, “Imm-based lane-change prediction in highways with low-cost gps/ins,” IEEE Trans. Intell. Transp. Syst., vol. 10, no. 1, pp. 180–185, 2009, doi: 10.1109/tits.2008.2011691.
• [7] R. Korbmacher and A. Tordeux, “Review of pedestrian trajectory prediction methods: Comparing deep learning and knowledge-based approaches,” IEEE Trans. Intell. Transp. Syst., vol. 23, no. 12, pp. 24 126–24 144, 2022, doi: 10.1109/tits.2022.3205676.
• [8] J. Donahue et al., “Long-term recurrent convolutional networks for visual recognition and description,” in Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR), June 2015, doi: 10.1109/cvpr.2015.7298878.
• [9] R. Emonet, J. Varadarajan, and J.-M. Odobez, “Extracting and locating temporal motifs in video scenes using a hierarchical non parametric bayesian model,” in CVPR 2011, 2011, pp. 3233–3240, doi: 10.1109/cvpr.2011.5995572.
• [10] T. Fernando, S. Denman, S. Sridharan, and C. Fookes, “Soft + hardwired attention: An lstm framework for human trajectory prediction and abnormal event detection,” Neural Netw., vol. 108, pp. 466–478, 2018, doi: 10.1016/j.neunet.2018.09.002.
• [11] S. Zamboni, Z.T. Kefato, S. Girdzijauskas, C. Norén, and L. Dal Col, “Pedestrian trajectory prediction with convolutional neural networks,” Pattern Recognit., vol. 121, p. 108252, 2022, doi: 10.1016/j.patcog.2021.108252.
• [12] A. Alahi, K. Goel, V. Ramanathan, A. Robicquet, L. Fei-Fei, and S. Savarese, “Social lstm: Human trajectory prediction in crowded spaces,” in Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR), June 2016, doi: 10.1109/cvpr.2016.110.
• [13] A. Gupta, J. Johnson, L. Fei-Fei, S. Savarese, and A. Alahi, “Social gan: Socially acceptable trajectories with generative adversarial networks,” in Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR), June 2018, doi: 10.1109/cvpr.2018.00240.
• [14] J. Li, H. Ma, and M. Tomizuka, “Conditional generative neural system for probabilistic trajectory prediction,” in 2019 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), 2019, pp. 6150–6156, doi: 10.1109/iros40897.2019.8967822.
• [15] A. Sadeghian, V. Kosaraju, A. Sadeghian, N. Hirose, H. Rezatofighi, and S. Savarese, “Sophie: An attentive gan for predicting paths compliant to social and physical constraints,” in Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), June 2019, doi: 10.1109/cvpr.2019.00144.
• [16] S. Bai, J.Z. Kolter, and V. Koltun, “An empirical evaluation of generic convolutional and recurrent networks for sequence modeling,” CoRR abs/1803.01271, 2018, doi: 10.48550/arXiv.1803.01271.
• [17] V. Kosaraju, A. Sadeghian, R. Martín-Martín, I. Reid, H. Rezatofighi, and S. Savarese, “Social-bigat: Multimodal trajectory forecasting using bicycle-gan and graph attention networks,” in Advances in Neural Information Processing Systems, vol. 32. Curran Associates, Inc., 2019. [Online]. Available: https://proceedings.neurips.cc/paper_files/paper/2019/file/d09bf41544a3365a46c9077ebb5e35c3-Paper.pdf
• [18] A. Mohamed, K. Qian, M. Elhoseiny, and C. Claudel, “Social-stgcnn: A social spatio-temporal graph convolutional neural network for human trajectory prediction,” in Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), June 2020, doi: 10.1109/cvpr42600.2020.01443.
• [19] H. Zhou, D. Ren, H. Xia, M. Fan, X. Yang, and H. Huang, “Astgnn: An attention-based spatio-temporal graph neural network for interaction-aware pedestrian trajectory prediction,” Neurocomputing, vol. 445, pp. 298–308, 2021, doi: 10.1016/j.neucom.2021.03.024.
• [20] D. Helbing and P. Molnár, “Social force model for pedestrian dynamics,” Phys. Rev. E, vol. 51, pp. 4282–4286, May 1995, doi: 10.1103/physreve.51.4282.
• [21] G. Antonini, M. Bierlaire, and M. Weber, “Discrete choice models of pedestrian walking behavior,” Transp. Res. Pt. B-Methodol., vol. 40, no. 8, pp. 667–687, 2006, doi: 10.1016/j.trb.2005.09.006.
• [22] A. Treuille, S. Cooper, and Z. Popović, “Continuum crowds,” ACM Trans. Graph., vol. 25, no. 3, pp. 1160–1168, 2006, doi: 10.1145/1179352.1142008.
• [23] S. Yi, H. Li, and X. Wang, “Pedestrian behavior understanding and prediction with deep neural networks,” in Computer Vision – ECCV 2016, B. Leibe, J. Matas, N. Sebe, and M. Welling, Eds., Cham: Springer International Publishing, 2016, pp. 263–279, doi: 10.1007/978-3-319-46448-0_16.
• [24] A. Alahi, V. Ramanathan, and L. Fei-Fei, “Socially-aware large-scale crowd forecasting,” in Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR), June 2014, doi: 10.1109/cvpr.2014.283.
• [25] Y. Xu, Z. Piao, and S. Gao, “Encoding crowd interaction with deep neural network for pedestrian trajectory prediction,” in Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR), June 2018, doi: 10.1109/cvpr.2018.00553.
• [26] P. Zhang, W. Ouyang, P. Zhang, J. Xue, and N. Zheng, “Sr-lstm: State refinement for lstm towards pedestrian trajectory prediction,” in Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), June 2019, doi: 10.1109/cvpr.2019.01236.
• [27] Q. Wang, S. Liu, J. Chanussot, and X. Li, “Scene classification with recurrent attention of vhr remote sensing images,” IEEE Trans. Geosci. Remote Sensing, vol. 57, no. 2, pp. 1155–1167, 2019, doi: 10.1109/tgrs.2018.2864987.
• [28] Q. Wang, T. Han, Z. Qin, J. Gao, and X. Li, “Multitask attention network for lane detection and fitting,” IEEE Trans. Neural Netw. Learn. Syst., vol. 33, no. 3, pp. 1066–1078, 2022, doi: 10.1109/tnnls.2020.3039675.
• [29] L. Shi et al., “Sgcn: Sparse graph convolution network for pedestrian trajectory prediction,” in Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), June 2021, pp. 8994–9003, doi: 10.1109/cvpr46437.2021.00888.
• [30] W. Chen, H. Sang, J. Wang, and Z. Zhao, “Stigcn: spatial–temporal interaction-aware graph convolution network for pedestrian trajectory prediction,” J. Supercomput., vol. 80, pp. 10 695–10 719, 2024, doi: 10.1007/s11227-023-05850-8.
• [31] H. Sang, W. Chen, J. Wang, and Z. Zhao, “Rdgcn: Reasonably dense graph convolution network for pedestrian trajectory prediction,” Measurement, vol. 213, p. 112675, 2023, doi: 10.1016/j.measurement.2023.112675.
• [32] Y. Huang, H. Bi, Z. Li, T. Mao, and Z. Wang, “Stgat: Modeling spatial-temporal interactions for human trajectory prediction,” in Proceedings of the IEEE/CVF International Conference on Computer Vision (ICCV), October 2019, doi: 10.1109/iccv.2019.00637.
• [33] S. Yan, Y. Xiong, and D. Lin, “Spatial temporal graph convolutional networks for skeleton-based action recognition,” in Proceedings of the AAAI conference on artificial intelligence, vol. 32, no. 1, 2018, doi: 10.1609/aaai.v32i1.12328.
• [34] L. Wu, C. Zhang, and Y. Zou, “Spatiotemporal focus for skeleton-based action recognition,” Pattern Recognit., vol. 136, p. 109231, 2023, doi: 10.1016/j.patcog.2022.109231.
• [35] X. Yin, G. Wu, J. Wei, Y. Shen, H. Qi, and B. Yin, “Multi-stage attention spatial-temporal graph networks for traffic prediction,” Neurocomputing, vol. 428, pp. 42–53, 2021, doi: 10.1016/j.neucom.2020.11.038.
• [36] J. Liang, L. Jiang, J.C. Niebles, A.G. Hauptmann, and L. Fei-Fei, “Peeking into the future: Predicting future person activities and locations in videos,” in Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, 2019, pp. 5725–5734, doi: 10.1109/CVPRW.2019.00358x.
• [37] S. Pellegrini, A. Ess, K. Schindler, and L. Van Gool, “You’ll never walk alone: Modeling social behavior for multi-target tracking,” in 2009 IEEE 12th international conference on computer vision. IEEE, 2009, pp. 261–268, doi: 10.1109/ICCV.2009.5459260.
• [38] A. Lerner, Y. Chrysanthou, and D. Lischinski, “Crowds by example,” in Computer graphics forum, vol. 26, no. 3. Wiley Online Library, 2007, pp. 655–664, doi: 10.1111/j.1467-8659.2007.01089.x.

Uwagi

Opracowanie rekordu ze środków MNiSW, umowa nr POPUL/SP/0154/2024/02 w ramach programu "Społeczna odpowiedzialność nauki II" - moduł: Popularyzacja nauki (2025).