A deep learning ball tracking system in soccer videos

• Autorzy: Kamble Paresh

Identyfikatory

DOI

10.1016/j.opelre.2019.02.003

• Języki publikacji: EN

Abstrakty

EN

Increasing interest, enthusiasm of sport lovers, and economics involved offer high importance to sports video recording and analysis. Being crucial for decision making, ball detection and tracking in soccer has become a challenging research area. This paper presents a novel deep learning approach for 2D ball detection and tracking (DLBT) in soccer videos posing various challenges. A new 2-stage buffer median filtering background modelling is used for moving objects blob detection. A deep learning approach for classification of an image patch into three classes, i.e. ball, player, and background is initially proposed. Probabilistic bounding box overlapping technique is proposed further for robust ball track validation. Novel full and boundary grid concepts resume tracking in ball_track_lost and ball_out_of_frame situations. DLBT does not require human intervention to identify ball from the initial frames unlike the most published algorithms. DLBT yields extraordinary accurate and robust tracking results compared to the other contemporary 2D trackers even in presence of various challenges including very small ball size and fast movements.

Słowa kluczowe

EN

ball detection; ball tracking; convolutional neural networks; deep learning; bounding box overlap

• Wydawca: Stowarzyszenie Elektryków Polskich ; Polska Akademia Nauk ; Wojskowa Akademia Techniczna im. Jarosława Dąbrowskiego
• Czasopismo: Opto-Electronics Review
• Rocznik: 2019
• Tom: Vol. 27, No. 1
• Strony: 58--69
• Opis fizyczny: Bibliogr. 54 poz., wykr., fot.

Twórcy

autor

Kamble Paresh

• Department of Electronics and Communication Engineering, Visvesvaraya National Institute of Technology (VNIT), South Ambazari Road, Nagpur, 440010, India

Bibliografia

• [1] World Cup 2018, World Cup 2018 Breaks Viewing Records Across StreamingPlatforms as Soccer Fans Tune in, 2018 (Accessed 23 November 2018) https://www.cnbc.com/2018/07/14/world-cup-2018-breaks-viewing-records-across-streaming-platforms-as-so.html.
• [2] P.R. Kamble, A.G. Keskar, K.M. Bhurchandi, Ball tracking in sports: a survey, Artif. Intell. Rev. (2017), http://dx.doi.org/10.1007/s10462-017-9582-2.
• [3] T. Kim, Y. Seo, K.-S. Hong, Physics-based 3D position analysis of a soccer ball from monocular image sequences, computer vision, in: Sixth International Conference on IEEE, 1998, 1998, pp. 721–726.
• [4] I.D. Reid, A. North, 3D Trajectories from a Single Viewpoint Using Shadows, BMVC, 1998, pp. 51–52.
• [5] Y. Ohno, J. Miura, Y. Shirai, Tracking players and estimation of the 3D position of a ball in soccer games, in: Pattern Recognition, 2000. Proceedings. 15th International Conference on IEEE, 2000, pp. 145–148.
• [6] G.S. Pingali, Y. Jean, I. Carlbom, Real time tracking for enhanced tennis broadcasts, computer vision and pattern recognition, in: Proc. 1998 IEEE Computer Society Conference on IEEE, 1998, 1998, pp. 260–265.
• [7] Y. Ohno, J. Miurs, Y. Shirai, Tracking players and a ball in soccer games, multisensor fusion and integration for intelligent systems, 1999, in: MFI’99. Proceedings. 1999 IEEE/SICE/RSJ International Conference on IEEE, 1999, pp. 147–152.
• [8] W. Chen, Y.-J. Zhang, Tracking ball and players with applications to highlight ranking of broadcasting table tennis video, in: Computational Engineering in Systems Applications, IMACS Multiconference on IEEE, 2006, pp. 1896–1903.
• [9] J. Ren, J. Orwell, G.A. Jones, M. Xu, Tracking the soccer ball using multiple fixed cameras, Comput. Vision Image Underst. 113 (2009) 633–642, http://dx.doi.org/10.1016/j.cviu.2008.01.007.
• [10] U.B. Desai, S.N. Merchant, M. Zaveri, G. Ajishna, M. Purohit, H. Phanish, Small object detection and tracking: algorithm, analysis and application, in: International Conference on Pattern Recognition and Machine Intelligence, 2005, pp. 108–117, Springer.
• [11] D. Liang, Q. Huang, Y. Liu, G. Zhu, W. Gao, Video2 Cartoon: a system for converting broadcast soccer video into 3D cartoon animation, IEEE Trans. Consum. Electr. 53 (2007) 1138–1146, http://dx.doi.org/10.1109/TCE.2007.4341597.
• [12] G. Zhu, C. Xu, Y. Zhang, Q. Huang, H. Lu, Event tactic analysis based on player and ball trajectory in broadcast video, in: Proceedings of the 2008 international conference on Content-based image and video retrieval, ACM, 2008, pp. 515–524.
• [13] Y. Huang, J. Llach, C. Zhang, A method of small object detection and tracking based on particle filters, in: PAttern Recognition, 2008. ICPR 2008. 19th International Conference on, IEEE, 2008, pp. 1–4.
• [14] G. Zhu, C. Xu, Q. Huang, Y. Rui, S. Jiang, W. Gao, H. Yao, Event tactic analysis based on broadcast sports video, IEEE Trans. Multimedia 11 (2009) 49–67, http://dx.doi.org/10.1109/TMM.2008.2008918.
• [15] X. Cheng, Y. Shiina, X. Zhuang, T. Ikenaga, Player tracking using prediction after intersection based particle filter for volleyball match video, Signal and Information Processing Association Annual Summit and Conference (APSIPA) (2014) 1–4.
• [16] G.S. Pingali, A. Opalach, Y.D. Jean, I.B. Carlbom, Instantly indexed multimedia databases of real world events, IEEE Trans. Multimedia 4 (2002) 269–282, http://dx.doi.org/10.1109/TMM.2002.1017739.
• [17] X. Yu, H.W. Leong, C. Xu, Q. Tian, Trajectory-based ball detection and tracking in broadcast soccer video, IEEE Trans. Multimedia 8 (2006) 1164–1178, http://dx.doi.org/10.1109/TMM.2006.884621.
• [18] Y. Liu, D. Liang, Q. Huang, W. Gao, Extracting 3D information from broadcast soccer video, Image Vision Comput. 24 (2006) 1146–1162, http://dx.doi.org/10.1016/j.imavis.2006.04.001.
• [19] B. Chakraborty, S. Meher, A real-time trajectory-based ball detection-and-tracking framework for basketball video, J. Opt. U. K. 42 (2013) 156–170, http://dx.doi.org/10.1007/s12596-012-0108-7.
• [20] C.Ó. Conaire, P. Kelly, D. Connaghan, N.E. O’Connor, Tennissense: a platform for extracting semantic information from multi-camera tennis data, in: 16th International Conference on Digital Signal Processing, IEEE, 2009, 2009, pp.1–6.
• [21] Q. Huang, S. Cox, X. Zhou, L. Xie, Detection of ball hits in a tennis game using audio and visual information, Signal & Information Processing Association Annual Summit and Conference (APSIPA ASC) (2012) 1–10.
• [22] F. Yan, W. Christmas, J. Kittler, Ball Tracking for Tennis Video Annotation, Computer Vision in Sports, Springer, 2014, pp. 25–45, http://dx.doi.org/10.1007/978-3-319-09396-3 2.
• [23] X. Zhou, Q. Huang, L. Xie, S. Cox, A two layered data association approach for ball tracking, in: IEEE International Conference on Acoustics, Speech and Signal Proc., IEEE, 2013, 2013, pp. 2317–2321.
• [24] J. Miura, T. Shimawaki, T. Sakiyama, Y. Shirai, Ball route estimation under heavy occlusion in broadcast soccer video, Comput. Vision Image Underst. 113 (2009) 653–662, http://dx.doi.org/10.1016/j.cviu.2008.10.005.
• [25] V. Pallavi, J. Mukherjee, A.K. Majumdar, S. Sural, Ball detection from broadcast soccer videos using static and dynamic features, J. Vision Commun. Image Represent. 19 (2008) 426–436, http://dx.doi.org/10.1016/j.jvcir.2008.06.007.
• [26] V. Pallavi, J. Mukherjee, A.K. Majumdar, S. Sural, Graph-based multiplayer detection and tracking in broadcast soccer videos, IEEE Trans. Multimedia 10 (2008) 794–805, http://dx.doi.org/10.1109/TMM.2008.922869.
• [27] X. Wang, V. Ablavsky, H.B. Shitrit, P. Fua, Take your eyes off the ball: improving ball-tracking by focusing on team play, Comput. Vision Image Underst. 119 (2014) 102–115, http://dx.doi.org/10.1016/j.cviu.2013.11.010.
• [28] X. Wang, E. Türetken, F. Fleuret, P. Fua, Tracking interacting objects optimally using integer programming, in: European Conference on Computer Vision, 2014, pp. 17–32, Springer.
• [29] A. Maksai, X. Wang, P. Fua, What Players Do with the Ball: a Physically Constrained Interaction Modeling, arXiv preprint arXiv:1511.06181, 2015.
• [30] M. Takahashi, S. Yokozawa, H. Mitsumine, T. Mishina, Real-time ball-position measurement using multi-view cameras for live football broadcast, Multimed. Tools Appl. (2018) 1–22, http://dx.doi.org/10.1007/s11042-018-5694-1.
• [31] V. Reno, N. Mosca, R. Marani, M. Nitti, T. D’Orazio, E. Stella, Convolutional neural networks based ball detection in tennis games, in: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition Workshops, 2018, pp. 1758–1764.
• [32] Y. LeCun, Y. Bengio, G. Hinton, Deep learning, Nature 521 (2015) 436, http://dx.doi.org/10.1038/nature14539.
• [33] I. Goodfellow, Y. Bengio, A. Courville, Y. Bengio, Deep Learning, MIT Press, Cambridge, 2016.
• [34] D.E. Rumelhart, G.E. Hinton, R.J. Williams, Learning representations by back-propagating errors, Nature 323 (1986) 533, http://dx.doi.org/10.1038/323533a0.
• [35] Y. Bengio, A. Courville, P. Vincent, Representation learning: a review and new perspectives, IEEE Trans. Pattern Anal. 35 (2013) 1798–1828, http://dx.doi.org/10.1109/TPAMI.2013.50.
• [36] J. Schmidhuber, Deep learning in neural networks: an overview, Neural Netw.61 (2015) 85–117, http://dx.doi.org/10.1016/j.neunet.2014.09.003.
• [37] R.S. Sutton, A.G. Barto, Reinforcement Learning: an Introduction, MIT Press Cambridge, 1998.
• [38] J. Patterson, A. Gibson, Deep Learning: A Practitioner’s Approach, first ed., O’Reilly Media, Inc., 2017.
• [39] A. Karpathy, CS231n Convolutional Neural Networks for Visual Recognition,2019 http://cs231n.github.io/.
• [40] V. Nair, G.E. Hinton, Rectified linear units improve restricted boltzmannmachines, in: Proc.27th International Conference on Machine Learning(ICML-10), 2010, pp. 807–814.
• [41] A. Krizhevsky, I. Sutskever, G.E. Hinton, Imagenet classification with deepconvolutional neural networks, Adv. Neural Inf. Process. Syst. (2012)1097–1105.
• [42] C.M. Bishop, Pattern Recognition and Machine Learning (Information Scienceand Statistics), Springer-Verlag, 2006.
• [43] K.P. Murphy, Machine Learning: A Probabilistic Perspective, MIT Press, 2012.
• [44] L.Y. Pratt, Discriminability-based transfer between neural networks, Adv.Neural Inf. Process. Syst. (1993) 204–211.
• [45] L. Pratt, B. Jennings, A survey of transfer between connectionist networks,Conn. Sci. 8 (1996) 163–184, http://dx.doi.org/10.1080/095400996116866.
• [46] K. Chatfield, K. Simonyan, A. Vedaldi, A. Zisserman, Return of the Devil in theDetails: Delving Deep into Convolutional Nets, arXiv preprintarXiv:1405.3531, 2014.
• [47] H. Nam, B. Han, Learning multi-domain convolutional neural networks forvisual tracking, computer vision and pattern recognition (CVPR), in: 2016IEEE Conference on, IEEE, 2016, pp. 4293–4302.
• [48] J. Deng, W. Dong, R. Socher, L.-J. Li, K. Li, L. Fei-Fei, Image net: a large-scalehierarchical image database, computer vision and pattern recognition, CVPR2009. IEEE Conference on, IEEE, 2009 (2009) 248–255, http://dx.doi.org/10.1109/CVPR.2009.5206848.
• [49] B. Babenko, M.-H. Yang, S. Belongie, Robust object tracking with onlinemultiple instance learning, IEEE Trans. Pattern Anal. 33 (2011) 1619–1632,http://dx.doi.org/10.1109/TPAMI.2010.226.
• [50] J.F. Henriques, R. Caseiro, P. Martins, J. Batista, High-speed tracking withkernelized correlation filters, IEEE Trans. Pattern Anal. 37 (2015) 583–596,http://dx.doi.org/10.1109/TPAMI.2014.2345390.
• [51] P. Jaccard, Étude comparative de la distribution florale dans une portion desAlpes et des Jura, Bull. Soc. Vaudoise Sci. Nat. 37 (1901) 547–579.
• [52] J. Redmon, S. Divvala, R. Girshick, A. Farhadi, You only look once: unified,real-time object detection, in: Proceedings of the IEEE Conference onComputer Vision and Pattern Recognition, 2016, pp. 779–788.
• [53] G.W.R.L, Fastest Football Kick, Guinness World Records Limited 2018, 2001,2018 (Accessed 23 November 2018) http://www.guinnessworldrecords.com/world-records/fastest-football-kick/.
• [54] T. D’Orazio, M. Leo, N. Mosca, P. Spagnolo, P.L. Mazzeo, A semi-automaticsystem for ground truth generation of soccer video sequences, advancedvideo and Signal based surveillance, in: AVSS’09. Sixth IEEE InternationalConference on, IEEE, 2009, 2009, pp. 559–564.

Uwagi

Opracowanie rekordu w ramach umowy 509/P-DUN/2018 ze środków MNiSW przeznaczonych na działalność upowszechniającą naukę (2019).