Nonlinear Evolutionary PDE-Based Refinement of Optical Flow

• Autorzy: Doshi Hirak , Kiran Nori Uday

Identyfikatory

DOI

10.22630/MGV.2021.30.1.3

• Języki publikacji: EN

Abstrakty

EN

The goal of this paper is to propose two nonlinear variational models for obtaining a refined motion estimation from an image sequence. Both the proposed models can be considered as a part of a generalized framework for an accurate estimation of physics-based flow fields such as rotational and fluid flow. The first model is novel in the sense that it is divided into two phases: the first phase obtains a crude estimate of the optical flow and then the second phase refines this estimate using additional constraints. The correctness of this model is proved using an evolutionary PDE approach. The second model achieves the same refinement as the first model, but in a standard manner, using a single functional. A special feature of our models is that they permit us to provide efficient numerical implementations through the first-order primal-dual Chambolle-Pock scheme. Both the models are compared in the context of accurate estimation of angle by performing an anisotropic regularization of the divergence and curl of the flow respectively. We observe that, although both the models obtain the same level of accuracy, the two-phase model is more efficient. In fact, we empirically demonstrate that the single-phase and the two-phase models have convergence rates of order O(1/N²) and O(1/N) respectively.

Słowa kluczowe

EN

optical flow; evolutionary PDE; variational methods; primal-dual; convergence

• Wydawca: Institute of Information Technology of the Warsaw University of Life Sciences – SGGW
• Czasopismo: Machine Graphics and Vision
• Rocznik: 2021
• Tom: Vol. 30, No. 1/4
• Strony: 45--65
• Opis fizyczny: Bibliogr. 30 poz., fot., tab., wykr.

Twórcy

autor

Doshi Hirak

• Department of Mathematics and Computer Science, Sri Sathya Sai Institute of Higher Learning, Puttaparthi, Indi

• https://orcid.org/0000-0002-2735-0735

autor

Kiran Nori Uday

• Department of Mathematics and Computer Science, Sri Sathya Sai Institute of Higher Learning, Puttaparthi, Indi

• https://orcid.org/0000-0001-9991-4846

Bibliografia

• [1] G. Aubert, R. Deriche, and P. Kornprobst. Computing optical flow via variational techniques. SIAM Journal of Applied Mathematics, 60(1):156–182, 1999. doi:10.1137/S0036139998340170.
• [2] S. Baker, D. Scharstein, J. P. Lewis, et al. Optical Flow, 2021. https://vision.middlebury.edu/flow/. [Last accessed Dec 2021].
• [3] S. Baker, D. Scharstein, J. P. Lewis, S. Roth, M. Black, and R. Szeliski. A database and evaluation methodology for optical flow. International Journal of Computer Vision, 92:1–31, 2011. doi:10.1007/s11263-010-0390-2.
• [4] A. Bruhn and J. Weickert. Towards ultimate motion estimation: Combining highest accuracy with real-time performance. In Proc. 10th IEEE Int. Conf. Computer Vision ICCV 2005, volume 1, pages 749–755. IEEE, Beijing, China, 17-21 Oct, 2005. doi:10.1109/ICCV.2005.240.
• [5] M. Burger, H. Dirks, and L. Frerking. On optical flow models for variational motion estimation. In M. Bergounioux, G. Peyré, C. Schnörr, J. Caillau, and T. Haberkorn, editors, Variational Methods In Imaging and Geometric Control, pages 225–251. De Gruyter, Berlin, Boston, 2017. doi:10.1515/9783110430394-007.
• [6] A. Chambolle. An algorithm for total variation minimization and applications. Journal of Mathematical Imaging and Vision, 20:89–97, 2004. doi:10.1023/B:JMIV.0000011325.36760.1e.
• [7] A. Chambolle and T. Pock. A first-order primal-dual algorithm for convex problems with applications in imaging. Journal of Mathematical Imaging and Vision, 40:120–145, 2011. doi:10.1007/s10851-010-0251-1.
• [8] I. Cohen. Nonlinear variational method for optical flow computation. InProc. 8th Scandinavian Conference on Image Analysis SCIA 1993, pages 523–530. IAPR, Tromso, Norway, 1993. https://hal.inria.fr/inria-00615717.
• [9] T. Corpetti, D. Heitz, G. Arroyo, E. Mémin, and A. Santa-Cruz. Fluid experimental flow estimation based on an optical flow scheme. Experiments in Fluids, 40:80–97, 2006. doi:10.1007/s00348-005-0048-y.
• [10] T. Corpetti, E. Mémin, and P. Pérez. Estimating fluid optical flow. In Proc. 15th Int. Conf. Pattern Recognition, ICPR 2000, volume 3, pages 1033–1036. IEEE, Barcelona, Spain, 3-7 Sep, 2000. doi:10.1109/ICPR.2000.90372
• [11] H. Dirks. Variational Methods for Joint Motion Estimation and Image Reconstruction. PhD thesis, Wilhems-Universitat, 2015.
• [12] H. Doshi and N. Uday Kiran. A framework for fluid motion estimation using a constraint-based refinement approach, 2022. arXiv:2011.12267v2. doi:10.48550/arXiv.2011.12267.
• [13] T. Goldstein, M. Li, X. Yuan, E. Esser, and R. Baraniuk. Adaptive primal-dual hybrid gradient methods for saddle-point problems, 2015. arXiv:1305.0546v2. doi:10.48550/arXiv.1305.0546.
• [14] D. Heitz, E. Mémin, and C. Schnörr. Variational fluid flow measurements from image sequences: Synopsis and perspectives. Experiments in Fluids, 48:369–393, 2010. doi:10.1007/s00348-009-0778-3.
• [15] W. Hinterberger, O. Scherzer, C. Schnörr, and J. Weickert. Analysis of optical flow models in the framework of calculus of variations. Numerical Functional Analysis and Optimization, 23(1-2):69–89, 2002. doi:10.1081/NFA-120004011.
• [16] B. K. P. Horn and B. G. Schunck. Determining optical flow. Artificial Intelligence, 17(1-3):185–203, 1981. doi:10.1016/0004-3702(81)90024-2.
• [17] A. Kumar, A. Tannenbaum, and G. Balas. Optical flow: A curve evolution approach. IEEE Transactions of Image Processing, 5(4):598–610, 1996. doi:10.1109/83.491336.
• [18] T. Liu. Open Optical Flow: An open source program for extraction of velocity fields from flow visualization images. Journal of Open Research Software, 5(1):29, 2017. doi:10.5334/jors.168.
• [19] T. Liu and L. Shen. Fluid flow and optical flow. Journal of Fluid Mechanics, 614:253–291, 2008. doi:10.1017/S0022112008003273.
• [20] A. Luttman, E. M. Bollt, R. Basnayake, S. Kramer, and N. B. Tufillaro. A framework for estimating potential fluid flow from digital imagery. Chaos: An Interdisciplinary Journal of Nonlinear Science, 23(3):033134, 2013. doi:10.1063/1.4821188.
• [21] A. Martin, E. Schiavi, and S. Segura de Leon. On 1-Laplacian elliptic equations modeling magnetic resonance imaging Rician denoising. Journal of Mathematical Imaging and Vision, 57:202–224, 2017. doi:10.1007/s10851-016-0675-3.
• [22] B. McCaine. Optical Flow Algorithm Evaluation. Computer Vision Research Group, Department of Computer Science, University of Otago Dunedin, New Zealand, 2021. http://of-eval.sourceforge.net/. [Last accessed Dec 2021].
• [23] C. Schnörr. Determining optical flow for irregular domains by minimizing quadratic functionals of a certain class. International Journal of Computer Vision, 6:25–38, 1991. doi:10.1007/BF00127124.
• [24] D. Sun, R. Roth, and M. J. Black. Secrets of optical flow estimation and their principles. In Proc. IEEE Computer Society Conf. Computer Vision and Pattern Recognition CVPR 2010, pages 2432–2439. IEEE, San Francisco, CA, USA, 13-18 Jun, 2010. doi:10.1109/CVPR.2010.5539939.
• [25] A. Wedel, T. Pock, C. Zach, H. Bischof, and D. Cremers. An improved algorithm for TV-L¹ optical flow. In D. Cremers et al., editors, Statistical and Geometrical Approaches to Visual Motion Analysis. Proc. International Dagstuhl Seminar, volume 5604 of Lecture Notes in Computer Science, pages 23–45, Dagstuhl Castle, Germany, Jul 13-18, 2008. Springer Berlin Heidelberg, 2009. doi:10.1007/978-3-642-03061-12.
• [26] L. Wei, R. P. Agarwal, and P. J. Y. Wong. Existence and iterative construction of solutions to non-linear Dirichlet boundary value problems with p-Laplacian operator. Complex Variables and Elliptic Equations: An International Journal, 55(5-6):601–608, 2010. doi:10.1080/17476930802657632.
• [27] C. Zach, T. Pock, and H. Bischof. A duality based approach for realtime TV-L¹ optical flow. In . A. Hamprecht et al., editors, Pattern Recognition. Proc. 29th DAGM Joint Pattern Recognition Symposium, volume 4713 of Lecture Notes in Computer Science, pages 214–223, Heidelberg, Germany, 12-14 Sep, 2007. Springer Berlin Heidelberg. doi:10.1007/978-3-540-74936-322.
• [28] B. Zhang and Z. Zhu. A primal-dual algorithm framework for convex saddle-point optimization. Journal of Inequalities and Applications, Article number: 267, 2017. doi:10.1186/s13660-017-1548-z.
• [29] X. Zhang, M. Burger, and S. Osher. A unified primal-dual algorithm framework based on Bregman iteration. Journal of Scientific Computing, 46:20–46, 2011. doi:10.1007/s10915-010-9408-8.
• [30] J. Zhao, Y. Wang, and H. Wang. Optical flow with harmonic constraint and oriented smoothness. In Proc. 6th Int. Conf. Image and Graphics ICIG 2011, pages 94–99, Hefei, China, 12-15 Aug, 2011. doi:10.1109/ICIG.2011.12