View and Depth Preprocessing for View Synthesis Enhancement

• Autorzy: Dziembowski A. , Domański M.

Identyfikatory

DOI

10.24425/123518

• Języki publikacji: EN

Abstrakty

EN

In the paper, two preprocessing methods for virtual view synthesis are presented. In the first approach, both horizontal and vertical resolutions of the real views and the corresponding depth maps are doubled in order to perform view synthesis on images with densely arranged points. In the second method, real views are filtered in order to eliminate blurred or improperly shifted edges of the objects. Both methods are performed prior to synthesis, thus they may be applied to different Depth-Image-Based Rendering algorithms. In the paper, for both proposed methods, the achieved quality gains are presented.

Słowa kluczowe

EN

virtual view synthesis; free-viewpoint television; free navigation; depth-image-based rendering

• Wydawca: Polish Academy of Sciences, Committee of Electronics and Telecommunication
• Czasopismo: International Journal of Electronics and Telecommunications
• Rocznik: 2018
• Tom: Vol. 64, No. 3
• Strony: 269--275
• Opis fizyczny: Bibliogr. 25 poz., il., rys., tab.

Twórcy

autor

Dziembowski A.

• Chair of Multimedia Telecommunications and Microelectronics, Poznan University of Technology, Poznań, Poland

autor

Domański M.

• Chair of Multimedia Telecommunications and Microelectronics, Poznan University of Technology, Poznań, Poland

Bibliografia

• [1] M. Tanimoto, M. P. Tehrani, T. Fujii, T. Yendo ”FTV for 3-D spatial communication”, Proc. IEEE, vol. 100, no. 4, pp. 905-917, 2012.
• [2] G. Lafruit, M. Domanski, K. Wegner, T. Grajek, T. Senoh, J. Jung, P. Kovacs, P. Goorts, L. Jorissen, A. Munteanu, B. Ceulemans, P. Carballeira, S. Garcia, M. Tanimoto, ”New visual coding exploration in MPEG: Super-MultiView and Free Navigation in Free viewpoint TV”, IST Electronic Imaging, Stereoscopic Displays and Applications XXVII, San Francisco 2016.
• [3] M. Domanski, M. Gotfryd, K. Wegner, ”View synthesis for multiview video transmission”, The Int. Conf. Image Proc., Computer Vision, and Pattern Recognition, 2009.
• [4] P. Goorts, M. Dumont, S. Rogmans, P. Bekaert, ”An end-to-end system for free viewpoint video for smooth camera transitions”, International Conference on 3D Imaging, IC3D 2012, Liege, Belgium, 3-5.12.2012.
• [5] M. Domanski, A. Dziembowski, A. Kuehn, M. Kurc, A. Luczak, D. Mieloch, J. Siast, O. Stankiewicz, K. Wegner, ”Experiments on acquisition and processing of video for free-viewpoint television”, 3DTV Conference 2014, Budapest, Hungary, 2-4.07.2014.
• [6] O. Stankiewicz, M. Domanski, A. Dziembowski, A. Grzelka, D. Mieloch, J. Samelak, ”A free-viewpoint television system for horizontal virtual navigation”, IEEE Transactions on Multimedia, early access, 2018.
• [7] T. Fujii, K. Mori, K. Takeda, K. Mase, M. Tanimoto, Y. Suenaga, ”Multipoint measuring system for video and sound - 100-camera and microphone system”, IEEE International Conference on Multimedia and Expo, ICME 2006, Toronto, Canada, 9-12.07.2006.
• [8] M. Tanimoto, ”Overview of FTV (free-viewpoint television)”, IEEE International Conference on Multimedia and Expo, ICME 2009, New York, United States, 28.06-3.07.2009.
• [9] Z. Sun; Ch. Jung ”Real-Time Depth-Image-Based Rendering on GPU,” 2015 International Conference on Cyber-Enabled Distributed Computing and Knowledge Discovery, Xi’an, 2015, pp: 324 – 328.
• [10] T. Senoh, K. Yamamoto, N. Tetsutani, H. Yasuda, K. Wegner, ”View Synthesis Reference Software (VSRS) 4.2 with improved inpainting and hole filling”, ISO/IEC JTC1/SC29/WG11 MPEG2013, M40657, Hobart, Australia, 3-7.04.2017.
• [11] A. Dziembowski, A. Grzelka, D. Mieloch, O. Stankiewicz, K. Wegner, M. Domanski, ”Multiview Synthesis – improved view synthesis for virtual navigation”, Picture Coding Symposium, 2016.
• [12] S. Zinger, Q. Do, P. de With, ”Free-viewpoint depth image based rendering”, Journal of Visual Communication and Image Representation, vol. 21, no. 5-6, pp. 533-541, 2010.
• [13] I. Ahn, C. Kim, ”A novel depth-based virtual view synthesis method for free viewpoint video”, IEEE Transactions on Broadcasting, vol. 59, pp. 614-626, 2013.
• [14] B. Ceulemans, S.P. Lu, G. Lafruit, A. Munteanu, ”Robust multiview synthesis for wide-baseline camera arrays”, IEEE Transactions on Multimedia, vol. PP, no. 99, pp. 1-1, 2018.
• [15] C. Lee, Y.S. Ho, ”Boundary filtering on synthesized views of 3D video”, 2nd International Conference on Future Generation Communication and Networking, FGCN 2008, Sanya, China, 13-15.12.2008.
• [16] K. Mueller, A. Smolic, K. Dix, P. Merkle, P. Kauff, T. Wiegand, ”View synthesis for advanced 3D video systems”, EURASIP Journal on Image and Video Processing, vol. 2008, pp. 1-11, 2008.
• [17] A. Heyden, M. Pollefeys, ”Multiple view geometry”, in: G. Medioni, S.B. Kang (eds.) ”Emerging Topics in Computer Vision”, Prentice Hall, s. 63-75, 2005.
• [18] A. Dziembowski, A. Grzelka, D. Mieloch, O. Stankiewicz, M. Domanski, ”Enhancing view synthesis with image and depth map upsampling”, International Conference on Systems, Signals and Image Processing, IWSSIP 2017, Pozna, Poland, 22-24.05.2017.
• [19] Z. Wang, A.C. Bovik, H.R. Sheikh, E.P. Simoncelli, ”Image quality assessment: from error visibility to structural similarity”, IEEE Transactions on Image Processing, vol. 13, no. 4, pp. 600-612, 04.2004.
• [20] P.T. Kovacs, A. Fekete, K. Lackner, V.K. Adhikarla, A. Zare, T. Balogh, ”[FTV AHG] Big Buck Bunny light-field test sequences”, ISO/IEC JTC1/SC29/WG11 MPEG2013, M36500,Warsaw, Poland, 22-26.06.2015.
• [21] M. Domanski, A. Dziembowski, A. Kuehn, M. Kurc, A. Luczak, D. Mieloch, J. Siast, O. Stankiewicz, K. Wegner, ”Poznan Blocks - a multiview video test sequence and camera parameters for free viewpoint television”, ISO/IEC JTC1/SC29/WG11 MPEG2014, M32243, San Jose, United States, 13-17.01.2014.
• [22] M. Domanski, A. Dziembowski, A. Grzelka, A. Luczak, D. Mieloch, O. Stankiewicz, K. Wegner, ”Multiview test video sequences for free navigation exploration obtained using pairs of cameras”, ISO/IEC JTC1/SC29/WG11 MPEG2016, M38247, Geneva, Switzerland, 30.05-3.06.2016.
• [23] C.L. Zitnick, S.B. Kang, M. Uyttendaele, S. Winder, R. Szeliski, ”Highquality video view interpolation using a layered representation”, ACM Transactions on Graphics, vol. 3, no. 23, pp. 600-608, 08.2004.
• [24] P. Goorts, ”Real-time adaptive plane sweeping for free viewpoint navigation in soccer scenes”, doctoral dissertation, Hasselt University, Belgium, pp. 175-186, 16.06.2014.
• [25] M. Domanski, T. Grajek, K. Klimaszewski, M. Kurc, O. Stankiewicz, J. Stankowski, K. Wegner, ”Poznan multiview video test sequences and camera parameters”, ISO/IEC JTC1/SC29/WG11 MPEG2009, M17050, Xi’an, China, 26-30.10.2009.

Uwagi

1. Opracowanie rekordu w ramach umowy 509/P-DUN/2018 ze środków MNiSW przeznaczonych na działalność upowszechniającą naukę (2018).

2. The presented work has been funded by the Polish Ministry of Science and Higher Education within the status activity (DS) task ”Theory and algorithms of multidimensional signal processing” in 2018.