Fast Depth Image Based Rendering for synthetic frame extrapolation

• Autorzy: Meder J. , Brüderlin B.
• Języki publikacji: EN

Abstrakty

EN

A persistent problem of interactive computer graphics is feasible rendering of massive (hundreds of millions of primitives) scenes with hardware capabilities and scene data size concurrently growing. While past attempts yielded algorithms and systems achieving realtime frame rates on basic consumer workstations, the display updates can still be too temporally sparse (depending on the scene and underlying hardware) for some applications. Both the professional CAD industry and other sectors like entertainment show a growing demand for stable high frame rates, for example equal or above the 60hz mark. Common reasons for this are increased visual comfortability and less user irritation due to reduced input lag. The work described in this paper focuses on fulfilling this demand in the context of CAD software. A fast custom application of the Depth Image Based Rendering (DIBR) technique is used to construct a view synthesis system, which generates a view image sequence at the required rate using past frames of a classical renderer running in parallel. Artifacts typically associated with this technique are also addressed in a cost-effective manner.

Słowa kluczowe

EN

image based rendering; depth; extrapolation; interactive frame rate

• Wydawca: Komisja Informatyki Polskiej Akademii Nauk, Oddział w Gdańsku
• Czasopismo: Journal of Theoretical and Applied Computer Science
• Rocznik: 2016
• Tom: Vol. 10, nr 3
• Strony: 3--18
• Opis fizyczny: Bibliogr. 29 poz., rys., tab., wz.

Twórcy

autor

Meder J.

• Computer Graphics Group, Ilmenau University of Technology, Ilmenau, Germany
• 3DInteractive GmbH, Ilmenau, Germany

autor

Brüderlin B.

• Computer Graphics Group, Ilmenau University of Technology, Ilmenau, Germany

Bibliografia

• [1] Brüderlin, B., Heyer, M., Pfützner, S.: Interviews3D: A Platform for Interactive Handling of Massive Data Sets. IEEE Computer Graphics and Applications, 27(6), pp. 48–59, 2007. ISSN 0272-1716.
• [2] Funkhouser, T. A., Sequin, C. H.: Adaptive Display Algorithm for Interactive Frame Rates During Visualization of Complex Virtual Environments. In: Proceedings of the 20th Annual Conference on Computer Graphics and Interactive Techniques, SIGGRAPH ’93, pp. 247–254. ACM, New York, NY, USA, 1993. ISBN 0-89791-601-8.
• [3] Lee, Y.-L., Nguyen, T.: High frame rate Motion Compensated Frame Interpolation in High-Definition video processing. In: Acoustics Speech and Signal Processing (ICASSP), 2010 IEEE International Conference on, pp. 858–861. 2010. ISSN 1520-6149.
• [4] Fehn, C.: A 3D-TV Approach Using Depth-Image-Based Rendering (DIBR). In: Proceedings of Visualization, Imaging and Image Processing, pp. 482–487. 2003.
• [5] Mark,W. R., McMillan, L., Bishop, G.: Post-rendering 3D Warping. In: Proceedings of the 1997 Symposium on Interactive 3D Graphics, I3D ’97, pp. 7–ff. ACM, New York, NY, USA, 1997. ISBN 0-89791-884-3.
• [6] Popescu, V., Lastra, A., Aliaga, D., Neto, M. D. O.: Efficient Warping for Architectural Walkthroughs Using Layered Depth Images. In: IEEE Visualization, pp. 211–215. 1998.
• [7] Koppel, M., Wang, X., Doshkov, D., Wiegand, T., Ndjiki-Nya, P.: Depth image-based rendering with spatio-temporally consistent texture synthesis for 3-D video with global motion. In: Image Processing (ICIP), 2012 19th IEEE International Conference on, pp. 2713–2716. 2012. ISSN 1522-4880.
• [8] Oh, K.-J., Yea, S., Ho, Y.-S.: Hole filling method using depth based in-painting for view synthesis in free viewpoint television and 3-D video. In: Picture Coding Symposium, 2009. PCS 2009, pp. 1–4. 2009.
• [9] Zhang, L., Tam, W. J.: Stereoscopic image generation based on depth images for 3D TV. Broadcasting, IEEE Transactions on, 51(2), pp. 191–199, 2005. ISSN 0018-9316.
• [10] Horng, Y.-R., Tseng, Y.-C., Chang, T.-S.: Stereoscopic images generation with directional Gaussian filter. In: Circuits and Systems (ISCAS), Proceedings of 2010 IEEE International Symposium on, pp. 2650–2653. 2010.
• [11] Ndjiki-Nya, P., Koppel, M., Doshkov, D., Lakshman, H., Merkle, P., Muller, K., Wiegand, T.: Depth Image-Based Rendering With Advanced Texture Synthesis for 3-D Video. Multimedia, IEEE Transactions on, 13(3), pp. 453–465, 2011. ISSN 1520-9210.
• [12] Xi, M.,Wang, L.-H., Yang, Q.-Q., Li, D.-X., Zhang, M.: Depth-image-based rendering with Spa tial and temporal texture synthesis for 3DTV. EURASIP Journal on Image and Video Processing, 2013(1), pp. 1–18, 2013.
• [13] Zinger, S., Do, L., de With, P. H. N.: Free-viewpoint Depth Image Based Rendering. J. Vis. Comun. Image Represent., 21(5-6), pp. 533–541, 2010. ISSN 1047-3203.
• [14] Jin, J., Wang, A., Zhao, Y., Lin, C., Zeng, B.: Region-Aware 3-D Warping for DIBR. IEEE Transactions on Multimedia, 18(6), pp. 953–966, 2016. ISSN 1520-9210.
• [15] Chen, S. E., Williams, L.: View Interpolation for Image Synthesis. In: Proceedings of the 20th Annual Conference on Computer Graphics and Interactive Techniques, SIGGRAPH ’93, pp. 279–288. ACM, New York, NY, USA, 1993. ISBN 0-89791-601-8.
• [16] Decoret, X., Schaufler, G., Sillion, F., Dorsey, J.: Multi-Layered Impostors for Accelerated Rendering. In: EUROGRAPHICS ’99, volume 18. 1999.
• [17] Darsa, L., Costa Silva, B., Varshney, A.: Navigating Static Environments Using Image-space Simplification and Morphing. In: Proceedings of the 1997 Symposium on Interactive 3D Graphics, I3D ’97, pp. 25–ff. ACM, New York, NY, USA, 1997. ISBN 0-89791-884-3.
• [18] Fu, C.,Wong, T., Heng, P.: Triangle-based View Interpolation Without Depth-Buffering. Journal of Graphics Tools, 3, pp. 13–31, 1998.
• [19] Ghiletiuc, J., Färber, M., Brüderlin, B.: Real-time Remote Rendering of Large 3D Models on Smartphones Using Multi-layered Impostors. In: Proceedings of the 6th International Conference on Computer Vision / Computer Graphics Collaboration Techniques and Applications, MIRAGE ’13, pp. 14:1–14:8. ACM, New York, NY, USA, 2013. ISBN 978-1-4503-2023-8.
• [20] Didyk, P., Ritschel, T., Eisemann, E., Myszkowski, K., Seidel, H.-P.: Adaptive image-space stereo view synthesis. In: Vision, Modeling and Visualization Workshop, pp. 299–306. Siegen, Germany, 2010.
• [21] Chang, C.-F., Ger, S.-H.: Enhancing 3D Graphics on Mobile Devices by Image-Based Rendering. In: Proceedings of the Third IEEE Pacific Rim Conference on Multimedia: Advances in Multimedia Information Processing, PCM ’02, pp. 1105–1111. Springer-Verlag, London, UK, UK, 2002. ISBN 3-540-00262-6.
• [22] Westover, L.: Footprint Evaluation for Volume Rendering. In: Proceedings of the 17th Annual Conference on Computer Graphics and Interactive Techniques, SIGGRAPH ’90, pp. 367–376. ACM, New York, NY, USA, 1990. ISBN 0-89791-344-2.
• [23] Morvan, Y., Farin, D., de With, P.: System architecture for free-viewpoint video and 3D-TV. Consumer Electronics, IEEE Transactions on, 54(2), pp. 925–932, 2008. ISSN 0098-3063.
• [24] Wang, S.: An unidirectional Criminisi Algorithm for DIBR-synthesized images. In: 2016 2nd IEEE International Conference on Computer and Communications (ICCC), pp. 574–578. 2016.
• [25] Max, N., Ohsaki, K.: Rendering Trees from Precomputed Z-Buffer Views. In: In Eurographics Rendering Workshop, pp. 45–54. 1995.
• [26] Su, C. L., Wu, J. H., Chen, K. P.: Interframe hole filling for DIBR in 3D videos. In: 2015 IEEE International Conference on Consumer Electronics - Taiwan, pp. 386–387. 2015.
• [27] Saito, T., Takahashi, T.: Comprehensible Rendering of 3-D Shapes. In: Proceedings of the 17th Annual Conference on Computer Graphics and Interactive Techniques, SIGGRAPH ’90, pp. 197–206. ACM, New York, NY, USA, 1990. ISBN 0-89791-344-2.
• [28] Liu, F., Huang, M.-C., Liu, X.-H., Wu, E.-H.: Single Pass Depth Peeling via CUDA Rasterizer. In: SIGGRAPH 2009: Talks, SIGGRAPH ’09, pp. 79:1–79:1. ACM, New York, NY, USA, 2009. ISBN 978-1-60558-834-6.
• [29] Meder, J., Brüderlin, B.: Decoupling Rendering and Display Using Fast Depth Image Based Rendering on the GPU. In: Computer Vision and Graphics: International Conference, ICCVG 2016, Warsaw, Poland, September 19-21, 2016, Proceedings, pp. 61–72. 2016. ISBN 978-3-319-46418-3.

Uwagi

Opracowanie ze środków MNiSW w ramach umowy 812/P-DUN/2016 na działalność upowszechniającą naukę (zadania 2017).