A Survey of Passive 3D Reconstruction Methods on the Basis of More than One Image

• Autorzy: Siudak M. , Rokita P.

Identyfikatory

DOI

10.22630/MGV.2014.23.3.5

• Języki publikacji: EN

Abstrakty

EN

The research on the 3D scene reconstruction on the basis of its images and video recordings has been in progress for many years. As a result there is a number of methods concerning how to manage the reconstruction problem. This article's goal is to present the most important methods of reconstruction including stereo vision, shape from motion, shape from defocus, shape form silhouettes. shape from photo-consistency. All the algorithms explained in this article can be used on images taken with casual cameras in an ordinary illuminated scene (passive methods).

Słowa kluczowe

EN

3D reconstruction; stereo vision; shape from motion; shape from defocus; shape from silhouettes; shape from photo-consistency; shape from X

• Wydawca: Institute of Information Technology of the Warsaw University of Life Sciences – SGGW
• Czasopismo: Machine Graphics and Vision
• Rocznik: 2014
• Tom: Vol. 23, No. 3/4
• Strony: 57--117
• Opis fizyczny: Bibliogr. 89 poz., il., rys.

Twórcy

autor

Siudak M.

• Military University of Technology, Institute of Teleinformatics and Automatics, Warsaw, Poland

autor

Rokita P.

• Warsaw University of Technology, Faculty of Electronics and Information Technology, Institute of Computer Science, Warsaw, Poland

• https://orcid.org/0000-0002-4433-2133

Bibliografia

• [1] Kruppa E. : Zur Ermittlung eines Objektes aus zwei Perspektiven mit innerer Orientierung. math.naturw.Abt.IIa, 122:1939-1948, 1913.
• [2] Baumgart B.G.: Geometric Modeling for Computer Vision. Stanford University, 1974.
• [3] Fischler M.A : Bolles R.C., Random sample consensus: a paradigm for model fitting with applications to image analysis and automated cartography. Communications of the ACM, 24:381-395, 1981.
• [4] Longuet-Higgins H.C. : A computer algorithm for reconstructing a scene from two projections. Nature, 1981, 133-135, 1981.
• [5] Kass M., Witkin A., Terzopoulos D. : Snakes: Active contour models. International Journal of Computer Vision, 1:321-331, 1988.
• [6] Subbarao M. : Parallel Depth Recovery By Changing Camera Parameters. Second International Conference on Computer Vision, 1988, p. 149-155, 1988.
• [7] Tomasi C., Kanade T. : Shape and motion from image streams: a factorization method: full report on the orthographic case. Cornell University, 1992.
• [8] Tomasi C., Kanade T. : Shape and motion from image streams under orthography: a factorization method. International Journal of Computer Vision, 9:137-154, 1992.
• [9] Poelman C.J., Kanade T. : A paraperspective factorization method for shape and motion recovery. DTIC Document, 1993.
• [10] Szeliski R. : Rapid octree construction from image sequences. CVGIP: Image Understanding, 58:23-32, 1993.
• [11] Laurentini A. : The visual hull concept for silhouette-based image understanding. IEEE Transactions on Pattern Analysis and Machine Intelligence, 16:150-162, 1994.
• [12] Subbarao M., SURYA G. : Depth from defocus: a spatial domain approach. International Journal of Computer Vision, 13:271-294, 1994.
• [13] Hartley R.I. : In defence of the 8-point algorithm. Proc. 5th International Conference on Computer Vision, p. 1064-1070, 1995.
• [14] Nayar S.K., Watanabe M., Noguchi M. : Real-time focus range sensor. Proc. 5th International Conference on Computer Vision, p. 995-1001, 1995.
• [15] Collins R.T. : A space-sweep approach to true multi-image matching. Proc. IEEE Computer Society Conference on Computer Vision and Pattern Recognition CVPR'96, p. 358-363, 1996.
• [16] Sturm P., Triggs B. : A factorization based algorithm for multi-image projective structure and motion. Proc. European Conference on Computer Vision ECCV'96, p. 709-720, 1996.
• [17] Triggs B. : Factorization methods for projective structure and motion. Proc. IEEE Computer Society Conference on Computer Vision and Pattern Recognition CVPR'96, p. 845-851, 1996.
• [18] Watanabe M., Nayar S.K. : Minimal operator set for passive depth from defocus. Proc. IEEE Computer Society Conference on Computer Vision and Pattern Recognition CVPR'96, p. 431-438, 1996.
• [19] Seitz S.M., Dyer C.R. : Photorealistic scene reconstruction by voxel coloring. Proc. Conference on Computer Vision and Pattern Recognition, p. 1067-1073, 1997.
• [20] Torr P.H.S., Murray D.W. : The development and comparison of robust methods for estimating the fundamental matrix. International Journal of Computer Vision, 24:271-300, 1997.
• [21] Rajagopalan A.N., Chaudhuri S. : Optimal recovery of depth from defocused images using an MRF model. Proc. 6th International Conference on Computer Vision, p. 1047-1052, 1998.
• [22] Tomasi C., Manduchi R. : Bilateral filtering for gray and color images. Proc. 6th International Conference on Computer Vision, p. 839-846, 1998.
• [23] Watanabe M., Nayar S.K. : Rational filters for passive depth from defocus. International Journal of Computer Vision, 27:203-225, 1998.
• [24] Zhang Z. : Determining the epipolar geometry and its uncertainty: A review. International Journal of Computer Vision, 27:161-195, 1998.
• [25] Hartley R.I. : Theory and practice of projective rectication. International Journal of Computer Vision, 35:115-127, 1999.
• [26] ISgro F., Trucco E. : On robust rectication for uncalibrated images. Proc. International Conference on Image Analysis and Processing, p.297-302, 1999.
• [27] Favaro P., Soatto S. : Shape and radiance estimation from the information divergence of blurred images. Proc. European Conference on Computer Vision ECCV 2000, p. 755-768, 2000.
• [28] Fusiello A., Trucco E., Verri A. : A compact algorithm for rectification of stereo pairs. Machine Vision and Applications, 12:16-22, 2000.
• [29] Koch R., Pollefeys M., Van Gool L. : Realistic surface reconstruction of 3D scenes from uncalibrated image sequences. The Journal of Visualization and Computer Animation, 11:115-127, 2000.
• [30] Kutulakos K.N., Seitz S.M. : A theory of shape by space carving. International Journal of Computer Vision, 38:199-218, 2000.
• [31] Boykov Y., Jolly M.P. : Interactive graph cuts for optimal boundary & region segmentation of objects in ND images. Proc. 8th IEEE International Conference on Computer Vision, ICCV 2001, 1, p. 105-112, 2001.
• [32] Boykov Y., Veksler O., Zabih R. : Fast approximate energy minimization via graph cuts. IEEE Transactions on Pattern Analysis and Machine Intelligence, 23:1222-1239, 2001.
• [33] Kuzu Y., Rodehorst V. : Volumetric modeling using shape from silhouette. Proc. 4th Turkish-German Joint Geodetic Days, p. 469-476, 2001.
• [34] Oram D. : Rectification for any epipolar geometry. Pro. British Machine Vision Conference BMVC 2001, 1, p. 653-662, 2001.
• [35] Weickert J., Schnorr C. : Variational optic flow computation with a spatio-temporal smoothness constraint. Journal of Mathematical Imaging and Vision, 14:245-255, 2001.
• [36] Cheung G.K.M.: Visual hull construction, alignment and refirement across time. Technical Report CMU-RI-TR-02-05, Carnegie Mellon University, 2002.
• [37] Favaro P., Soatto S. : Learning shape from defocus. Proc. European Conference on Computer Vision ECCV 2002, p. 823-824, 2002.
• [38] Jin H., Favaro P. : A variational approach to shape from defocus. Proc. European Conference on Computer Vision ECCV 2002, p. 18-30, 2002.
• [39] Matusik W., Buehler C., Mcmillan L., Gortler S.J.: An efficient visual hull computation algorithm. MIT LCS Technical Memo 623, MIT Laboratory for Computer Science, Cambridge, 2002.
• [40] Tarini M., Callieri M., Montani C., Rocchini C., Olsson K., Persson T. : Marching Intersections: An Efficient Approach to Shape-from-Silhouette. Proc. Vision, Modeling, and Visualization Conference VMV 2002, p. 283-290, 2002.
• [41] Cheung K.M.G., Visual hull construction, alignment and refinement for human kinematic modeling, motion tracking and rendering. Citeseer, 2003.
• [42] Cheung G.K.M, Baker S., Kanade T. : Visual hull alignment and refinement across time: A 3D reconstruction algorithm combining shape-from-silhouette with stereo. Proc. IEEE Computer Society Conference on Computer Vision and Pattern Recognition, 2, p. 375-382, 2003.
• [43] Grauman K., Shakhnarovich G., Darrell T. : A bayesian approach to image-based visual hull reconstruction. Proc. IEEE Computer Society Conference on Computer Vision and Pattern Recognition, 1, p. 187-195, 2003.
• [44] Hartley R., Zisserman A.: Multiple view geometry in computer vision. Cambridge University Press, 2003.
• [45] Hemayed E.E., A survey of camera self-calibration. Proc. IEEE Conference on Advanced Video and Signal Based Surveillance, p. 351-357, 2003.
• [46] Hernandez E.C., Schmitt F. : Silhouette and stereo fusion for 3D object modeling. Computer Vision and Image Understanding, 96:367-392, 2004.
• [47] Nister D. : An Efficient solution to the ve-point relative pose problem. IEEE Transactions on Pattern Analysis and Machine Intelligence, 26:756-770, 2004.
• [48] Bruhn A., Weickert J., Feddern C., Kohlberger T., Schnorr C. : Variational optical flow computation in real time. IEEE Transactions on Image Processing, 14:608-615, 2005.
• [49] Franco J.S., Boyer E. : Fusion of multiview silhouette cues using a space occupancy grid. Proc. 10Th IEEE International Conference on Computer Vision ICCV 2005, 2, p. 1747-1753, 2005.
• [50] Mallon J., Whelan P.F. : Projective rectication from the fundamental matrix. Image and Vision Computing, 23:643-650, 2005.
• [51] Mercier B., Meneveaux D.: Shape from silhouette: Image pixels for marching cubes. Vaclav Skala - UNION Agency, 2005.
• [52] Rzeszotarski D., Strumillo P., Pelczynski P., Wiecek B., Lorenc A. : Stereovision system for 3D reconstruction of image sequneces (in Polish). Zeszyty Naukowe Elektronika, 10:165-184, 2006.
• [53] Guan Li, Sinha S., Franco J.S., Pollefeys M. : Visual hull construction in the presence of partial occlusion. Proc. 3rd International Symposium on 3D Data Processing, Visualization, and Transmission, p. 413-420, 2006.
• [54] Kangni F., Laganiere R. : Projective rectication of image triplets from the fundamental matrix. Proc. IEEE International Conference on Acoustics, Speech and Signal Processing ICASSP 2006, 2, p. II, 2006.
• [55] Michoud B., Guillou E., Bouakaz S. : Shape from silhouette: Towards a solution for partial visibility problem. Proc. of Eurographics, p. 13-16, 2006.
• [56] Montenegro A.A., Velho L., Carvalho P.C.P., Sossai J. : Polygonization of volumetric reconstructions from silhouettes. Proc. 19th Brazilian Symposium on Computer Graphics and Image Processing, SIBGRAPI 06, p. 11-18, 2006.
• [57] Snavely N., Seitz S.M., Szeliski R. : Photo tourism: exploring photo collections in 3D. ACM transactions on graphics (TOG), 25:835-846, 2006.
• [58] Szeliski R., Zabih R., Scharstein D., Veksler O., Kolmogorov V., Agarwala A., Tappen M., Rother C. : A comparative study of energy minimization methods for markov random elds. Proc. European Conference on Computer Vision ECCV 2006, p. 16-29, 2006.
• [59] Weickert J., Bruhn A., Brox T., Papenberg N. : A survey on variational optic flow methods for small displacements. In: Mathematical models for registration and applications to medical imaging, O. Scherzer (Ed.). Springer. p. 103-136, 2006.
• [60] Favaro P. : Shape from focus and defocus: Convexity, quasiconvexity and defocus-invariant textures. Proc. IEEE 11th International Conference on Computer Vision ICCV 2007, p. 1-7, 2007.
• [61] Hallmann I.: Lokalizacja robota mobilnego wzgldem automatycznie wybieranych obiektw. Ph.D. Thesis. IPPT PAN, Warszawa, 2007.
• [62] Kolmogorov V., Rother C. : Minimizing nonsubmodular functions with graph cuts-a review. IEEE Transactions on Pattern Analysis and Machine Intelligence, 29:1274-1279, 2007.
• [63] Mileva Y., Bruhn A., Weickert J. : Illumination-robust variational optical ow with photometric invariants. Proc. 29th DAGM Symposium on Pattern Recognition, Lecture Notes in Computer Science Vol. 4713, p. 152-162, 2007.
• [64] Pons J.P., Keriven R., Faugeras O. : Multi-view stereo reconstruction and scene ow estimation with a global image-based matching score. International Journal of Computer Vision, 72:179-193, 2007.
• [65] Vogiatzis G., Hernandez C., Torr P.H.S., Cipolla R. : Multiview stereo via volumetric graph-cuts and occlusion robust photo-consistency. IEEE Transactions on Pattern Analysis and Machine Intelligence, 29:2241-2246, 2007.
• [66] Chica A., Williams J., Andujar C., Brunet P., Navazo I., Rossignac J., Vinacua A. : Pressing: Smooth isosurfaces with flats from binary grids. Computer Graphics Forum, 27:36-46, 2008.
• [67] Favaro P., Soatto S., Burger M., Osher S.J. : Shape from defocus via diffusion. IEEE Transactions on Pattern Analysis and Machine Intelligence, 30:518-531, 2008.
• [68] Fusiello A., Irsara L. : Quasi-euclidean uncalibrated epipolar rectication. Proc. 19th International Conference on Pattern Recognition ICPR 2008, p. 1-4, 2008.
• [69] Kolev K., Cremers D. : Integration of multiview stereo and silhouettes via convex functionals on convex domains. Proc. European Conference on Computer Vision ECCV 2008, p. 752-765, 2008.
• [70] Lin Huei-Yung, Wu Jing-Ren : 3d reconstruction by combining shape from silhouette with stereo. Proc. 19th International Conference on Pattern Recognition ICPR 2008, p. 1-4, 2008.
• [71] Franco J.S., Boyer E. : Efficient polyhedral modeling from silhouettes. IEEE Transactions on Pattern Analysis and Machine Intelligence, 31:414-427, 2009.
• [72] Liansheng S., Jiulong Z., Duwu C. : Image rectication using affine epipolar geometric constraint. Journal of Software, 4:27, 2009.
• [73] Campbell N.D.F., Vogiatzis G., Hernandez C., Cipolla R. : Automatic 3D object segmentation in multiple views using volumetric graph-cuts. Image and Vision Computing, 28:14-25, 2010.
• [74] Favaro P. : Recovering thin structures via nonlocal-means regularization with application to depth from defocus. Proc. IEEE Conference on Computer Vision and Pattern Recognition CVPR 2010, p. 1133-1140, 2010.
• [75] Haro G., Pardas M. : Shape from incomplete silhouettes based on the reprojection error. Image and Vision Computing, 28:1354-1368, 2010.
• [76] Lempitsky V. : Surface extraction from binary volumes with higher-order smoothness. Proc. IEEE Conference on Computer Vision and Pattern Recognition CVPR 2010, p. 1197-1204, 2010.
• [77] Stuhmer J., Gumhold S., Cremers D. : Real-time dense geometry from a handheld camera. Proc. 32nd DAGM Symposium on Pattern Recognition, Lecture Notes in Computer Science Vol. 6376, p. 11-20, 2010.
• [78] Agarwal S., Furukawa Y., Snavely N., Simon I., Curless B., Seitz S.M., Szeliski R. : Building rome in a day. Communications of the ACM, 54:105-112, 2011.
• [79] Ben-Ari R., Raveh G. : Variational depth from defocus in real-time. Proc. IEEE International Conference on Computer Vision Workshops ICCV Workshops 2011, p. 522-529, 2011.
• [80] Brox T., Malik J. : Large displacement optical flow: descriptor matching in variational motion estimation. IEEE Transactions on Pattern Analysis and Machine Intelligence, 33:500-513, 2011.
• [81] Delong A. : Advances in graph-cut optimization: multi-surface models, label costs, and hierarchical costs (Spine title: Advances in Graph-Cut Optimization). Citeseer, 2011.
• [82] Wu Changchang, Agarwal S., Curless B., Seitz S.M. : Multicore bundle adjustment. Proc. IEEE Conference on Computer Vision and Pattern Recognition CVPR 2011, p. 3057-3064, 2011.
• [83] Haro G. : Shape from Silhouette Consensus. Pattern Recognition, 45:3231-3244, 2012.
• [84] Valgaerts L., Bruhn A., Mainberger M., Weickert J. : Dense versus sparse approaches for estimating the fundamental matrix. International Journal of Computer Vision, 96:212-234, 2012.
• [85] Khan W. : Image Segmentation Techniques: A Survey. Journal of Image and Graphics, 1(4):166-170, 2013.
• [86] Kim D., Ruttle J., Dahyot R. : Bayesian 3d shape from silhouettes. Digital Signal Processing, 23:1844-1855, 2013.
• [87] Nieradka G.: Dopasowanie obrazw pary stereoskopowej z wykorzystaniem logiki rozmytej. Politechnika Warszawska Wydzia Elektroniki i Technik Informacyjnych, 2013.
• [88] Peng B., Zhang L., Zhang D. : A survey of graph theoretical approaches to image segmentation. Pattern Recognition, 46:1020-1038, 2013.
• [89] Singh M., Misal A. : A Survey Paper on Various Visual Image Segmentation Techniques. International Journal of Computer Science and Management Research, 2:1282-1288, 2013.