Multi-frame Image Super-resolution Reconstruction Using Multi-grained Cascade Forest

• Autorzy: Wang Yaming , Luo Zhikang , Huang Wenqing

Identyfikatory

DOI

10.24425/ijet.2019.130250

• Języki publikacji: EN

Abstrakty

EN

Super-resolution image reconstruction utilizes two algorithms, where one is for single-frame image reconstruction, and the other is for multi-frame image reconstruction. Singleframe image reconstruction generally takes the first degradation and is followed by reconstruction, which essentially creates a problem of insufficient characterization. Multi-frame images provide additional information for image reconstruction relative to single frame images due to the slight differences between sequential frames. However, the existing super-resolution algorithm for multi-frame images do not take advantage of this key factor, either because of loose structure and complexity, or because the individual frames are restored poorly. This paper proposes a new SR reconstruction algorithm for images using Multi-grained Cascade Forest. Multi-frame image reconstruction is processed sequentially. Firstly, the image registration algorithm uses a convolutional neural network to register low-resolution image sequences, and then the images are reconstructed after registration by the Multi-grained Cascade Forest reconstruction algorithm. Finally, the reconstructed images are fused. The optimal algorithm is selected for each step to get the most out of the details and tightly connect the internal logic of each sequential step. This novel approach proposed in this paper, in which the depth of the cascade forest is procedurally generated for recovered images, rather than being a constant. After training each layer, the recovered image is automatically evaluated, and new layers are constructed for training until an optimal restored image is obtained. Experiments show that this method improves the quality of image reconstruction while preserving the details of the image.

Słowa kluczowe

EN

multi-frame image SR; image registration; SRMCF; image fusion

• Wydawca: Polish Academy of Sciences, Committee of Electronics and Telecommunication
• Czasopismo: International Journal of Electronics and Telecommunications
• Rocznik: 2019
• Tom: Vol. 65, No. 4
• Strony: 687--692
• Opis fizyczny: Bibliogr. 20 poz., rys., fot., tab.

Twórcy

autor

Wang Yaming

• School of Information Science and Technology, Zhejiang Sci-Tech University, Hangzhou, China

autor

Luo Zhikang

• Machine vision and pattern recognition laboratory, Zhejiang Sci-Tech University, Hangzhou, China

autor

Huang Wenqing

• School of Information Science and Technology, Zhejiang Sci-Tech University, Hangzhou, China

Bibliografia

• [1] R. C. Gonzalez,and R. E. Woods , ”Digital image processingr”, 3rd ed. Upper Saddle River, NJ, USA: Prentice-Hall, 2008.
• [2] M. Jung.,A. Marquina, and L. A. Vese, ”Variational multiframe restoration of images degraded by noisy (stochastic) blur kernels”, J. Comput. Appl. Math vol. 240, no. 1, pp. 45–67, 2015.
• [3] Y. Fan , X. Wei, and S. Qin, ”Fast and robust deblurring method with multi-frame images based on PSF estimation and total variation optimization”, Optik-Int. J. Light Electron Opt vol. 124, no. 16, pp. 2285–2291, 2013.
• [4] F. W. Wheeler, X. Liu, P. H. Peter, and R. T. Hoctor, ”Multiframe image restoration for face recognition”, in Proc. IEEE Workshop Signal Process. Appl. Public Secur. Forensics,pp. 16, Apr. 2007.
• [5] R.Y. Tsai,and T.S. Huang, ”Multipleframe image restoration and registration”, Advances in computer vision and image processing. Greenwich, CT: JAI Press Inc., 1984, pp. 317-339.
• [6] N.K. Bose, H.C. Kim,and H.M. Valenzuela, ”Recursive implementation of total least squares algorithm for image reconstruction from noisy, undersampled multiframes”, in Proc. IEEE Conf. Acoustics, Speech and Signal Processing, Minneapolis, MN, 1993, pp. 269-272.
• [7] S.H. Rhee, and M.G. Kang, ”Discrete cosine transform based regularized high-resolution image reconstruction algorithm”, Opt. Eng vol. 38, no. 8, pp. 1348–1356, 1999.
• [8] H. Hanchuan, W. Peikang, ”Super-resolution image reconstruction based on an improvement frequency domain registration approach”, Electron Meas Technol vol. 33, no. 5, pp. 58–61, 2010.
• [9] P. Vanderwalle, Sbaiz L., J. Vanderwalle, ”Super-resolution from unregistered and totally aliased signals using subspace methods”, IEEE Trans. Signal Procss vol. 55, no. 70, pp. 3687–3703, 2007.
• [10] C.C. Chen, ”A multi-frame super-resolution algorithm using POCS and wavelet”,Montreal: Concordia University, 2010.
• [11] C. Liu, D. Q. Sun, ”A bayesian approach to adaptive video super resolution”, IEEE Conf. Comput. Vis Pattern Recognit, vol.42, no. 7, pp. 209-216, 2011.
• [12] F. Shi, J. Cheng, L. Wang, ”Low-rank total variation for image superresolution”, Proc. Conf. Med. Image Comput. and Comput-Assist Interv, vol.16, no. 1, pp. 155-162, 2013.
• [13] A. Kappeler, S. Yoo, Q. Dai, ”Video super-resolution with convolutional neural networks”, IEEE Trans. Comput Image vol. 2, no. 2, pp. 1–1, 2016.
• [14] C. Dong, C.L. Chen, K. He, ”Image super-resolution using deep convolutianal networks, IEEE Trans Pattern Anal Mach Intell vol. 38, no. 2, pp. 295–307, 2016.
• [15] W. Shi, J. Caballero, F. Huszar, ”Real-time single image and video superresolution using an efficient sub-pixel convolutional neural network”, IEEE Conf Comput Vis Pattern Recognit pp.1874-1883, 2016.
• [16] J. Caballero, C. Ledig, A. Aitken, ”Real-time video super-resolution with spatio-temporal networks and motion compensation”, Comput Vis Pattern Recognit, 2017, arXiv preprint arXiv:1611. 05250v2.
• [17] S. Jeong, I. Yoon, J. Joonki, ”Multi-frame example-based superresolution using locally directional self-similarity, IEEE Trans. Consum. Electron vol. 61, no. 3, pp. 353–358, 2015.
• [18] Y. M. Wang , Z. K. Luo, ”Image SR based on MultiGrained Cascade Forests. Multiresolution and Information Processing”.
• [19] X. Liu , J. Yang , J. Sun, ”Image registration approach based on SIFT, Infrared and Laser Eng vol. 37, no. 1, pp. 156–160, 2008.
• [20] Z. H. Zhou, J. Feng, ”Deep forest: Towards an alternative to deep neural networks, arXiv preprint arXiv:1702.08835, 2017.

Uwagi

This work was supported by the Natural Science Foundation of Zhejiang Province (LZ15F020004), the Natural Science Foundation of China (61272311) and 521 Project of Zhejiang Sci-Tech University.