A Novel Two-stage Residual Learning Based Convolutional Neural Network for Image Super Resolution

• Autorzy: Sharma Shailza , Bawa Vivek Singh , Kumar Vinay

Identyfikatory

DOI

10.3233/FI-2019-1835

• Języki publikacji: EN

Abstrakty

EN

Image super resolution has gained a lot of attention due to its applications in different fields of image processing. It is used to produce high-resolution images from low-resolution input. Because of the excellent learning capability of convolution neural networks, these networks are able to learn complex spatial structures for image super-resolution. In this paper, two different architectures have been proposed for image super resolution. The first architecture is Dual Subpixel Layer Convolution Neural Network (DSL-CNN), which stacks two subpixel CNN architectures to enhance model depth for better representational capability. Two stages provide an effective upscaling factor of 4. In the second architecture, named as Residue based Dual Subpixel Layer Convolution Neural Network (RDSL-CNN), two-stage residual learning has been introduced which effectively sustains the high frequency details and provides superior results than the previous state-of-the-art methods. The performance of the two architectures has been evaluated on various image datasets, and compared with other state-of-the-art methods.

Słowa kluczowe

EN

super resolution; convolutional neural network; residual learning; deep learning; subpixel layer

• Wydawca: IOS Press
• Czasopismo: Fundamenta Informaticae
• Rocznik: 2019
• Tom: Vol. 168, nr 2-4
• Strony: 335--351
• Opis fizyczny: Bibliogr. 46 poz., fot., rys., tab., wykr.

Twórcy

autor

Sharma Shailza

• Department of Electronics and Communication Engineering, Thapar Institute of Engineering and Technology, Patiala, India

autor

Bawa Vivek Singh

• Department of Electronics and Communication Engineering, Thapar Institute of Engineering and Technology, Patiala, India

autor

Kumar Vinay

• Department of Electronics and Communication Engineering, Thapar Institute of Engineering and Technology, Patiala, India

Bibliografia

• [1] LeCun Y, Bengio Y, Hinton G. Deep learning. nature, 2015. 521(7553):436. doi:10.1038/nature14539.
• [2] Liu W, Wang Z, Liu X, Zeng N, Liu Y, Alsaadi FE. A survey of deep neural network architectures and their applications. Neurocomputing, 2017. 234:11-26. doi:10.1016/j.neucom.2016.12.038.
• [3] Schmidhuber J. Deep learning in neural networks: An overview. Neural networks, 2015. 61:85-117. doi:10.1016/j.neunet.2014.09.003.
• [4] Graves A, Mohamed Ar, Hinton G. Speech recognition with deep recurrent neural networks. In: Acoustics, speech and signal processing (icassp), 2013 ieee international conference on. IEEE, 2013 pp. 6645-6649. doi:10.1109/ICASSP.2013.6638947.
• [5] Yu J, Yang X, Gao F, Tao D. Deep multimodal distance metric learning using click constraints for image ranking. IEEE transactions on cybernetics, 2017. 47(12):4014-4024. doi:10.1109/TCYB.2016.2591583.
• [6] Yu J, Zhang B, Kuang Z, Lin D, Fan J. iPrivacy: image privacy protection by identifying sensitive objects via deep multi-task learning. IEEE Transactions on Information Forensics and Security, 2017. 12(5):1005-1016. doi:10.1109/TIFS.2016.2636090.
• [7] Collobert R, Weston J. A unified architecture for natural language processing: Deep neural networks with multitask learning. In: Proceedings of the 25th international conference on Machine learning. ACM, 2008 pp. 160-167. doi:10.1145/1390156.1390177.
• [8] Simonyan K, Zisserman A. Very deep convolutional networks for large-scale image recognition. arXiv preprint arXiv:1409.1556, 2014. URL https://arxiv.org/abs/1409.1556.
• [9] Hong C, Yu J, Wan J, Tao D, Wang M. Multimodal deep autoencoder for human pose recovery. IEEE Transactions on Image Processing, 2015. 24(12):5659-5670. doi:10.1109/TIP.2015.2487860.
• [10] Yu J, Sang J, Gao X. Machine learning and signal processing for big multimedia analysis. 2017. doi:10.1016/j.neucom.2017.01.091.
• [11] Dong C, Loy CC, Tang X. Accelerating the super-resolution convolutional neural network. 2016. pp. 391-407. doi:10.1007/978-3-319-46475-6_25.
• [12] Irani M, Peleg S. Improving resolution by image registration. CVGIP: Graphical models and image processing, 1991. 53(3):231-239. doi:10.1016/1049-9652(91)90045-L.
• [13] Freeman WT, Pasztor EC, Carmichael OT. Learning low-level vision. International journal of computer vision, 2000. 40(1):25-47. doi:10.1023/A:1026501619075.
• [14] Glasner D, Bagon S, Irani M. Super-resolution from a single image. 2009. pp. 349-356. doi:10.1109/ICCV.2009.5459271.
• [15] Isaac JS, Kulkarni R. Super resolution techniques for medical image processing. In: Technologies for Sustainable Development (ICTSD), 2015 International Conference on. IEEE, 2015 pp. 1-6. doi:10.1109/ICTSD.2015.7095900.
• [16] Plenge E, Poot DH, Bernsen M, Kotek G, Houston G, Wielopolski P, van der Weerd L, Niessen WJ, Meijering E. Super-resolution reconstruction in MRI: better images faster? 2012. 8314:83143V. doi:10.1117/12.911235.
• [17] Luo Y, Zhou L, Wang S, Wang Z. Video Satellite Imagery Super Resolution via Convolutional Neural Networks. IEEE Geoscience and Remote Sensing Letters, 2017. 14(12):2398-2402. doi:10.1109/LGRS.2017.2766204.
• [18] Park SC, Park MK, Kang MG. Super-resolution image reconstruction: a technical overview. IEEE signal processing magazine, 2003. 20(3):21-36. doi:10.1109/MSP.2003.1203207.
• [19] Borman S, Stevenson RL. Super-resolution from image sequences-a review. In: Circuits and Systems, 1998. Proceedings. 1998 Midwest Symposium on. IEEE, 1998 pp. 374-378. doi:10.1109/MWSCAS.1998.759509.
• [20] Sun J, Xu Z, Shum HY. Gradient profile prior and its applications in image super-resolution and enhancement. IEEE Transactions on Image Processing, 2011. 20(6):1529-1542. doi:10.1109/TIP.2010.2095871.
• [21] Takeda H, Milanfar P, Protter M, Elad M. Super-resolution without explicit subpixel motion estimation. IEEE Transactions on Image Processing, 2009. 18(9):1958-1975 doi:10.1109/TIP.2009.2023703.
• [22] Kao CC, Lai YT, Tseng CF. Improved edge-directed super resolution. International Journal of Computers and Applications, 2015. 37(3-4):160-167. doi:10.1080/1206212X.2016.1188558.
• [23] Fattal R. Image upsampling via imposed edge statistics. In: ACM transactions on graphics (TOG), volume 26. ACM, 2007 p. 95. doi:10.1145/1239451.1239546.
• [24] Dai S, Han M, Xu W, Wu Y, Gong Y, Katsaggelos AK. Softcuts: a soft edge smoothness prior for color image super-resolution. IEEE Transactions on Image Processing, 2009. 18(5):969-981. doi:10.1109/TIP.2009.2012908.
• [25] Fan W, Yeung DY. Image hallucination using neighbor embedding over visual primitive manifolds. 2007. pp. 1-7. doi:10.1109/CVPR.2007.383001.
• [26] Li X, Orchard MT. New edge-directed interpolation. IEEE transactions on image processing, 2001. 10(10):1521-1527. doi:10.1109/83.951537.
• [27] Lin WT, Lai SH. Single image super-resolution based on local self-similarity. In: Pattern Recognition (ACPR), 2013 2nd IAPR Asian Conference on. IEEE, 2013 pp. 191-195. doi:10.1109/ACPR.2013.107.
• [28] Chang H, Yeung DY, Xiong Y. Super-resolution through neighbor embedding. 2004. 1:I-I. doi:10.1109/CVPR.2004.1315043.
• [29] Yu WS, Chen Sq. An Improved Neighbor Embedding Method to Super-resolution Reconstruction of a Single Image. Procedia Engineering, 2011. 15:2418-2422. doi:10.1016/j.proeng.2011.08.454.
• [30] Freeman WT, Jones TR, Pasztor EC. Example-based super-resolution. IEEE Computer graphics and Applications, 2002. 22(2):56-65. doi:10.1109/38.988747.
• [31] Yang J, Wright J, Huang TS, Ma Y. Image super-resolution via sparse representation. IEEE transactions on image processing, 2010. 19(11):2861-2873. doi:10.1109/TIP.2010.2050625.
• [32] Aharon M, Elad M, Bruckstein A, et al. K-SVD: An algorithm for designing overcomplete dictionaries for sparse representation. IEEE Transactions on signal processing, 2006. 54(11):4311. doi:10.1109/TSP.2006.881199.
• [33] Cui Z, Chang H, Shan S, Zhong B, Chen X. Deep network cascade for image super-resolution. In: European Conference on Computer Vision. Springer, 2014 pp. 49-64. doi:10.1007/978-3-319-10602-1_4.
• [34] Dong C, Loy CC, He K, Tang X. Image super-resolution using deep convolutional networks. IEEE transactions on pattern analysis and machine intelligence, 2016. 38(2):295-307. doi:10.1109/TPAMI.2015.2439281.
• [35] Shi W, Caballero J, Huszár F, Totz J, Aitken AP, Bishop R, Rueckert D, Wang Z. Real-time single image and video super-resolution using an efficient sub-pixel convolutional neural network. 2016. pp. 1874-1883. doi:10.1109/CVPR.2016.207.
• [36] Du B, Xiong W, Wu J, Zhang L, Zhang L, Tao D. Stacked convolutional denoising auto-encoders for feature representation. IEEE transactions on cybernetics, 2017. 47(4):1017-1027. doi:10.1109/TCYB.2016.2536638.
• [37] Chen Y, Jiang H, Li C, Jia X, Ghamisi P. Deep feature extraction and classification of hyperspectral images based on convolutional neural networks. IEEE Transactions on Geoscience and Remote Sensing, 2016. 54(10):6232-6251. doi:10.1109/TGRS.2016.2584107.
• [38] He K, Zhang X, Ren S, Sun J. Spatial pyramid pooling in deep convolutional networks for visual recognition. 2014. pp. 346-361. doi:10.1007/978-3-319-10578-9_23.
• [39] Tang Y. Deep learning using linear support vector machines. arXiv preprint arXiv:1306.0239, 2013. URL https://arxiv.org/abs/1306.0239.
• [40] Agostinelli F, Hoffman M, Sadowski P, Baldi P. Learning activation functions to improve deep neural networks. arXiv preprint arXiv:1412.6830, 2014. URL https://arxiv.org/abs/1412.6830.
• [41] Kim J, Kwon Lee J, Mu Lee K. Accurate image super-resolution using very deep convolutional networks. 2016. pp. 1646-1654. doi:10.1109/CVPR.2016.182.
• [42] Everingham M, Van Gool L, Williams CK, Winn J, Zisserman A. The pascal visual object classes (voc) challenge. International journal of computer vision, 2010. 88(2):303-338. doi:10.1007/s11263-009-0275-4.
• [43] Bevilacqua M, Roumy A, Guillemot C, Alberi-Morel ML. Low-complexity single-image super-resolution based on nonnegative neighbor embedding. 2012. doi:10.5244/C.26.135.
• [44] Zeyde R, Elad M, Protter M. On single image scale-up using sparse-representations. 2010. pp. 711-730. doi:10.1007/978-3-642-27413-8_47.
• [45] Martin D, Fowlkes C, Tal D, Malik J. The berkeley segmentation dataset and benchmark. University of California, Berkeley, http://www.cs.berkeley.edu/projects/vision/grouping/segbench, 2007. doi:10.1109/ICCV.2001.937655.
• [46] Kingma DP, Ba J. Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980, 2014. URL https://arxiv.org/abs/1412.6980.

Uwagi

Opracowanie rekordu w ramach umowy 509/P-DUN/2018 ze środków MNiSW przeznaczonych na działalność upowszechniającą naukę (2019).