Impact of low resolution on image recognition with deep neural networks: An experimental study

• Autorzy: Koziarski M. , Cyganek B.

Identyfikatory

DOI

10.2478/amcs-2018-0056

• Języki publikacji: EN

Abstrakty

EN

Due to the advances made in recent years, methods based on deep neural networks have been able to achieve a state-of-the-art performance in various computer vision problems. In some tasks, such as image recognition, neural-based approaches have even been able to surpass human performance. However, the benchmarks on which neural networks achieve these impressive results usually consist of fairly high quality data. On the other hand, in practical applications we are often faced with images of low quality, affected by factors such as low resolution, presence of noise or a small dynamic range. It is unclear how resilient deep neural networks are to the presence of such factors. In this paper we experimentally evaluate the impact of low resolution on the classification accuracy of several notable neural architectures of recent years. Furthermore, we examine the possibility of improving neural networks’ performance in the task of low resolution image recognition by applying super-resolution prior to classification. The results of our experiments indicate that contemporary neural architectures remain significantly affected by low image resolution. By applying super-resolution prior to classification we were able to alleviate this issue to a large extent as long as the resolution of the images did not decrease too severely. However, in the case of very low resolution images the classification accuracy remained considerably affected.

Słowa kluczowe

EN

image recognition; deep neural network; convolutional neural network; low resolution; super resolution

PL

rozpoznawanie obrazu; sieć neuronowa głęboka; sieć neuronowa konwolucyjna; niska rozdzielczość; nadrozdzielczość

• Wydawca: Oficyna Wydawnicza Uniwersytetu Zielonogórskiego
• Czasopismo: International Journal of Applied Mathematics and Computer Science
• Rocznik: 2018
• Tom: Vol. 28, no. 4
• Strony: 735--744
• Opis fizyczny: Bibliogr. 39 poz., rys., tab., wykr.

Twórcy

autor

Koziarski M.

• Department of Electronics, AGH University of Science and Technology, al. Mickiewicza 30, 30-059 Kraków, Poland

autor

Cyganek B.

• Department of Electronics, AGH University of Science and Technology, al. Mickiewicza 30, 30-059 Kraków, Poland

Bibliografia

• [1] Abadi, M., Agarwal, A., Barham, P., Brevdo, E., Chen, Z., Citro, C., Corrado, G. S., Davis, A., Dean, J., Devin, M., Ghemawat, S., Goodfellow, I., Harp, A., Irving, G., Isard, M., Jia, Y., Jozefowicz, R., Kaiser, L., Kudlur, M., Levenberg, J., Mane, D., Monga, R., Moore, S., Murray, D., Olah, C., Schuster, M., Shlens, J., Steiner, B., Sutskever, I., Talwar, K., Tucker, P., Vanhoucke, V., Vasudevan, V., Viegas, F., Vinyals, O., Warden, P., Wattenberg, M., Wicke, M., Yu, Y. and Zheng, X. (2016). TensorFlow: Large-scale machine learning on heterogeneous distributed systems, arXiv: 1603.04467.
• [2] Bevilacqua, M., Roumy, A., Guillemot, C. and Alberi-Morel, M.L. (2012). Low-complexity single-image super-resolution based on nonnegative neighbor embedding, British Machine Vision Conference (BMVC), Guildford, UK.
• [3] Chang, H., Yeung, D.-Y. and Xiong, Y. (2004). Super-resolution through neighbor embedding, Proceedings of the 2004 IEEE Computer Society Conference on Computer Vision and Pattern Recognition, CVPR, Washington, DC, USA, Vol. 1, pp. I–I.
• [4] da Costa, G.B.P., Contato, W.A., Nazare, T.S., Neto, J.E. and Ponti, M. (2016). An empirical study on the effects of different types of noise in image classification tasks, arXiv: 1609.02781.
• [5] Dodge, S. and Karam, L. (2016). Understanding how image quality affects deep neural networks, 8th International Conference on Quality of Multimedia Experience (QoMEX), Lisbon, Portugal, pp. 1–6.
• [6] Dong, C., Loy, C.C., He, K. and Tang, X. (2014). Learning a deep convolutional network for image super-resolution, European Conference on Computer Vision, Zurich, Switzerland, pp. 184–199.
• [7] Dong, C., Loy, C.C. and Tang, X. (2016). Accelerating the super-resolution convolutional neural network, European Conference on Computer Vision, Amsterdam, The Netherlands, pp. 391–407.
• [8] Dutta, A., Veldhuis, R.N. and Spreeuwers, L.J. (2012). The impact of image quality on the performance of face recognition, 33rd WIC Symposium on Information Theory in the Benelux, Boekelo, The Netherlands, pp. 141–148.
• [9] Freeman, W.T., Jones, T.R. and Pasztor, E.C. (2002). Example-based super-resolution, IEEE Computer Graphics and Applications 22(2): 56–65.
• [10] Goodfellow, I., Bengio, Y. and Courville, A. (2016). Deep Learning, MIT Press, Cambridge, MA.
• [11] Goodfellow, I., Pouget-Abadie, J., Mirza, M., Xu, B., Warde-Farley, D., Ozair, S., Courville, A. and Bengio, Y. (2014). Generative adversarial nets, in Z. Ghahramani et al. (Eds.), Advances in Neural Information Processing Systems, Curran Associates, Inc., Red Hook, NY, pp. 2672–2680.
• [12] He, K., Zhang, X., Ren, S. and Sun, J. (2016). Deep residual learning for image recognition, Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Las Vegas, NV, USA, pp. 770–778.
• [13] Huang, J.-B., Singh, A. and Ahuja, N. (2015). Single image super-resolution from transformed self-exemplars, Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Boston, MA, USA, pp. 5197–5206.
• [14] Johnson, J., Karpathy, A. and Fei-Fei, L. (2016). DenseCap: Fully convolutional localization networks for dense captioning, Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Las Vegas, VN, USA, pp. 4565–4574.
• [15] Karahan, S., Yildirum, M.K., Kirtac, K., Rende, F.S., Butun, G. and Ekenel, H.K. (2016). How image degradations affect deep CNN-based face recognition, International Conference of the Biometrics Special Interest Group (BIOSIG), Darmstadt, Germany, pp. 1–5.
• [16] Kim, J., Kwon Lee, J. and Mu Lee, K. (2016). Accurate image super-resolution using very deep convolutional networks, Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Las Vegas, NV, USA, pp. 1646–1654.
• [17] Kingma, D. and Ba, J. (2014). Adam: A method for stochastic optimization, arXiv: 1412.6980.
• [18] Koziarski, M. and Cyganek, B. (2017). Image recognition with deep neural networks in presence of noise—dealing with and taking advantage of distortions, Integrated Computer-Aided Engineering 24(4): 337–349.
• [19] Krizhevsky, A., Sutskever, I. and Hinton, G.E. (2012). ImageNet classification with deep convolutional neural networks, Neural Information Processing Systems, Lake Tahoe, CA, USA, pp. 1097–1105.
• [20] Ledig, C., Theis, L., Huszár, F., Caballero, J., Cunningham, A., Acosta, A., Aitken, A., Tejani, A., Totz, J., Wang, Z. and Shi, W. (2016). Photo-realistic single image super-resolution using a generative adversarial network, arXiv: 1609.04802.
• [21] Lim, B., Son, S., Kim, H., Nah, S. and Lee, K.M. (2017). Enhanced deep residual networks for single image super-resolution, IEEE Conference on Computer Vision and Pattern Recognition (CVPR) Workshop, Honolulu, HI, USA, Vol. 1, p. 3.
• [22] Long, J., Shelhamer, E. and Darrell, T. (2015). Fully convolutional networks for semantic segmentation, Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Boston, MA, USA, pp. 3431–3440.
• [23] Mao, X., Shen, C. and Yang, Y.-B. (2016). Image restoration using very deep convolutional encoder-decoder networks with symmetric skip connections, Neural Information Processing Systems, Barcelona, Spain, pp. 2802–2810.
• [24] Martin, D., Fowlkes, C., Tal, D. and Malik, J. (2001). A database of human segmented natural images and its application to evaluating segmentation algorithms and measuring ecological statistics, 8th IEEE International Conference on Computer Vision, ICCV 2001, Vancouver, Canada, Vol. 2, pp. 416–423.
• [25] Peng, X., Hoffman, J., Stella, X.Y. and Saenko, K. (2016). Fine-to-coarse knowledge transfer for low-res image classification, IEEE International Conference on Image Processing (ICIP), Phoenix, AZ, USA, pp. 3683–3687.
• [26] Ronneberger, O., Fischer, P. and Brox, T. (2015). U-net: Convolutional networks for biomedical image segmentation, International Conference on Medical Image Computing and Computer-Assisted Intervention, Munich, Germany, pp. 234–241.
• [27] Russakovsky, O., Deng, J., Su, H., Krause, J., Satheesh, S., Ma, S., Huang, Z., Karpathy, A., Khosla, A., Bernstein, M., Berg, A.C. and Fei-Fei, L. (2015). ImageNet large scale visual recognition challenge, International Journal of Computer Vision 115(3): 211–252.
• [28] Sanchez, A., Moreno, A.B., Velez, D. and Vélez, J.F. (2016). Analyzing the influence of contrast in large-scale recognition of natural images, Integrated Computer-Aided Engineering 23(3): 221–235.
• [29] Schmidhuber, J. (2015). Deep learning in neural networks: An overview, Neural Networks 61(1): 85–117.
• [30] Schulter, S., Leistner, C. and Bischof, H. (2015). Fast and accurate image upscaling with super-resolution forests, Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Boston, MA, USA, pp. 3791–3799.
• [31] Shi, W., Caballero, J., Huszár, F., Totz, J., Aitken, A.P., Bishop, R., Rueckert, D. and Wang, Z. (2016). Real-time single image and video super-resolution using an efficient sub-pixel convolutional neural network, Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Las Vegas, NV, USA, pp. 1874–1883.
• [32] Simonyan, K. and Zisserman, A. (2014). Very deep convolutional networks for large-scale image recognition, arXiv: 1409.1556.
• [33] Sun, Y., Wang, X. and Tang, X. (2013). Deep convolutional network cascade for facial point detection, Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Portland, OR, USA, pp. 3476–3483.
• [34] Tai, Y., Yang, J. and Liu, X. (2017). Image super-resolution via deep recursive residual network, 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), Honolulu, HI, USA, pp. 2790–2798.
• [35] Timofte, R., De Smet, V. and Van Gool, L. (2014). A+: Adjusted anchored neighborhood regression for fast super-resolution, Asian Conference on Computer Vision, Singapore, Singapore, pp. 111–126.
• [36] Vasiljevic, I., Chakrabarti, A. and Shakhnarovich, G. (2016). Examining the impact of blur on recognition by convolutional networks, arXiv: 1611.05760.
• [37] Yang, J., Wright, J., Huang, T.S. and Ma, Y. (2010). Image super-resolution via sparse representation, IEEE Transactions on Image Processing 19(11): 2861–2873.
• [38] Zeyde, R., Elad, M. and Protter, M. (2010). On single image scale-up using sparse-representations, International Conference on Curves and Surfaces, Paris, France, pp. 711–730.
• [39] Zou, W.W. and Yuen, P.C. (2012). Very low resolution face recognition problem, IEEE Transactions on Image Processing 21(1): 327–340.

Uwagi

PL

Opracowanie rekordu w ramach umowy 509/P-DUN/2018 ze środków MNiSW przeznaczonych na działalność upowszechniającą naukę (2018).