Underwater image enhancement via efficient generative adversarial network

• Autorzy: Qian Xin , Ge Peng

Identyfikatory

DOI

10.37190/oa210402

• Języki publikacji: EN

Abstrakty

EN

Underwater image enhancement has been receiving much attention due to its significance in facilitating various marine explorations. Inspired by the generative adversarial network (GAN) and residual network (ResNet) in many vision tasks, we propose a simplified designed ResNet model based on GAN called efficient GAN (EGAN) for underwater image enhancement. In particular, for the generator of EGAN we design a new pair of convolutional kernel size for the residual block in the ResNet. Secondly, we abandon batch normalization (BN) after every convolution layer for faster training and less artifacts. Finally, a smooth loss function is introduced for halo-effect alleviation. Extensive qualitative and quantitative experiments show that our methods accomplish considerable improvements compared to the state-of-the-art methods.

Słowa kluczowe

EN

underwater; image dehazing; generative adversarial network; GAN

• Wydawca: Oficyna Wydawnicza Politechniki Wrocławskiej
• Czasopismo: Optica Applicata
• Rocznik: 2021
• Tom: Vol. 51, nr 4
• Strony: 483--497
• Opis fizyczny: Bibliogr. 32 poz., rys., tab.

Twórcy

autor

Qian Xin

• School of Electronics and Information Engineering, South China University of Technology, 510640 Guangzhou, China

autor

Ge Peng

• School of Electronics and Information Engineering, South China University of Technology, 510640 Guangzhou, China
• School of Physics and Opto-electronic, South China University of Technology, 510640 Guangzhou, China

Bibliografia

• [1] JAFFE J.S., Underwater optical imaging: the past, the present, and the prospects, IEEE Journal of Oceanic Engineering 40(3), 2015, pp. 683–700, DOI: 10.1109/JOE.2014.2350751.
• [2] WANG Y., SONG W., FORTINO G., QI L.Z., ZHANG W., LIOTTA A., An experimental-based review of image enhancement and image restoration methods for underwater imaging, IEEE Access 7, 2019, pp. 140233–140251, DOI: 10.1109/ACCESS.2019.2932130.
• [3] JAFFE J.S., Computer modeling and the design of optimal underwater imaging systems, IEEE Journal of Oceanic Engineering 15(2), 1990, pp. 101–111, DOI: 10.1109/48.50695.
• [4] AKKAYNAK D., TREIBITZ T., A revised underwater image formation model, [In] 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2018, pp. 6723–6732, DOI: 10.1109/CVPR.2018.00703.
• [5] HOU W., A simple underwater imaging model, Optics Letters 34(17), 2009, pp. 2688–2690, DOI: 10.1364/OL.34.002688.
• [6] GIBSON R., ATKINSON R., GORDON J. [Eds], A Review of Underwater Stereo-Image Measurement for Marine Biology and Ecology Applications, Oceanography and Marine Biology: An Annual Review, Vol. 47, 2016, pp. 257–292.
• [7] DREWS P. JR, DO NASCIMENTO E., MORAES F., BOTELHO S., CAMPOS M., Transmission estimation in underwater single images, [In] 2013 IEEE International Conferen ce on Computer Vision Workshops, 2013, pp. 825–830, DOI: 10.1109/ICCVW.2013.113.
• [8] GALDRAN A., PARDO D., PICÓN A., ALVAREZ-GILA A., Automatic Red-Channel underwater image restoration, Journal of Visual Communication and Image Representation 26, 2015, pp. 132–145, DOI: 10.1016/j.jvcir.2014.11.006.
• [9] BERMAN D., LEVY D., AVIDAN S., TREIBITZ T., Underwater single image color restoration using hazelines and a new quantitative dataset, IEEE Transactions on Pattern Analysis and Machine Intelligence 43(8), 2021, pp. 2822–2837, DOI: 10.1109/TPAMI.2020.2977624.
• [10] LU H., LI Y., ZHANG L., SERIKAWA S., Contrast enhancement for images in turbid water, Journal of the Optical Society of America A 32(5), 2015, pp. 886–893, DOI: 10.1364/JOSAA.32.000886.
• [11] HAN P., LIU F., YANG K., MA J., LI J., SHAO X., Active underwater descattering and image recovery, Applied Optics 56(23), 2017, pp. 6631–6638, DOI: 10.1364/AO.56.006631.
• [12] PUROHIT K., MANDAL S., RAJAGOPALAN A.N., Multilevel weighted enhancement for underwater image dehazing, Journal of the Optical Society of America A 36(6), 2019, pp. 1098–1108, DOI: 10.1364/JOSAA.36.001098.
• [13] ANCUTI C., ANCUTI C.O., HABER T., BEKAERT P., Enhancing underwater images and videos by fusion, [In] 2012 IEEE Conference on Computer Vision and Pattern Recognition, 2012, pp. 81–88, DOI: 10.1109/CVPR.2012.6247661.
• [14] PENG Y.T., COSMAN P.C., Underwater image restoration based on image blurriness and light absorption, IEEE Transactions on Image Processing 26(4), 2017, pp. 1579–1594, DOI: 10.1109/TIP.2017.2663846.
• [15] SONG W., WANG Y., HUANG D., TJONDRONEGORO D., A rapid scene depth estimation model based on underwater light attenuation prior for underwater image restoration, [In] Hong R., Cheng W.H., Yamasaki T., Wang M., Ngo C.W. [Eds], Advances in Multimedia Information Processing – PCM 2018. PCM 2018. Lecture Notes in Computer Science, Vol. 11164, Springer, Cham, pp. 678–688, DOI: 10.1007/978-3-030-00776-8_62.
• [16] ANWAR S., LI C., PORIKLI F., Deep underwater image enhancement, arXiv:1807.03528 [cs.CV], 2018.
• [17] LI C., GUO C., REN W., CONG R., HOU J., KWONG S., TAO D., An underwater image enhancement benchmark dataset and beyond, IEEE Transactions on Image Processing 29, 2020, pp. 4376–4389, DOI: 10.1109/TIP.2019.2955241.
• [18] LEDIG C., THEIS L., HUSZÁR F., CABALLERO J., CUNNINGHAM A., ACOSTA A., AITKEN A., TEJANI A., TOTZ J., WANG Z., SHI W., Photo-realistic single image super-resolution using a generative adversarial network, [In] 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2017, pp. 105–114, DOI: 10.1109/CVPR.2017.19.
• [19] DU Y., LI X., Recursive image dehazing via perceptually optimized generative adversarial network (POGAN), [In] 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops (CVPRW), 2019, pp. 1824–1832, DOI: 10.1109/CVPRW.2019.00233.
• [20] ISOLA P., ZHU J.Y., ZHOU T., EFROS A.A., Image-to-image translation with conditional adversarial networks, arXiv:1611.07004 [cs.CV], 2017.
• [21] LIU X., GAO Z., CHEN B.M., MLFcGAN: multilevel feature fusion-based conditional GAN for underwater image color correction, IEEE Geoscience and Remote Sensing Letters 17(9), 2020, pp. 1488–1492, DOI: 10.1109/LGRS.2019.2950056.
• [22] LI J., SKINNER K.A., EUSTICE R.M., JOHNSON-ROBERSON M., WaterGAN: Unsupervised generative network to enable real-time color correction of monocular underwater images, IEEE Robotics and Automation Letters 3(1), 2018, pp. 387–394, DOI: 10.1109/LRA.2017.2730363.
• [23] FABBRI C., ISLAM M.J., SATTAR J., Enhancing underwater imagery using generative adversarial networks, [In] 2018 IEEE International Conferenceon Robotics and Automation (ICRA), 2018, pp. 7159–7165, DOI: 10.1109/ICRA.2018.8460552.
• [24] HE K., SUN J., Convolutional neural networks at constrained time cost, [In] 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2015, pp. 5353–5360, DOI: 10.1109/CVPR.2015.7299173.
• [25] HE K., ZHANG X., REN S., SUN J., Deep residual learningfor image recognition, [In] 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2016, pp. 770–778, DOI: 10.1109/CVPR.2016.90.
• [26] LIM B., SON S., KIM H., NAH S., LEE K.M., Enhanced deep residual networks for single image super-resolution, [In] 2017 IEEE Conference on Computer Vision and Pattern Recognition Workshops (CVPRW), 2017, pp. 1132–1140, DOI: 10.1109/CVPRW.2017.151.
• [27] RONNEBERGER O., FISCHER P., BROX T., U-Net: convolutional networks for biomedical image segmentation, [In] Navab N., Hornegger J., Wells W., Frangi A. [Eds], Medical Image Computing and Computer-Assisted Intervention – MICCAI 2015. MICCAI 2015. Lecture Notes in Computer Science, Vol. 9351, Springer, Cham, 2015, DOI: 10.1007/978-3-319-24574-4_28.
• [28] RADFORD A., METZ L., CHINTALA S., Unsupervised representation learning with deep convolutional generative adversarial networks, arXiv:1511.06434 [cs.LG], 2015.
• [29] SIMONYAN K., ZISSERMAN A., Very deep convolutional networks for large-scale image recognition, arXiv:1409.1556 [cs.CV], 2014.
• [30] LI C., ANWAR S., PORIKLI F., Underwater scene prior inspired deep underwater image and video enhancement, Pattern Recognition 98, 2020, article 107038, DOI: 10.1016/j.patcog.2019.107038.
• [31] ZHANG K., SUN M., HAN T.X., YUAN X., GUO L., LIU T., Residual networks of residual networks: multilevel residual networks, IEEE Transactions on Circuits and Systems for Video Technology 28(6), 2018, pp. 1303–1314, DOI: 10.1109/TCSVT.2017.2654543.
• [32] WANG X., YU K., WU S., GU J., LIU Y., DONG C., QIAO Y., LOY C.C., ESRGAN: enhanced super-resolution generative adversarial networks, [In] Leal-Taixé L., Roth S. [Eds], Computer Vision – ECCV 2018 Workshops. ECCV 2018. Lecture Notes in Computer Science, Vol. 11133, Springer, Cham, 2018, DOI: 10.1007/978-3-030-11021-5_5.