Performance analysis of a dual stage deep rain streak removal convolution neural network module with a modified deep residual dense network

• Autorzy: Jayaraman Thiyagarajan , Chinnusamy Shankar

Identyfikatory

DOI

10.34768/amcs-2022-0009

• Języki publikacji: EN

Abstrakty

EN

The visual appearance of outdoor captured images is affected by various weather conditions, such as rain patterns, haze, fog and snow. The rain pattern creates more degradation in the visual quality of the image due to its physical structure compared with other weather conditions. Also, the rain pattern affects both foreground and background image information. The removal of rain patterns from a single image is a critical process, and more attention is given to remove the structural rain pattern from real-time rain images. In this paper, we analyze the single image deraining problem and present a solution using the dual stage deep rain streak removal convolutional neural network. The proposed single image deraining framework primarily consists of three main blocks: a derain streaks removal CNN (derain SRCNN), a modified residual dense block (MRDB), and a six-stage scale feature aggregation module (3SFAM). The ablation study is conducted to evaluate the performance of various modules available in the proposed deraining network. The robustness of the proposed deraining network is evaluated over the popular synthetic and real-time data sets using four performance metrics such as the peak signal-to-noise ratio (PSNR), the feature similarity index (FSIM), the structural similarity index measure (SSIM), and the universal image quality index (UIQI). The experimental results show that the proposed framework outperforms both synthetic and real-time images compared with other state-of-the-art single image deraining approaches. In addition, the proposed network takes less running and training time.

Słowa kluczowe

EN

single image deraining; deep learning; modified residual dense network; PyTorch

PL

obraz pojedynczy; uczenie głębokie; sieć gęsta; PyTorch

• Wydawca: Oficyna Wydawnicza Uniwersytetu Zielonogórskiego
• Czasopismo: International Journal of Applied Mathematics and Computer Science
• Rocznik: 2022
• Tom: Vol. 32, no. 1
• Strony: 111--123
• Opis fizyczny: Bibliogr. 40 poz., rys., tab.

Twórcy

autor

Jayaraman Thiyagarajan

• Department of Mechatronics Engineering Sona College of Technology Junction Main Road, Salem, Tamil Nadu, India

autor

Chinnusamy Shankar

• Department of Electrical and Electronics Engineering KSR College of Engineering KSR Kalvi Nagar, Namakkal, Tamil Nadu, Tiruchengode, India

Bibliografia

• [1] Barnum, P.C., Narasimhan, S. and Kanade, T. (2009). Analysis of rain and snow in frequency space, International Journal of Computer Vision 86(2): 256, DOI: 10.1007/s11263-008-0200-2.
• [2] Chen, Y. and Wang, W. (2020). Recursive modified dense network for single-image deraining, Journal of Electronic Imaging 29(3): 10–12.
• [3] Ding, X., Chen, L., Zheng, X., Huang, Y. and Zeng, D. (2016). Single image rain and snow removal via guided L0 smoothing filter, Multimedia Tools and Applications 75(5): 2697–2712, DOI: 10.1007/s11042-015-2657-7.
• [4] Fu, X., Huang, J., Zeng, D., Huang, Y., Ding, X. and Paisley, J. (2017). Removing rain from single images via a deep detail network, Proceedings of the 30th IEEE Conference on Computer Vision and Pattern Recognition, CVPR 2017, Honolulu, USA, pp. 1715–1723.
• [5] Garg, K. and Nayar, S.K. (2004). Detection and removal of rain from videos, Proceedings of the 2004 IEEE Computer Society Conference on Computer Vision and Pattern Recognition, CVPR 2004, Washington, USA, Vol. 1, pp. I–I.
• [6] Gu, S., Meng, D., Zuo, W. and Zhang, L. (2017). Joint convolutional analysis and synthesis for sparse representation for single image layer separation, 2017 IEEE International Conference on Computer Vision (ICCV), Venice, Italy, pp. 1717–1725.
• [7] He, K., Zhang, X., Ren, S. and Sun, J. (2016). Deep residual learning for image recognition, Proceedings of the IEEE Computer Society Conference on Computer Vision and Pattern Recognition, Las Vegas, USA, pp. 770–778.
• [8] Huang, G., Liu, Z., Van Der Maaten, L. and Weinberger, K.Q. (2017). Densely connected convolutional networks, Proceedings of the 30th IEEE Conference on Computer Vision and Pattern Recognition, CVPR 2017, Honolulu, USA, pp. 2261–2269.
• [9] Jayaraman, T. and Chinnusamy, G.S. (2020a). Analysis of deep rain streaks removal convolutional neural network-based post-processing techniques in HEVC encoder, Journal of Circuits, Systems and Computers 30(2): 1–21, Paper no. 2150020.
• [10] Jayaraman, T. and Chinnusamy, G.S. (2020b). Investigation of filtering of rain streaks affected video sequences under various quantisation parameter in HEVC encoder using an enhanced V-BM4D algorithm, IET Image Processing 14(2): 337–347.
• [11] Kang, L., Lin, C. and Fu, Y. (2012). Automatic single-image-based rain streaks removal via image decomposition, IEEE Transactions on Image Processing 21(4): 1742–1755.
• [12] Kim, J., Lee, C., Sim, J. and Kim, C. (2013). Single-image deraining using an adaptive nonlocal means filter, IEEE International Conference on Image Processing, Melbourne, Australia, pp. 914–917.
• [13] Kou, F., Chen, W., Wen, C. and Li, Z. (2015). Gradient domain guided image filtering, IEEE Transactions on Image Processing 24(11): 4528–4539.
• [14] Kowal, M., Skobel, M., Gramacki, A. and Korbicz, J. (2021). Breast cancer nuclei segmentation and classification based on a deep learning approach, International Journal of Applied Mathematics and Computer Science 31(1): 85–106, DOI: 10.34768/amcs-2021-0007.
• [15] Koziarski, M. and Cyganek, B. (2018). Impact of low resolution on image recognition with deep neural networks: An experimental study, International Journal of Applied Mathematics and Computer Science 28(4): 735–744, DOI: 10.2478/amcs-2018-0056.
• [16] Li, P., Tian, J., Tang, Y., Wang, G. and Wu, C. (2020). Model-based deep network for single image deraining, IEEE Access 8(1): 14036–14047.
• [17] Li, X., Wu, J., Lin, Z., Liu, H. and Zha, H. (2018). Recurrent squeeze-and-excitation context aggregation net for single image deraining, Proceedings of the European Conference on Computer Vista (ECCV), Amsterdam, The Netherlands, pp. 254–269.
• [18] Li, Y., Tan, R.T., Guo, X., Lu, J. and Brown, M.S. (2016). Rain streak removal using layer priors, Proceedings of the IEEE Computer Society Conference on Computer Vision and Pattern Recognition, Las Vegas, USA, pp. 2736–2744.
• [19] Lin, C.Y., Tao, Z., Xu, A.S., Kang, L.W. and Akhyar, F. (2020). Sequential dual attention network for rain streak removal in a single image, IEEE Transactions on Image Processing 29(1): 9250–9265.
• [20] Papiez, A., Badie, C. and Polanska, J. (2019). Machine learning techniques combined with dose profiles indicate radiation response biomarkers, International Journal of Applied Mathematics and Computer Science 29(1): 169–178, DOI: 10.2478/amcs-2019-0013.
• [21] Paszke, A., Gross, S., Massa, F., Lerer, A., Bradbury, J., Chanan, G., Killeen, T., Lin, Z., Gimelshein, N., Antiga, L., Desmaison, A., Köpf, A., Yang, E., DeVito, Z., Raison, M., Tejani, A., Chilamkurthy, S., Steiner, B., Fang, L., Bai, J. and Chintala, S. (2019). PyTorch: An imperative style, high-performance deep learning library, arXiv 1912.01703 (NeurIPS).
• [22] Ren, D., Shang, W., Zhu, P., Hu, Q., Meng, D. and Zuo, W. (2020). Single image deraining using bilateral recurrent network, IEEE Transactions on Image Processing 29(1): 6852–6863.
• [23] Ren, D., Zuo, W., Hu, Q., Zhu, P. and Meng, D. (2019). Progressive image deraining networks: A better and simpler baseline, IEEE Conference on Computer Vision and Pattern Recognition, Long Beach, USA pp. 3937–3946.
• [24] Sharma, P.K., Basavaraju, S. and Sur, A. (2021). High-resolution image de-raining using conditional GAN with sub-pixel upscaling, Multimedia Tools and Applications 80(1): 1075–1094, DOI: 10.1007/s11042-020-09642-7.
• [25] Sheikh, H.R. and Bovik, A.C. (2006). Image information and visual quality, IEEE Transactions on Image Processing 15(2): 430–444.
• [26] Thiyagarajan, J. and Gowri Shankar, C. (2020). Quality improvement and performance analysis of high efficiency video coding under high quantization parameters and rain streaks, Signal, Image and Video Processing 14(2): 387–395, DOI: 10.1007/s11760-019-01565-7.
• [27] Wang, C., Zhang, M., Su, Z., Yao, G., Wang, Y., Sun, X. and Luo, X. (2019). From coarse to fine: A stage-wise deraining net, IEEE Access 7(1): 84420–84428.
• [28] Wang, M., Chen, L., Liang, Y., Hao, Y., He, H. and Li, C. (2020a). Single image rain removal with reusing original input squeeze-and-excitation network, IET Image Processing 14(8): 1467–1474.
• [29] Wang, M., Chen, L., Liang, Y., Huang, H. and Cai, R. (2020b). Deep learning method for rain streaks removal from single image, Journal of Engineering 2020(13): 555–560.
• [30] Wang, Y., Gong, D., Yang, J., Shi, Q., van den Hengel, A., Xie, D. and Zeng, B. (2020c). Deep Single Image Deraining via Modeling Haze-like Effect, IEEE Transactions on Multimedia 23(1): 1–1.
• [31] Wang, Y., Zhang, D. and Dai, G. (2020d). Classification of high resolution satellite images using improved U-Net, International Journal of Applied Mathematics and Computer Science 30(3): 399–413, DOI: 10.34768/amcs-2020-0030.
• [32] Wang, Z. and Bovik, A.C. (2002). A universal image quality index, IEEE Signal Processing Letters 9(3): 81–84.
• [33] Wang, Z., Bovik, A.C., Sheikh, H.R. and Simoncelli, E.P. (2004). Image quality assessment: From error visibility to structural similarity, IEEE Transactions on Image Processing 13(4): 600–612.
• [34] Wei, W., Meng, D., Zhao, Q., Xu, Z. and Wu, Y. (2018). Semi-supervised transfer learning for image rain removal, 32nd IEEE/CVF Conference on Computer Vision and Pattern Recognition, CVPR 2018, Long Beach, USA, pp. 3877–3886.
• [35] Wu, S. and Zhou, J. (2020). MSFA-Net: A network for single image deraining, Journal of Physics: Conference Series 1584(1), Paper no. 012047.
• [36] Yang, W., Tan, R.T., Feng, J., Guo, Z., Yan, S. and Liu, J. (2020). Joint rain detection and removal from a single image with contextualized deep networks, IEEE Transactions on Pattern Analysis and Machine Intelligence 42(6): 1377–1393.
• [37] Yang, W., Tan, R.T., Feng, J., Liu, J., Guo, Z. and Yan, S. (2017). Deep joint rain detection and removal from a single image, 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), Honolulu, USA, pp. 1685–1694.
• [38] Zhang, H. and Patel, V.M. (2018). Density-aware single image de-raining using a multi-stream dense network, Proceedings of the IEEE Computer Society Conference on Computer Vision and Pattern Recognition, San Juan, USA, pp. 695–704.
• [39] Zhang, L., Zhang, L., Mou, X. and Zhang, D. (2011). FSIM: A feature similarity index for image quality assessment, IEEE Transactions on Image Processing 20(8): 2378–2386.
• [40] Zhang, Y., Tian, Y., Kong, Y., Zhong, B. and Fu, Y. (2018). Residual dense network for image super-resolution, Proceedings of the IEEE Computer Society Conference on Computer Vision and Pattern Recognition, Salt Lake City, USA, pp. 2472–2481.