Denseformer for single image deraining

• Autorzy: Wang Tianming , Wang Kaige , Li Qing

Identyfikatory

DOI

10.34768/amcs-2023-0046

• Języki publikacji: EN

Abstrakty

EN

Image is one of the most important forms of information expression in multimedia. It is the key factor to determine the visual effect of multimedia software. As an image restoration task, image deraining can effectively restore the original information of the image, which is conducive to the downstream task. In recent years, with the development of deep learning technology, CNN and Transformer structures have shone brightly in computer vision. In this paper, we summarize the key to success of these structures in the past, and on this basis, we introduce the concept of a layer aggregation mechanism to describe how to reuse the information of the previous layer to better extract the features of the current layer. Based on this layer aggregation mechanism, we build the rain removal network called DenseformerNet. Our network strengthens feature promotion and encourages feature reuse, allowing better information and gradient flow. Through a large number of experiments, we prove that our model is efficient and effective, and expect to bring some illumination to the future rain removal network.

Słowa kluczowe

EN

artificial intelligence; convolutional neural network; image deraining

PL

sztuczna inteligencja; sieć neuronowa konwolucyjna; obraz pojedynczy

• Wydawca: Oficyna Wydawnicza Uniwersytetu Zielonogórskiego
• Czasopismo: International Journal of Applied Mathematics and Computer Science
• Rocznik: 2023
• Tom: Vol. 33, no. 4
• Strony: 651--661
• Opis fizyczny: Bibliogr. 37 poz., rys., tab., wykr.

Twórcy

autor

Wang Tianming

• Intelligent Manufacturing Electronics Research Center, Institute of Microelectronics of the Chinese Academy of Sciences, 3 Beitu Cheng West Road, Chaoyang District, Beijing 100029, China
• School of Integrated Circuits, University of the Chinese Academy of Sciences, 3 Beitu Cheng West Road, Chaoyang District, Beijing 100029, China

autor

Wang Kaige

• Intelligent Manufacturing Electronics Research Center, Institute of Microelectronics of the Chinese Academy of Sciences, 3 Beitu Cheng West Road, Chaoyang District, Beijing 100029, China
• School of Integrated Circuits, University of the Chinese Academy of Sciences, 3 Beitu Cheng West Road, Chaoyang District, Beijing 100029, China

autor

Li Qing

• Intelligent Manufacturing Electronics Research Center, Institute of Microelectronics of the Chinese Academy of Sciences, 3 Beitu Cheng West Road, Chaoyang District, Beijing 100029, China
• School of Integrated Circuits, University of the Chinese Academy of Sciences, 3 Beitu Cheng West Road, Chaoyang District, Beijing 100029, China

Bibliografia

• [1] Belongie, T.L.D.G.H.H. (2017). Feature pyramid networks for object detection, Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Hawaii, USA, pp. 2117-2125.
• [2] Chen, C., Seff, A., Kornhauser, A. and Xiao, J. (2015). Deepdriving: Learning affordance for direct perception in autonomous driving, Proceedings of the IEEE International Conference on Computer Vision, Santiago, Chile, pp. 2722-2730.
• [3] Dosovitskiy, A., Beyer, L., Kolesnikov, A., Weissenborn, D., Zhai, X., Unterthiner, T., Dehghani, M., Minderer, M., Heigold, G., Gelly, S., Uszkoreit, J. and Houlsby, N. (2021). An image is worth 16x16 words: Transformers for image recognition at scale, International Conference on Learning Representations, pp. 1-21, (online).
• [4] Fan, H., Xiong, B., Mangalam, K., Li, Y., Yan, Z., Malik, J. and Feichtenhofer, C. (2021). Multiscale vision transformers, Proceedings of the IEEE/CVF International Conference on Computer Vision, pp. 6824-6835, (online).
• [5] Fu, X., Huang, J., Ding, X., Liao, Y. and Paisley, J. (2017a). Clearing the skies: A deep network architecture for single-image rain removal, IEEE Transactions on Image Processing 26(6): 2944-2956.
• [6] Fu, X., Huang, J., Zeng, D., Huang, Y., Ding, X. and Paisley, J. (2017b). Removing rain from single images via a deep detail network, Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Honolulu, USA, pp. 3855-3863.
• [7] Graves, A. (2012). Supervised Sequence Labelling with Recurrent Neural Networks, Springer, Berlin, pp. 37-45.
• [8] He, K., Sun, J. and Tang, X. (2010). Guided image filtering, European Conference on Computer Vision, Heraklion, Greece, pp. 1-14.
• [9] He, K., Zhang, X., Ren, S. and Sun, J. (2016). Deep residual learning for image recognition, Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Las Vegas, USA, pp. 770-778.
• [10] Kang, L.-W., Lin, C.-W. and Fu, Y.-H. (2011). Automatic single-image-based rain streaks removal via image decomposition, IEEE Transactions on Image Processing 21(4): 1742-1755.
• [11] Karlupia, N., Mahajan, P., Abrol, P. and Lehana, P.K. (2023). A genetic algorithm based optimized convolutional neural network for face recognition, International Journal of Applied Mathematics and Computer Science 33(1): 21-31, DOI: 10.34768/amcs-2023-0002.
• [12] Kian Ara, R., Matiolanski, A., Grega, M., Dziech, A. and Baran, R. (2023). Efficient face detection based crowd density estimation using convolutional neural networks and an improved sliding window strategy, International Journal of Applied Mathematics and Computer Science 33(1): 7-20, DOI: 10.34768/amcs-2023-0001.
• [13] Li, W., Zhang, Q., Zhang, J., Huang, Z., Tian, X. and Tao, D. (2022). Toward real-world single image deraining: A new benchmark and beyond, arXiv: 2206.05514.
• [14] Li, Y., Tan, R.T., Guo, X., Lu, J. and Brown, M.S. (2016). Rain streak removal using layer priors, Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Las Vegas, USA, pp. 2736-2744.
• [15] Liang, J., Cao, J., Sun, G., Zhang, K., Van Gool, L. and Timofte, R. (2021). SwinIR: Image restoration using swin transformer, Proceedings of the IEEE/CVF International Conference on Computer Vision, Montreal, Canada, pp. 1833-1844.
• [16] Luo, Y., Xu, Y. and Ji, H. (2015). Removing rain from a single image via discriminative sparse coding, Proceedings of the IEEE International Conference on Computer Vision, Boston, USA, pp. 3397-3405.
• [17] Mittal, A., Moorthy, A.K. and Bovik, A.C. (2012). No-reference image quality assessment in the spatial domain, IEEE Transactions on Image Processing 21(12): 4695-4708.
• [18] Mou, C., Wang, Q. and Zhang, J. (2022). Deep generalized unfolding networks for image restoration, Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, New Orleans, USA, pp. 17399-17410.
• [19] Nowak, T., Nowicki, M.R. and Skrzypczyński, P. (2022). Vision-based positioning of electric buses for assisted docking to charging stations, International Journal of Applied Mathematics and Computer Science 32(4): 583-599, DOI: 10.34768/amcs-2022-0041.
• [20] Ren, D., Zuo, W., Hu, Q., Zhu, P. and Meng, D. (2019). Progressive image deraining networks: A better and simpler baseline, Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, Long Beach, USA, pp. 3937-3946.
• [21] Ronneberger, O., Fischer, P. and Brox, T. (2015). U-net: Convolutional networks for biomedical image segmentation, International Conference on Medical Image Computing and Computer-Assisted Intervention, Munich, Germany, pp. 234-241.
• [22] Shi, X., Chen, Z., Wang, H., Yeung, D.-Y., Wong, W.-K. and Woo, W.-c. (2015). Convolutional LSTM network: A machine learning approach for precipitation nowcasting, Proceedings of the 2015 Neural Information Processing Systems (NIPS) Conference, Montreal, Canada, pp. 802-810.
• [23] Tolstikhin, I.O., Houlsby, N., Kolesnikov, A., Beyer, L., Zhai, X., Unterthiner, T., Yung, J., Steiner, A., Keysers, D., Uszkoreit, J., Lucic, M. and Dosovitskiy, A. (2021). MLP-Mixer: An ALL-MLP, Proceedings of the 2021 Neural Information Processing Systems (NIPS) Conference, NIPS2021, pp. 24261-24272, (online).
• [24] Vaswani, A., Shazeer, N., Parmar, N., Uszkoreit, J., Jones, L., Gomez, A.N., Kaiser, Ł. and Polosukhin, I. (2017). Attention is all you need, Proceedings of the 2017 Neural Information Processing Systems (NIPS) Conference, Long Beach, USA, pp. 5998-6008.
• [25] Wang, C., Wu, Y., Su, Z. and Chen, J. (2020a). Joint self-attention and scale-aggregation for self-calibrated deraining network, Proceedings of the 28th ACM International Conference on Multimedia, Seattle, USA, pp. 2517-2525.
• [26] Wang, G., Zhao, Y., Tang, C., Luo, C. and Zeng, W. (2022). When shift operation meets vision transformer: An extremely simple alternative to attention mechanism, Proceedings of the AAAI Conference on Artificial Intelligence, pp. 2423-2430, (online).
• [27] Wang, H., Xie, Q., Zhao, Q. and Meng, D. (2020b). A model-driven deep neural network for single image rain removal, Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, Seattle, USA, pp. 3103-3112.
• [28] Wang, Y., Liu, S., Chen, C. and Zeng, B. (2017). A hierarchical approach for rain or snow removing in a single color image, IEEE Transactions on Image Processing 26(8): 3936-3950.
• [29] Wang, Z., Cun, X., Bao, J. and Liu, J. (2021). Uformer: A general U-shaped transformer for image restoration, arXiv: 2106.03106.
• [30] Xiao, J., Fu, X., Liu, A., Wu, F. and Zha, Z.-J. (2022). Image de-raining transformer, IEEE Transactions on Pattern Analysis and Machine Intelligence 45(11): 12978-12995.
• [31] Yang,W., Tan, R.T., Feng, J., Liu, J., Guo, Z. and Yan, S. (2017). Deep joint rain detection and removal from a single image, Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Honolulu, USA, pp. 1357-1366.
• [32] Yu, W., Luo, M., Zhou, P., Si, C., Zhou, Y., Wang, X., Feng, J. and Yan, S. (2022). Metaformer is actually what you need for vision, Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, New Orleans, USA, pp. 10819-10829.
• [33] Yu, X., Zhang, G., Tan, F., Li, F. and Xie,W. (2023). Progressive hybrid-modulated network for single image deraining, Mathematics 11(3): 691.
• [34] Yuan, L., Hou, Q., Jiang, Z., Feng, J. and Yan, S. (2021). Volo: Vision outlooker for visual recognition, arXiv: 2106.13112.
• [35] Zamir, S.W., Arora, A., Khan, S., Hayat, M., Khan, F.S. and Yang, M.-H. (2022). Restormer: Efficient transformer for high-resolution image restoration, Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, New Orleans, USA, pp. 5728-5739.
• [36] Zamir, S.W., Arora, A., Khan, S., Hayat, M., Khan, F.S., Yang, M.-H. and Shao, L. (2021). Multi-stage progressive image restoration, Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, New Orleans, USA, pp. 14821-14831, (online).
• [37] Zheng, X., Liao, Y., Guo, W., Fu, X. and Ding, X. (2013). Single-image-based rain and snow removal using multi-guided filter, International Conference on Neural Information Processing, Daegu, Korea, pp. 258-265.