Application of a deep-learning neural network for image reconstruction from a single-pixel infrared camera

• Autorzy: Urbaś Sebastian , Więcek Bogusław

Identyfikatory

DOI

10.24425/opelre.2024.148877

• Języki publikacji: EN

Abstrakty

EN

The article presents the simulation results of a single-pixel infrared camera image reconstruction obtained by using a convolutional neural network (CNN). Simulations were carried out for infrared images with a resolution of 80 × 80 pixels, generated by a low-cost, low-resolution thermal imaging camera. The study compares the reconstruction results using the CNN and the ℓ₁ reconstruction algorithm. The results obtained using the neural network confirm a better quality of the reconstructed images with the same compression rate expressed by the peak signal-to-noise ratio (PSNR) and structural similarity index measure (SSIM).

Słowa kluczowe

EN

single-pixel imaging; compressive sensing; thermal imaging; convolutional neural network; dataset augmentation

• Wydawca: Stowarzyszenie Elektryków Polskich ; Polska Akademia Nauk ; Wojskowa Akademia Techniczna im. Jarosława Dąbrowskiego
• Czasopismo: Opto-Electronics Review
• Rocznik: 2024
• Tom: Vol. 32, No. 1
• Strony: art. no. e148877
• Opis fizyczny: Bibliogr. 40 poz., rys., tab.

Twórcy

autor

Urbaś Sebastian

• Institute of Electronics, Lodz University of Technology, Al. Politechniki 6, 90-924 Lodz, Poland

• https://orcid.org/0000-0002-7699-5425

autor

Więcek Bogusław

• Institute of Electronics, Lodz University of Technology, Al. Politechniki 6, 90-924 Lodz, Poland

• https://orcid.org/0000-0002-5003-1687

Bibliografia

• [1] Trofimov, A. A., Watkins, T. R., Muth, T. R., Cola, G. M. & Wang, H. Infrared thermometry in high temperature materials processing: influence of liquid water and steam. Quant. Infrared Thermogr. J. 20, 123-141 (2023). https://doi.org/10.1080/17686733.2022.2043617.
• [2] Kim, C., Park, G., Jang, H. & Kim, E.-J. Automated classification of thermal defects in the building envelope using thermal and visible images. Quant. Infrared Thermogr. J. 20, 106-122 (2023). https://doi.org/10.1080/17686733.2022.2033531.
• [3] Gabbi, A. M. et al. Use of infrared thermography to estimate enteric methane production in dairy heifers. Quant. Infrared Thermogr. J. 19, 187-195 (2022). https://doi.org/10.1080/17686733.2021.1882075.
• [4] Tao, N., Wang, C., Zhang, C. & Sun, J. Quantitative measurement of cast metal relics by pulsed thermal imaging. Quant. Infrared Thermogr. J. 19, 27-40 (2022). https://doi.org/10.1080/17686733.2020.1799304.
• [5] Shoa, P., Hemmat, A., Amirfattahi, R. & Gheysari, M. Automatic extraction of canopy and artificial reference temperatures for determination of crop water stress indices by using thermal imaging technique and a fuzzy-based image-processing algorithm. Quant. Infrared Thermogr. J. 19, 85-96 (2022). https://doi.org/10.1080/17686733.2020.1819707.
• [6] Ervural, S. & Ceylan, M. Thermogram classification using deep siamese network for neonatal disease detection with limited data. Quant. Infrared Thermogr. J. 19, 312-330 (2022). https://doi.org/10.1080/17686733.2021.2010379.
• [7] Yoon, S. T., Park, J. C. & Cho, Y. J. An experimental study on the evaluation of temperature uniformity on the surface of a blackbody using infrared cameras. Quant. Infrared Thermogr. J. 19, 172-186 (2022). https://doi.org/10.1080/17686733.2021.1877918.
• [8] Yixian, D., Dexin, H., Zewen, D. & Shuliang, Y. Non-destructive evaluation method for thermal parameters of prismatic Li-ion cell using infrared thermography. Quant. Infrared Thermogr. J. 20, 14-24 (2023). https://doi.org/10.1080/17686733.2021.2010380.
• [9] Goetten de Lima, G. et al. Effect of unidirectional freezing using a thermal camera on polyvinyl (alcohol) for aligned porous cryogels. Quant. Infrared Thermogr. J. 18, 177-186 (2021). https://doi.org/10.1080/17686733.2020.1732735.
• [10] Koroteeva, E. Yu. & Bashkatov, A. A. Thermal signatures of liquid droplets on a skin induced by emotional sweating. Quant. Infrared Thermogr. J. 19, 115-125 (2022). https://doi.org/10.1080/17686733.2020.1846113.
• [11] Kidangan, R. T., Krishnamurthy, C. V. & Balasubramaniam, K. Detection of dis-bond between honeycomb and composite facesheet of an inner fixed structure bond panel of a jet engine nacelle using infrared thermographic techniques. Quant. Infrared Thermogr. J. 19, 12-26 (2022). https://doi.org/10.1080/17686733.2020.1793284.
• [12] Schramm, S., Osterhold, P., Schmoll, R. & Kroll, A. Combining modern 3D reconstruction and thermal imaging: generation of large-scale 3D thermograms in real-time. Quant. Infrared Thermogr. J. 19, 295-311 (2022). https://doi.org/10.1080/17686733.2021.1991746.
• [13] Rogalski, A. Infrared Detectors, 2nd ed. (CRC Press, Boca Raton, 2011).
• [14] Higham, C. F, Murray-Smith, R., Padgett, M. J. & Edgar, M. P. Deep learning for real-time single-pixel video. Sci. Rep. 8, 2369 (2018). https://doi.org/10.1038/s41598-018-20521-y.
• [15] Baraniuk, R. G. Compresive sensing. IEEE Signal Process. Mag. 24, 118-121 (2007). https://doi.org/10.1109/MSP.2007.4286571.
• [16] Yi, K. et al. Novel LCDs with IR-sensitive backlights. J. Soc. Inf. Disp. 19, 48-56 (2011). https://doi.org/10.1889/JSID19.1.48.
• [17] Kim, B. H., Kim, M. Y. & Chae, Y. S. Background registration-based adaptive noise filtering of LWIR/MWIR imaging sensors for UAV applications. Sensors 18, 60 (2018). https://doi.org/10.3390/s18010060.
• [18] Tralic, D. & Grgic, S. Signal Reconstruction Via Compressive Sensing. in 53rd International Symposium Elmar-2011 5-9 (IEEE, 2011).
• [19] Candes, E., Romberg, J. & Tao, T. Robust uncertainty principles: exact signal reconstruction from highly incomplete frequency information. IEEE Trans. Inf. Theory 52, 489-509 (2006). https://doi.org/10.1109/TIT.2005.862083.
• [20] Bengio, Y., Simard, P. & Frasconi, P. Learning long-term dependencies with gradient descent is difficult. IEEE Trans. Neural Netw. 5, 157-166 (1994). https://doi.org/10.1109/72.279181.
• [21] Gerstoft, P., Mecklenbrauker, Ch. F., Seong, W. & Biancol, M. Introduction to compressive sensing in acoustic. J. Acoust. Soc. Am. 143, 3731-3736 (2018). https://doi.org/10.1121/1.5043089.
• [22] Gibson, G. M., Johnson, S. D. & Padgett, M. J. Single-pixel imaging 12 years on. Opt. Express 28, 28190-28208 (2020). https://doi.org/10.1364/OE.403195.
• [23] Kłosowski, G. et al. Using machine learning in electrical tomography for building energy efficiency through moisture detection. Energies 16, 1818 (2023). https://doi.org/10.3390/en16041818.
• [24] ℓ1-magic. https://candes.su.domains/software/l1magic/examples.html (2023).
• [25] Minkina, W. & Dudzik, S. Infrared Thermography: Errors and Uncertainties, 1st Ed. (Wiley, 2009).
• [26] Szajewska, A. Simulation of the operation of a single pixel camera with compressive sensing in the long-wave infrared. Pomiary Autom. Robot. 25, 53-60 (2021). https://doi.org/10.14313/PAR_240/53.
• [27] Strąkowski, R. & Więcek, B. Temperature Drift Compensation in Metrological Microbolometer Camera Using Multi Sensor Approach. in 13th Quantitative Infrared Thermography Conference (QIRT) 791-798 (QIRT Council, 2016). https://doi.org/10.21611/qirt.2016.126.
• [28] Shorten, C. & Khoshgoftaar, T. M. A survey on image data augmentation for deep learning. J. Big Data 6, 60 (2019). https://doi.org/10.1186/s40537-019-0197-0.
• [29] Więcek, P. & Sankowski, D. A New Deep-Learning Neural Network for Super-Resolution Up-Scaling of Thermal Images. in 15th Quantitative InfraRed Thermography Conference 1-7 (QIRT Council, 2020). http://www.qirt.org/archives/qirt2020/papers/134.pdf.
• [30] Valles, A., He, J., Ohno, S., Omatsu, T. & Miyamoto, K. Broadband high-resolution terahertz single-pixel imaging. Opt. Express 28, 28868-28881 (2020). https://doi.org/10.1364/OE.404143.
• [31] Strąkowska, M. & Strzelecki, M. Thermal time constant CNN-based spectrometry for biomedical applications. Sensors 23, 6658 (2023). https://doi.org/10.3390/s23156658.
• [32] Szegedy, Ch., Ioffe, S., Vanhoucke, V. & Alemi, A. Inception-v4, Inception-ResNet and the impact of residual connections on learning. https://arxiv.org/pdf/1602.07261v2.pdf (2016).
• [33] Takhar, D. et al. A Compressed Sensing Camera: New Theory and An Implementation Using Digital Micromirrors. in Proc. Comput. Imaging IV SPIE Electronic Imaging 1-10 (SPIE, 2006). https://doi.org/10.1117/12.659602.
• [34] Urbaś, S. & Więcek, B. Development of low-resolution, low-power and low-cost infrared System. Pomiary Autom. Robot. 25, 47-52 (2021). https://doi.org/10.14313/PAR_240/47.
• [35] Wang, Z., Bovik, A. C., Sheikh, H. R. & Simoncelli, E. P. Image quality assessment: from error visibility to structural similarity. IEEE Trans. Image Process. 13, 600-612 (2004). https://doi.org/10.1109/TIP.2003.819861.
• [36] Chollet, F. Keras https://keras.io (2015).
• [37] Urbaś, S., Więcek, P. & Więcek, B. Simulation of Single-Pixel IR Camera With CNN Reconstruction Algorithm. in 16th Quantitative InfraRed Thermography Conference (QIRT) (QIRT Council, 2022). https://www.ndt.net/article/qirt2022/papers/2017.pdf.
• [38] Teledyne Flir. https://www.flir.eu (Dec. 7th, 2023).
• [39] Bakurov, I., Castelli, M., Buzzelli, M. & Schettini, R. Parameters Optimization of The Structural Similiarity Index. in Proc. IS&T London Imaging Meeting 2020: Future Colour Imaging 19-23 (Society for Imaging Science and Technology, 2020). https://doi.org/10.2352/issn.2694-118X.2020.LIM-13.
• [40] Olbrycht, R. A novel method for sensitivity modelling of optical gas imaging thermal cameras with warm filters. Quant. Infrared Thermogr. J. 19, 331-346 (2022). https://doi.org/10.1080/17686733.2021.1962096.

Uwagi

Opracowanie rekordu ze środków MNiSW, umowa nr SONP/SP/546092/2022 w ramach programu "Społeczna odpowiedzialność nauki" - moduł: Popularyzacja nauki i promocja sportu (2024).