Conditional generative adversarial network based data augmentation for fault diagnosis of diesel engines applied with infrared thermography and deep convolutional neural network

• Autorzy: Wang Rongcai , Jia Xisheng , Liu Zichang , Dong Enzhi , Li Siyu , Cheng Zhonghua

Identyfikatory

DOI

10.17531/ein/175291

• Języki publikacji: EN

Abstrakty

EN

This paper tries to introduce a new intelligent method for the early fault diagnosis of diesel engines. Firstly, infrared thermography (IRT) is introduced into diesel engine condition monitoring, then infrared images of diesel engines in four health states, such as normal condition, single-cylinder misfire, multi-cylinder misfire and air filter blockage, are collected and the region of interest (ROI) of infrared images are extracted. Next, conditional generative adversarial network (CGAN) is deployed to perform data augmentation on infrared image datasets. Then, deep convolutional neural network (DCNN) and Softmax regression (SR) classifier are used for automatically extracting infrared image fault features and pattern recognition, respectively. Finally, a comparison with three deep learning (DL) models is performed. The validation results show that the data augmentation method proposed in the paper can significantly improve the early fault diagnosis accuracy, and DCNN has the best fault diagnosis effect andresistance to temperature fluctuation interference among the four DL models.

Słowa kluczowe

EN

diesel engines; data augmentation; conditional generative adversarial network; infrared thermography; deep convolutional neural network; fault diagnosis

• Wydawca: Polskie Naukowo-Techniczne Towarzystwo Eksploatacyjne
• Czasopismo: Eksploatacja i Niezawodność
• Rocznik: 2024
• Tom: Vol. 26, no. 1
• Strony: art. no. 175291
• Opis fizyczny: Bibliogr. 55 poz., rys., tab., wykr.

Twórcy

autor

Wang Rongcai

• Shijiazhuang Campus, Army Engineering University of PLA, China

• https://orcid.org/0000-0001-6839-1181

autor

Jia Xisheng

• Shijiazhuang Campus, Army Engineering University of PLA, China

autor

Liu Zichang

• Shijiazhuang Campus, Army Engineering University of PLA, China

autor

Dong Enzhi

• Shijiazhuang Campus, Army Engineering University of PLA, China

autor

Li Siyu

• Shijiazhuang Campus, Army Engineering University of PLA, China

autor

Cheng Zhonghua

• Shijiazhuang Campus, Army Engineering University of PLA, China

Bibliografia

• 1. Baranitharan P, Kumanan S, Sudhagar S. An intelligent novel approach for diesel engine life monitoring using infrared image processing technic. Environmental Progress and Sustainable Energy 2022; 41(1):e13712, https://doi.org/10.1002/ep.13712.
• 2. Bi X, Lin J, Tang D, et al. VMD-KFCM algorithm for the fault diagnosis of diesel engine vibration signals. Energies 2020; 13(1): 228, https://doi.org/10.3390/en13010228.
• 3. Chen Z, Duan J, Kang L, Qiu G. Supervised anomaly detection via conditional generative adversarial network and ensemble active learning. IEEE Transactions on Pattern Analysis and Machine Intelligence 2022; 45(6): 7781–7798, https://doi.org/10.1109/TPAMI.2022.3225476.
• 4. Choudhary A, Mian T, Fatima S. Convolutional neural network based bearing fault diagnosis of rotating machine using thermal images. Measurement 2021; 176: 109196, https://doi.org/10.1016/j.measurement.2021.109196.
• 5. Ciabattoni L, Ferracuti F, Freddi A, Monteri A. Statistical spectral analysis for fault diagnosis of rotating machines. IEEE Transactions on Industrial Electronics 2018; 65(5): 4301–4310, https://doi.org/10.1109/TIE.2017.2762623.
• 6. Courtrai L, Pham M T, Lefèvre S. Small object detection in remote sensing images based on super-resolution with auxiliary generative adversarial networks. Remote Sensing 2020; 12(19): 3152, https://doi.org/10.3390/rs12193152.
• 7. Dong J, Yin R, Sun X, et al. Inpainting of remote sensing SST images with deep convolutional generative adversarial network. IEEE Geoscience and Remote Sensing Letters 2019; 16(2): 173–177, https://doi.org/10.1109/LGRS.2018.2870880.
• 8. Engine D, Clearance V, Diagnosis F. Diesel engine valve clearance fault diagnosis based on improved variational mode decomposition and bispectrum. Energies 2019; 12(4): 661, https://doi.org/10.3390/en12040661.
• 9. Flett J, Bone G M. Fault detection and diagnosis of diesel engine valve trains. Mechanical Systems and Signal Processing 2016; (72–73): 316–327, https://doi.org/10.1016/j.ymssp.2015.10.024.
• 10. Gan C, Xiao J, Wang Z, et al. Facial expression recognition using densely connected convolutional neural network and hierarchical spatial attention. Image and Vision Computing 2022; 117: 104342, https://doi.org/10.1016/j.imavis.2021.104342.
• 11. Goodfellow I, Pouget-Abadie J, Mirza M, et al. Generative adversarial networks. Communications of the ACM 2020; 63(11): 139–144, https://doi.org/10.1145/3422622
• 12. Gu C, Qiao X, Li H, Jin Y. Misfire fault diagnosis method for diesel engine based on MEMD and dispersion entropy. Shock and Vibration 2021; https://doi.org/10.1155/2021/9213697.
• 13. He M, He D. Deep learning based approach for bearing fault diagnosis. IEEE Transactions on Industry Applications 2017; 53(3):3057–3065, https://doi.org/10.1109/TIA.2017.2661250.
• 14. Hou L, Zou J, Du C, Zhang J. A fault diagnosis model of marine diesel engine cylinder based on modified genetic algorithm andmultilayer perceptron. Soft Computing 2020; 24: 7603–7613, https://doi.org/10.1007/s00500-019-04388-3.
• 15. Isola P, Zhu J Y, Zhou T, Efros A A. Image-to-image translation with conditional adversarial networks. 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) 2017; pp. 5967-5976, https://doi.org/10.1109/CVPR.2017.632.
• 16. Janssens O, Schulz R, Slavkovikj V, Stockman K, Loccufier M, Walle R V D, Hoecke S V. Thermal image based fault diagnosis forrotating machinery. Infrared Physics and Technology 2015; 73: 78–87, https://doi.org/0.1016/j.infrared.2015.09.004.
• 17. Jia F, Lei Y, Lin J, et al. Deep neural networks: A promising tool for fault characteristic mining and intelligent diagnosis of rotating machinery with massive data. Mechanical Systems and Signal Processing 2016; 72–73: 303–315, https://doi.org/10.1016/j.ymssp.2015.10.025.
• 18. Jia Z, Liu Z, Vong C M, Pecht M. A rotating machinery fault diagnosis method based on feature learning of thermal images. IEEE Access 2019; 7: 12348–12359, https://doi.org/10.1109/ACCESS.2019.2893331.
• 19. Kowalski J, Krawczyk B, Woźniak M. Fault diagnosis of marine 4-stroke diesel engines using a one-vs-one extreme learning ensemble. Engineering Applications of Artificial Intelligence 2017; 57: 134–141, https://doi.org/10.1016/j.engappai.2016.10.015.
• 20. LeCun Y, Bottou L, Bengio Y, et al. Gradient-based learning applied to document recognition. Proceedings of the IEEE 1998; 86(11): 2278–2323, https://doi.org/10.1109/5.726791.
• 21. Lee W, Seong J J, Ozlu B, et al. Biosignal sensors and deep learning-based speech recognition: A review. Sensors 2021; 21(4): 1399, https://doi.org/10.3390/s21041399.
• 22. Lei Y, Jia F, Lin J, et al. An intelligent fault diagnosis method using unsupervised feature learning towards mechanical big data. IEEE Transactions on Industrial Electronics 2016; 63(5): 3137–3147, https://doi.org/10.1109/TIE.2016.2519325.
• 23. Li Y. Copy-move detection method based on conditional generative adversarial networks. Master's Thesis, University of Chinese Academy of Sciences 2021. (In Chinese)
• 24. Li Y, Du X, Wang X, Si S. Industrial gearbox fault diagnosis based on multi-scale convolutional neural networks and thermal imaging. ISA Transactions 2022; 129: 309–320, https://doi.org/10.1016/j.isatra.2022.02.048.
• 25. Li Y, Du X, Wan F, Wang X, Yu H. Rotating machinery fault diagnosis based on convolutional neural network and infrared thermal imaging. Chinese Journal of Aeronautics 2020; 33(2): 427–438, https://doi.org/10.1016/j.cja.2019.08.014.
• 26. Li Y, Gu J, Zhen D, Xu M, Ball A. An evaluation of gearbox condition monitoring using infrared thermal images applied with convolutional neural networks. Sensors 2019; 19(9): 2205, https://doi.org/10.3390/s19092205.
• 27. Liu H, Li L, Ma J. Rolling bearing fault diagnosis based on STFT-deep learning and sound signals. Shock and Vibration 2016; https://doi.org/10.1155/2016/6127479.
• 28. Liu Q, He X, Teng Q, Qing L, Chen H. BDNet: A BERT-based dual-path network for text-to-image cross-modal person re-identification. Pattern Recognition 2023; 141: 109636, https://doi.org/10.1016/j.patcog.2023.109636.
• 29. Liu Y, Lai K W C. The performance index of convolutional neural network-based classifiers in class imbalance problem. Pattern Recognition 2023; 137: 109284, https://doi.org/10.1016/j.patcog.2022.109284.
• 30. Mirza M, Osindero S. Conditional Generative Adversarial Nets. arXiv:1411.1784 2014; https://doi.org/10.48550/arXiv.1411.1784.
• 31. Molinier N, Painchaud-april G, Duff A L, Toews M, Bélanger P. Ultrasonic imaging using conditional generative adversarial networks. Ultrasonics 2023; 133: 107015, https://doi.org/10.1016/j.ultras.2023.107015.
• 32. Nikbakht R, Jonsson A, Lozano A. Unsupervised learning for parametric optimization. IEEE Communications Letters 2021; 25(3): 678–681, https://doi.org/10.1109/LCOMM.2020.3027981.
• 33. Pan M H, Xin H Y, Xia C Q, et al. Few-shot classification with task-adaptive semantic feature learning. Pattern Recognition 2023; 141:
• 34. Ramteke S M, Chelladurai H, Amarnath M. Diagnosis and classification of diesel engine components faults using time–frequency and machine learning approach. Journal of Vibration Engineering and Technologies 2022; 10: 175–192, https://doi.org/10.1007/s42417-021-00370-2.
• 35. Russakovsky O, Deng J, Su H, Krause J, Satheesh S, et al. ImageNet Large Scale Visual Recognition Challenge. International Journal of Computer Vision 2015; 115(3): 211–252, https://doi.org/10.1007/s11263-015-0816-y.
• 36. Shi G, Li Y, Qian Y. Research on the applications of infrared technique in the diagnosis and prediction of diesel engine exhaust fault. Journal of Thermal Science 2011; 20(2): 189–194, https://doi.org/10.1007/s11630-011-0456-7.
• 37. Su Q, Xi X, Yuan Y, et al. Application of ultrasonic infrared thermography technology in crack detection of aeroengine blades. Nondestructive Testing 2019; 41(4): 54-57. (In Chinese)
• 38. Sun J, Yan C, Wen J. Intelligent bearing fault diagnosis method combining compressed data acquisition and deep learning. IEEETransactions on Instrumentation and Measurement 2018; 67(1): 185–195, https://doi.org/10.1109/TIM.2017.2759418.
• 39. Tran V T, Yang B S, Gu F, Ball A. Thermal image enhancement using bi-dimensional empirical mode decomposition in combination with relevance vector machine for rotating machinery fault diagnosis. Mechanical Systems and Signal Processing 2013; 38(2): 601–614, https://doi.org/10.1016/j.ymssp.2013.02.001.
• 40. Varshney D, Ekbal A, Tiwari M, Nagaraja G P. EmoKbGAN: Emotion controlled response generation using Generative Adversarial Network for knowledge grounded conversation. PLoS ONE 2023; 18(2): e0280458, https://doi.org/10.1371/journal.pone.0280458.
• 41. Verstraete D, Ferrada A, Droguett E L, et al. Deep learning enabled fault diagnosis using time-frequency image analysis of rolling element bearings. Shock and Vibration 2017; https://doi.org/10.1155/2017/5067651.
• 42. Wang R, Zhan X, Bai H, Dong E, Cheng Z, Jia X. A review of fault diagnosis methods for rotating machinery using infrared thermography. Micromachines 2022; 13(10): 1644, https://doi.org/10.3390/mi13101644.
• 43. Wang R, Chen H, Guan C. DPGCN model: A novel fault diagnosis method for marine diesel engines based on imbalanced datasets. IEEE Transactions on Instrumentation and Measurement 2023; 72: 1–11, https://doi.org/10.1109/TIM.2022.3228002.
• 44. Wang W, Li Q. Crack identification of infrared thermal imaging steel sheet based on convolutional neural network. MATEC Web of Conferences 2018; 232(11): 01053, https://doi.org/10.1051/matecconf/201823201053.
• 45. Wang Z, Xia H, Zhang J, Yang B, Yin W. Imbalanced sample fault diagnosis method for rotating machinery in nuclear power plants based on deep convolutional conditional generative adversarial network. Nuclear Engineering and Technology 2023; 55(6): 2096–2106, https://doi.org/10.1016/j.net.2023.02.036.
• 46. Wu K, Mei Y. Multi-focus image fusion based on unsupervised learning. Machine Vision and Applications 2022; 33(5): 1–14, https://doi.org/10.1007/s00138-022-01326-6.
• 47. Xia M, Li T, Xu L, et al. Fault diagnosis for rotating machinery using multiple sensors and convolutional neural networks. IEEE/ASME Transactions on Mechatronics 2018; 23(1): 101–110, https://doi.org/10.1109/TMECH.2017.2728371.
• 48. Xiao L, Mao Q, Lan P, Zang X, Liao Z. A fault diagnosis method of insulator string based on infrared image feature extractionand probabilistic neural network. 10th International Conference on Intelligent Computation Technology and Automation (ICICTA) 2017; pp. 80–85; https://doi.org/10.1109/ICICTA.2017.25.
• 49. Xu G, Liu M, Jiang Z, Dirk Söffker D, Shen W. Bearing fault diagnosis method based on deep convolutional neural network and random forest ensemble learning. Sensors 2019; 19(5): 1088, https://doi.org/10.3390/s19051088.
• 50. Xu J, Zhang Z, Hu X. Extracting semantic knowledge from GANs with unsupervised learning. IEEE Transactions on Pattern Analysis and Machine Intelligence 2023; 45(8): 9654–9668, https://doi.org/10.1109/TPAMI.2023.3262140.
• 51. Yan H, Bai H, Zhan X, Wu Z, Wen L, Jia X. Combination of VMD mapping MFCC and LSTM : A new acoustic fault diagnosis method of diesel engine. Sensors 2022; 22(21): 8325, https://doi.org/10.3390/s22218325.
• 52. Yang Y, Zhou Y, Yue X, et al. Real-time detection of crop rows in maize fields based on autonomous extraction of ROI. Expert Systems with Applications 2023; 213(Part A): 118826, https://doi.org/10.1016/j.eswa.2022.118826.
• 53. Zabihi-hesari A, Ansari-rad S, Shirazi F A. Fault detection and diagnosis of a 12-cylinder trainset diesel engine based on vibration signature analysis and neural network. Proceedings of the Institution of Mechanical Engineers, Part C: Journal of Mechanical Engineering Science –1923, https://doi.org/10.1177/0954406218778313.
• 54. Zhao H, Zhang J, Jiang Z, Wei D, Zhang X, Mao Z. A new fault diagnosis method for a diesel engine based on an optimized vibration Mel frequency under multiple operation conditions. Sensors 2019; 19(11): 2590, https://doi.org/10.3390/s19112590.
• 55. Zhao M, Lin J. Health assessment of rotating machinery using a rotary encoder. IEEE Transactions on Industrial Electronics 2018; 65(3): 2548–2556, https://doi.org/10.1109/TIE.2017.2739689.