Supervised pearlitic–ferritic steel microstructure segmentation by U-Net convolutional neural network

• Autorzy: Motyl Mateusz , Madej Łukasz

Identyfikatory

DOI

10.1007/s43452-022-00531-4

• Języki publikacji: EN

Abstrakty

EN

The aim of this work is to develop an automated procedure based on machine learning capabilities for the identification of the pearlite islands within the two-phase pearlitic–ferritic steel. The input parameters for the custom implementation of a braided neural network are provided as a data set of scanning electron microscopy images of metallographic specimens. The procedures related to the processing of the data and the optimization parameters affecting the final architecture and effectiveness of the network learning stage are examined. The objective is to find the best solution to the problem of ferritic–pearlitic microstructure segmentation, allowing further processing during, e.g., 3D reconstruction of data from serial sectioning. The work examines the various quality of input data and different U-Net architectures to find the one that can identify pearlite islands with the highest precision. Two types of images acquired from secondary electron (SE) and electron backscattered diffraction (EBSD) detectors are used during the investigation. The work revealed that the developed approach offers improvements in metallographic investigations by removing the requirement for expert knowledge for the interpretation of image data prior to further characterization. It has also been proven that artificial neural networks based on the deep learning process using extensible U-Net network architectures and nonlinear learning tools can identify pearlite islands within a two-phase microstructure, while the overtraining level remains low. Convolutional neural networks do not require manual feature extraction and are able to automatically find appropriate search functions to recognize pearlite structure areas in the training process without human intervention. It was shown that the network recognizes areas of analyzed steel with satisfactory precision of 79% for EBSD and 87% for SE images.

Słowa kluczowe

EN

artificial neural networks; supervised learning; microstructure characterization

PL

sztuczna sieć neuronowa; mikrostruktura; mikroskop elektronowy

• Wydawca: Springer ; Wrocław University of Science and Technology
• Czasopismo: Archives of Civil and Mechanical Engineering
• Rocznik: 2022
• Tom: Vol. 22, no. 4
• Strony: art. no. e206, 2022
• Opis fizyczny: Bibliogr. 35 poz., rys., tab., wykr.

Twórcy

autor

Motyl Mateusz

• Faculty of Metals Engineering and Industrial Computer Science, AGH University of Science and Technology, Mickiewicza 30 av., 30-059 Krakow, Poland

autor

Madej Łukasz

• Faculty of Metals Engineering and Industrial Computer Science, AGH University of Science and Technology, Mickiewicza 30 av., 30-059 Krakow, Poland

Bibliografia

• [1] Raabe D, Sun B, Kwiatkowski Da Silva A, Gault B, Yen H-W, Sedighiani K, Sukumar PT, Filho IRS, Katnagallu S, Jägle E, Kürnsteiner P, Kusampudi N, Stephenson L, Herbig M, Liebscher CH, Springer H, Zaefferer S, Shah V, Wong S-L, Baron C, Diehl M, Roters F, Ponge D. Current challenges and opportunities in microstructure-related properties of advanced high-strength steels. Metall Mater Trans A. 2020;51:5517–86.
• [2] Adamczyk-Cieślak B, Koralnik M, Kuziak R, Majchrowicz K, Mizera J. Studies of bainitic steel for rail applications based on carbide-free, low-alloy steel. Metall Mater Trans A. 2021;52:5429–42.
• [3] Liu X. Microstructural characterisation of pearlitic and complex phase steels using image analysis method, Xi Liu, PhD thesis, Birmingham University; 2014.
• [4] Roskosz S, Chrapoński J, Madej L. Application of systematic scanning and variance analysis method to evaluation of pores arrangement in sintered steel. Measurements. 2021;168: 108325.
• [5] Banerjee S, Ghosh SK, Datta S, Saha KS. Segmentation of dual phase steel micrograph: an automated approach. Measurement. 2013;46:2435–40.
• [6] Bhadeshia HKDH. Neural networks in materials science. ISIJ Int. 1999;39:966–79.
• [7] Gurney K. An introduction to neural networks. New York: UCL Press; 1997.
• [8] Widrow B, Lehr MA. 30 years of adaptive neural networks: perceptron, madaline, and backpropagation. Conf Proc IEEE. 1990;78:1415–42.
• [9] Fukushima K. Neocognitron: a self organizing neural network model for a mechanism of pattern recognition unaffected by shift in position. Biol Cybern. 1980;36:193–202.
• [10] Cireşan D. High-performance neural networks for visual object classification. www.arxiv.org; 2011.
• [11] Cireşan D. Multi-column deep neural network for traffic sign classification. Neural Netw. 2012;32:333–8.
• [12] Bozinovski S. Teaching space: a representation concept for adaptive pattern classification. COINS Technical Report; 1981. p.81–2.
• [13] Bengio IGY, Courville A. Deep learning. New York: MIT Press; 2017.
• [14] Russell S, Norvig P. Artificial intelligence, a modern approach. 2nd ed. New York: Prentice Hall; 2003.
• [15] Alpaydin E. Introduction to machine learning. New York: MIT Press; 2020.
• [16] Kim H, Inoue J, Kasuya T. Unsupervised microstructure segmentation by mimicking metallurgists’ approach to pattern recognition. Sci Rep. 2020;10:17835.
• [17] Sitko M, Mojżeszko M, Rychłowski Ł, Cios G, Bała P, Muszka K, Madej L. Numerical procedure of three-dimensional reconstruction of ferrite–pearlite microstructure data from SEM/EBSD serial sectioning. Proc Manuf. 2020;47:1217–22.
• [18] de Albuquerque VHC, Cortez PC, de Alexandria AR, Tavares JMRS. A new solution for automatic microstructures analysis from images based on a backpropagation artificial neural network. Nondestruct Test Eval. 2008;23:273–83.
• [19] Wang S, Xia X, Ye L, Yang B. Automatic detection and classification of steel surface defect using deep convolutional neural networks. Metals. 2021;11:388.
• [20] Abu M, Amir A, Lean YH, Zahri NAH, Azemi SA. The performance analysis of transfer learning for steel defect detection by using deep learning. J Phys Conf Ser. 2021;1755:1.
• [21] Yeom J, Stan T, Hong S, Voorhees PW. Segmentation of experimental datasets via convolutional neural networks trained on phase field simulations. Acta Mater. 2021;214:1.
• [22] Ostormujof TM, Purushottam Raj Purohit RRP, Breumier S, Gey N, Salib M, Germain L. Deep Learning for automated phase segmentation in EBSD maps. A case study in dual phase steel microstructures. Mater Charact. 2022;184:111638.
• [23] Ackermann M, Iren D, Wesselmecking S, Shetty D, Krupp U. Automated segmentation of martensite-austenite islands in bainitic steel. Mater Charact. 2022;191:112091.
• [24] Breumier S, Ostormujof TM, Frincu B, Gey N, Couturier A, Loukachenko N, Abaperea PE, Germain L. Leveraging EBSD data by deep learning for bainite, ferrite and martensite segmentation. Mater Charact. 2022;186:111805.
• [25] Luengo J, Moreno R, Sevillano I, Charte D, Peláez-Vegas A, Fernández-Moreno M, Mesejo P, Herrera F. A tutorial on the segmentation of metallographic images: taxonomy, new Metal-DAM dataset, deep learning-based ensemble model, experimental analysis and challenges. Inf Fus. 2022;78:232–53.
• [26] Tian W, Cheng X, Liu Q, Yu C, Gao F, Chi Y. Meso-structure segmentation of concrete CT image based on mask and regional convolution neural network. Mater Des. 2021;208:1.
• [27] Madej L. Virtual microstructures in application to metals engineering—a review. Arch Civ Mech Eng. 2017;17:839–54.
• [28] Ronneberger O, Fischer P, Brox T. U-Net: convolutional networks for biomedical image segmentation. In: Confernece of proceeding of the medical image computing and computer-assisted intervention, Munich; 2015. p. 234–41.
• [29] Sara U, Akter M, Uddin MS. Image quality assessment through FSIM, SSIM, MSE and PSNR, a comparative study. J Comput Commun. 2019;7:8–18.
• [30] Gulli A, Pal S. Deep learning with Keras. New York: Packt Publishing; 2017.
• [31] Long J, Shelhamer E, Darrell T. Fully convolutional networks for semantic segmentation. www.arxiv.org; 2014.
• [32] Abu M, Amir A, Lean YH, Zahri NAH, Azemi SA. The performance analysis of transfer learning for steel defect detection by using deep learning. J Phys Conf Ser. 2020;1755:1.
• [33] Ruder S. An overview of gradient descent optimization algorithms. www.arxiv.org; 2016.
• [34] Wang Z, Simoncelli EP, Bovik AC. Multiscale structural similarity for image quality assessment. In: Conference proceeding of the asilomar conference on signals, systems and computers, vol. 2; 2004. p. 1398–402.
• [35] Kumar R, Moyal V. Visual image quality assessment technique using FSIM. Int J Comput Appl Technol Res. 2013;2:250–4. https://doi.org/10.7753/IJCATR0203.1008.

Uwagi

PL

Opracowanie rekordu ze środków MEiN, umowa nr SONP/SP/546092/2022 w ramach programu "Społeczna odpowiedzialność nauki" - moduł: Popularyzacja nauki i promocja sportu (2022-2023)