CNN-based superresolution reconstruction of 3D MR images using thick-slice scans

• Autorzy: Jurek Jakub , Kocinski Marek , Materka Andrzej , Elgalal Marcin , Majos Agata

Identyfikatory

DOI

10.1016/j.bbe.2019.10.003

• Języki publikacji: EN

Abstrakty

EN

Due to inherent physical and hardware limitations, 3D MR images are often acquired in the form of orthogonal thick slices, resulting in highly anisotropic voxels. This causes the partial volume effect, which introduces blurring of image details, appearance of staircase artifacts and significantly decreases the diagnostic value of images. To restore high resolution isotropic volumes, we propose to use a convolutional neural network (CNN) driven by patches taken from three orthogonal thick-slice images. To assess the validity and efficiency of this postprocessing approach, we used 1x1x1 mm3-voxel brain images of different modalities, available via the well known BrainWeb database. They served as a high resolution reference and were numerically preprocessed to create input images of different slice thickness and anatomical orientation, for CNN training, validation and testing. The visual quality of reconstructed images was indeed superior, compared to images obtained by fusion of interpolated thick-slice images, or to images reconstructed with the CNN using a single input MR scan. The significant increase of objectively computed figures of merit, e.g. the Structural Similarity Image Metric, was also noticed. Keeping in mind that any single value of such quality metrics represents a number of psychophysical effects, we applied the CNN trained on brain images for superresolution reconstruction of synthetic and acquired blood vessel tree images. We then used the restored superresolution volumes for estimation of vessel radii. It was demonstrated that vessel radius values derived from superresolution images of simulated vessel trees are significantly more accurate than those obtained from a standard fusion of interpolated thick-slice orthogonal scans. Superiority of the CNN-based superresolution images was also observed for scanner-acquired MR scans according to the evaluated parameters. These three experiments show the efficiency of CNN-based image reconstruction for qualitative and quantitative improvement of its diagnostic quality, as well as illustrates the practical usefulness of transfer learning - networks trained on example images of one kind can be used to restore superresolution images of physically different objects.

Słowa kluczowe

EN

MRI superresolution; orthogonal MR imaging; deep learning; convolutional neural network; transfer learning; blood vessel radius estimation

PL

rezonans magnetyczny; uczenie głębokie; konwolucyjna sieć neuronowa; transfer nauki; naczynia krwionośne

• Wydawca: Nałęcz Institute of Biocybernetics and Biomedical Engineering of the Polish Academy of Sciences ; Elsevier
• Czasopismo: Biocybernetics and Biomedical Engineering
• Rocznik: 2020
• Tom: Vol. 40, no. 1
• Strony: 111--125
• Opis fizyczny: Bibliogr. 34 poz., rys., tab., wykr.

Twórcy

autor

Jurek Jakub

• Institute of Electronics, Lodz University of Technology, Lodz, Poland

autor

Kocinski Marek

• Institute of Electronics, Lodz University of Technology, Lodz, Poland

autor

Materka Andrzej

• Institute of Electronics, Lodz University of Technology, Lodz, Poland

autor

Elgalal Marcin

• Department of Radiology, Medical University of Lodz, Lodz, Poland

autor

Majos Agata

• Molecular and Nanostructural Biophysics Laboratory, Bionanopark, Lodz, Poland

Bibliografia

• [1] Reisæter LA, Fütterer JJ, Halvorsen OJ, Nygård Y, Biermann M, Andersen E, Gravdal K, Haukaas S, Monssen JA, Huisman HJ, Akslen LA, Beisland C, Rørvik J. 1.5-t multiparametric mri using pi-rads: a region by region analysis to localize the index-tumor of prostate cancer in patients undergoing prostatectomy. Acta Radiol 2014;56(4):500–11. http://dx.doi.org/10.1177/0284185114531754. URL https:// doi.org/10.1177/0284185114531754.
• [2] Peled S, Yeshurun Y. Superresolution in mri: application to human white matter fiber tract visualization by diffusion tensor imaging. Magnetic resonance in medicine 2001;45:29–35.
• [3] Van Reeth E, Tham IWK, Tan CH, Poh CL. Super-resolution in magnetic resonance imaging: A review. Concepts in Magnetic Resonance Part A 2012;40A(6):306–25. http://dx.doi.org/10.1002/cmr.a.21249. arXiv:https:// onlinelibrary.wiley.com/doi/pdf/10.1002/cmr.a.21249, URL https://onlinelibrary.wiley.com/doi/abs/10.1002/cmr.a.21249.
• [4] Greenspan H, Oz G, Kiryati N, Peled S. Super-resolution in MRI. Proc IEEE Int Symp Biomedical Imaging 2002;943–6. http://dx.doi.org/10.1109/ISBI.2002.1029417.
• [5] Manjón JV, Coupé P, Buades A, Collins DL, Robles M. Mri superresolution using self-similarity and image priors. International journal of biomedical imaging 2010;2010:425891. http://dx.doi.org/10.1155/2010/425891.
• [6] Woo J, Murano EZ, Stone M, Prince JL. Reconstruction of high-resolution tongue volumes from mri. IEEE Transactions on Biomedical Engineering 2012;59(12):3511–24.
• [7] Jurek J, Kocinski M, Materka A, Losnegård A, Reisæter L, Halvorsen OJ, Beisland C, Rørvik J, Lundervold A. Reconstruction of high-resolution t2W mr images of the prostate using maximum a posteriori approach and Markov random field regularization. Proc and Applications (SPA) 2017 Signal Processing: Algorithms Architectures Arrangements 2017;96–9. http://dx.doi.org/10.23919/SPA.2017.8166845.
• [8] Shilling RZ, Robbie TQ, Bailloeul T, Mewes K, Mersereau RM, Brummer ME. A super-resolution framework for 3-D high- resolution and high-contrast imaging using 2-D multislice MRI. IEEE Transactions on Medical Imaging 2009;28(5):633–44. http://dx.doi.org/10.1109/TMI.2008.2007348.
• [9] Jia Y, He Z, Gholipour A, Warfield SK. Single anisotropic 3-D mr image upsampling via overcomplete dictionary trained from in-plane high resolution slices. IEEE Journal of Biomedical and Health Informatics 2016;20(6):1552–61. http://dx.doi.org/10.1109/JBHI.2015.2470682.
• [10] Jia Y, Gholipour A, He Z, Warfield SK. A new sparse representation framework for reconstruction of an isotropic high spatial resolution mr volume from orthogonal anisotropic resolution scans. IEEE Transactions on Medical Imaging 2017;36(5):1182–93. http://dx.doi.org/10.1109/TMI.2017.2656907.
• [11] Jurek J, Kocinski M, Materka A, Losnegård A, Reisæter L, Halvorsen OJ, Beisland C, Rørvik J, Lundervold A. Dictionary-based through-plane interpolation of prostate cancer t2-weighted mr images. Proc and Applications (SPA) 2018 Signal Processing: Algorithms Architectures Arrangements 2018;168–73. http://dx.doi.org/10.23919/SPA.2018.8563411.
• [12] Oktay O, Ferrante E, Kamnitsas K, Heinrich M, Bai W, Caballero J, Cook SA, de Marvao A, Dawes T, O'Regan DP, Kainz B, Glocker B, Rueckert D. Anatomically constrained neural networks (acnns): Application to cardiac image enhancement and segmentation. IEEE Transactions on Medical Imaging 2018;37(2):384–95. http://dx.doi.org/10.1109/TMI.2017.2743464.
• [13] Shi J, Liu Q, Wang C, Zhang Q, Ying S, Xu H. Superresolution reconstruction of mr image with a novel residual learning network algorithm. Physics in medicine and biology 2018;63:085011. http://dx.doi.org/10.1088/1361-6560/aab9e9.
• [14] Shi J, Li Z, Ying S, Wang C, Liu Q, Zhang Q, Yan P. Mr image super-resolution via wide residual networks with fixed skip connection. IEEE Journal of Biomedical and Health Informatics 2018;1. http://dx.doi.org/10.1109/JBHI.2018.2843819.
• [15] Chen Y, Shi F, Christodoulou AG, Xie Y, Zhou Z, Li D. Efficient and accurate mri super-resolution using a generative adversarial network and 3d multi-level densely connected network. Medical Image Computing and Computer Assisted Intervention - MICCAI 2018. http://dx.doi.org/10.1007/978-3-030-00928-1_11. URL https:// doi.org/10.1007/978-3-030-00928-1_11.
• [16] Litjens G, Kooi T, Bejnordi BE, Setio AAA, Ciompi F, Ghafoorian M, Van Der Laak JA, Van Ginneken B, Sánchez CI. A survey on deep learning in medical image analysis. Medical image analysis 2017;42:60–88.
• [17] S. McDonagh, B. Hou, A. Alansary, O. Oktay, K. Kamnitsas, M. Rutherford, J.V. Hajnal, B. Kainz, Context-sensitive super-resolution for fast fetal magnetic resonance imaging, Molecular Imaging, Reconstruction and Analysis of Moving Body Organs, and Stroke Imaging and Treatmentdoi:10.1007/978-3-319-67564-0_12. URL https:// doi.org/10.1007/978-3-319-67564-0_12.
• [18] Pham C-H, Fablet R, Rousseau F. Multi-scale brain mri super-resolution using deep 3d convolutional networks; 2017, Accessed on: 1.04.2019. URL https://hal.archives- ouvertes.fr/hal-01635455.
• [19] Carmi E, Liu S, Liu S, Alon N, Fiat A, Fiat D. Resolution enhancement in mri. Magnetic resonance imaging 2006;24:133–54. http://dx.doi.org/10.1016/j.mri.2005.09.011.
• [20] Oktay O, Bai W, Lee M, Guerrero R, Kamnitsas K, Caballero J, Marvao A, Cook S, O'Regan D, Rueckert D. Multi-input cardiac image super-resolution using convolutional neural networks. Medical Image Computing and Computer- Assisted Intervention - MICCAI 2016. http://dx.doi.org/10.1007/978-3-319-46726-9_29. URL https:// doi.org/10.1007/978-3-319-46726-9_29.
• [21] Li F-F, Karpathy A, Johnson J. Convolutional neural networks for visual recognition; 2016, Accessed on: 1.04.2019. URL http://cs231n.stanford.edu/slides/2016/ winter1516_lecture11.pdf.
• [22] Pham C, Ducournau A, Fablet R, Rousseau F. Brain MRI super-resolution using deep 3D convolutional networks. Proc IEEE 14th Int Symp Biomedical Imaging (ISBI 2017) 2017;197–200. http://dx.doi.org/10.1109/ISBI.2017.7950500.
• [23] http://www.bic.mni.mcgill.ca/brainweb/, Accessed on: 1.02.2019.
• [24] R. K. S. Kwan, A.C. Evans, G.B. Pike, An extensible mri simulator for post-processing evaluation, Visualization in Biomedical Computingdoi:10.1007/BFb0046947. URL https:// doi.org/10.1007/BFb0046947.
• [25] Collins DL, Zijdenbos AP, Kollokian V, Sled JG, Kabani NJ, Holmes CJ, Evans AC. Design and construction of a realistic digital brain phantom. IEEE transactions on medical imaging 1998;17:463–8. http://dx.doi.org/10.1109/42.712135.
• [26] Karch R, Neumann F, Neumann M, Szawlowski P, Schreiner W. Voronoi polyhedra analysis of optimized arterial tree models. Annals of biomedical engineering 2003;31:548–63.
• [27] Kocinski M, Klepaczko A, Materka A, Chekenya M, Lundervold A. 3d image texture analysis of simulated and real-world vascular trees. Computer methods and programs in biomedicine 2012;107:140–54. http://dx.doi.org/10.1016/j.cmpb.2011.06.004.
• [28] Kocinski M, Materka A, Deistung A, Reichenbach JR. Centerline-based surface modeling of blood-vessel trees in cerebral 3D MRA. Proc and Applications (SPA) 2016 Signal Processing: Algorithms Architectures Arrangements 2016;85–90. http://dx.doi.org/10.1109/SPA.2016.7763592.
• [29] Kocinski M, Materka A, Elgalal M, Majos A. On accuracy of personalized 3D-printed MRI-based models of brain arteries. Proc Signals and Image Processing (IWSSIP) 2017 Int Conf Systems 2017;1–5. http://dx.doi.org/10.1109/IWSSIP.2017.7965601.
• [30] Dong C, Loy CC, He K, Tang X. Image super-resolution using deep convolutional networks. IEEE transactions on pattern analysis and machine intelligence 2016;38(2):295–307.
• [31] Srinivasan K, Ankur A, Sharma A. Super-resolution of magnetic resonance images using deep convolutional neural networks. Proc IEEE Int Conf Consumer Electronics - Taiwan (ICCE-TW) 2017;41–2. http://dx.doi.org/10.1109/ICCE-China.2017.7990985.
• [32] V. Dumoulin, F. Visin, A guiSrinivasan K., Ankur A., Sharma A., Supemetic for deep learning, Accessed on: 1.04.2019. URL http://arxiv.org/abs/1603.07285v2.
• [33] Wang Z, Simoncelli EP, Bovik AC. Multiscale structural similarity for image quality assessment. The Thirty- Seventh Asilomar Conference on Signals Systems & Computers 2003 Vol 2 2003;1398–402.
• [34] Kjer MH, Wilm J. Evaluation of surface registration algorithms for PET motion correction; 2010, Accessed on: 10.07.2019. URL http://www2.imm.dtu.dk/jakw/ publications/bscthesis.pdf.

Uwagi

PL

Opracowanie rekordu ze środków MNiSW, umowa Nr 461252 w ramach programu "Społeczna odpowiedzialność nauki" - moduł: Popularyzacja nauki i promocja sportu (2020).