The Development of a Generative Approach for Joint Super-Resolution Image Reconstruction from Highly Sparse Raw Data in the Context of MR-PET Imaging

• Autorzy: Malczewski Krzysztof

Identyfikatory

DOI

10.22630/MGV.2023.32.3.9

• Języki publikacji: EN

Abstrakty

EN

The present study introduces a rapid and efficient approach for reconstructing high-resolution images in hybrid MRI-PET scanners. The application of sparsity, compressed sensing (CS), and super-resolution reconstruction (SRR) methodologies can significantly decrease the demands of data acquisition while concurrently attaining high-resolution output. G-guided generative multilevel networks for sparsely sampled MR-PET input are shown here. Compressed Sensing using conjugate symmetry and Partial Fourier methodology speeds up data collection over k-space sampling methods. GANs and k-space adjustments are used in this image domain technique. The employed methodology utilizes discrete preprocessing stages to effectively tackle the challenges associated with the deblurring, reducing motion artifacts, and denoising of layers. Initial trials offer contextual details and accelerate evaluations. Preliminary experiments provide contextual information and expedite assessments.

Słowa kluczowe

EN

GAN; WGAN; super-resolution; compressive sensing; medical modalities

• Wydawca: Institute of Information Technology of the Warsaw University of Life Sciences – SGGW
• Czasopismo: Machine Graphics and Vision
• Rocznik: 2023
• Tom: Vol. 32, No. 3/4
• Strony: 161--191
• Opis fizyczny: Bibliogr. 49 poz., rys., tab.

Twórcy

autor

Malczewski Krzysztof

• Institute of Information Technology, Warsaw University of Life Sciences – SGGW, Warsaw, Poland

• https://orcid.org/0000-0002-6450-6444

Bibliografia

• [1] G. Antoch and A. Bockisch. Combined PET/MRI: a new dimension in whole-body oncology imaging? European Journal of Nuclear Medicine and Molecular Imaging, 36(S1):113-120, 2008. doi:10.1007/s00259-008-0951-6.
• [2] M. P. Branco, A. Gaglianese, D. R. Glen, D. Hermes, Z. S. Saad, et al. ALICE: A tool for automatic localization of intra-cranial electrodes for clinical and high-density grids. Journal of Neuroscience Methods, 301:43-51, 2018. doi:10.1016/j.jneumeth.2017.10.022.
• [3] J. Bruna, P. Sprechmann, and Y. LeCun. Super-resolution with deep convolutional sufficient statistics. In: Proc. Int. Conf. Learning Representation (ICLR), 2015. Proceedings published in arXiv, see https://iclr.cc/archive/www/doku.php%3Fid=iclr2015:accepted-main. html. doi:10.48550/arXiv.1412.7022.
• [4] H. Chang, D. Y. Yeung, and Y. Xiong. Super-resolution through neighbor embedding. In: Proc. 2004 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR), vol. 1. Washington, USA, 2004. doi:10.1109/CVPR.2004.1315043.
• [5] Y. Chen, F. Shi, A. G. Christodoulou, Y. Xie, Z. Zhou, et al. Efficient and accurate MRI super-resolution using a generative adversarial network and 3D multi-level densely connected network. In: A. F. Frangi, J. A. Schnabel, C. Davatzikos, C. Alberola-López, and G. Fichtinger, eds., Proc. Conf. Medical Image Computing and Computer Assisted Intervention (MICCAI) 2018, vol. 11070 of Lecture Notes in Computer Sciences, pp. 91-99. Springer International Publishing, Granada, Spain, 16-20 Sep 2018. doi:10.1007/978-3-030-00928-1_11.
• [6] K. Dabov, A. Foi, V. Katkovnik, and K. Egiazarian. Image denoising by sparse 3-D transform-domain collaborative filtering. IEEE Transactions on Image Processing, 16(8):2080-2095, 2007. doi:10.1109/TIP.2007.901238.
• [7] C. Dong, C. C. Loy, K. He, and X. Tang. Learning a deep convolutional network for image super-resolution. In: Proc. European Conference on Computer Vision (ECCV), vol. 8692 of Lecture Notes in Computer Science, p. 184-199. Springer, Zurich, Switzerland, 6-12 Sep 2014. doi:10.1007/978-3-319-10593-2_13.
• [8] C. Dong, C. C. Loy, and X. Tang. Accelerating the super-resolution convolutional neural net-work. In: Proc. European Conference on Computer Vision (ECCV), vol. 9906 of Lecture Notes in Computer Science, p. 391-407. Springer, Amsterdam, The Netherlands, 2016. doi:10.1007/978-3-319-46475-6_25.
• [9] H. Dong, A. Supratak, L. Mai, F. Liu, A. Oehmichen, et al. TensorLayer: A versatile library for efficient deep learning development. In: Proc. 25th ACM International Conference on Multimedia, MM ’17, p. 1201-1204. Association for Computing Machinery, 2017. doi:10.1145/3123266.3129391.
• [10] L. A. Gatys, A. S. Ecker, and M. Bethge. Texture synthesis using convolutional neural net-works. In: Advances in Neural Information Processing Systems 28 - Proc. NIPS 2015, vol. 28 of NeurIPS Proceedings, pp. 262-270, 2015. https://proceedings.neurips.cc/paper/2015/hash/a5e00132373a7031000fd987a3c9f87b-Abstract.html.
• [11] I. Goodfellow, J. Pouget-Abadie, M. Mirza, B. Xu, D. Warde-Farley, et al. Generative adversarial nets. In: Advances in Neural Information Processing Systems 27 - Proc. NIPS 2014, vol. 27 of NeurIPS Proceedings, pp. 2672-2680, 2014. https://proceedings.neurips.cc/paper/2015/hash/a5e00132373a7031000fd987a3c9f87b-Abstract.html.
• [12] D. N. Greve and B. Fischl. Accurate and robust brain image alignment using boundary-based registration. NeuroImage, 48(1):63-72, 2009. doi:10.1016/j.neuroimage.2009.06.060.
• [13] D. M. Groppe, S. Bickel, A. R. Dykstra, X. Wang, P. Mégevand, et al. iELVis: An open source MATLAB toolbox for localizing and visualizing human intracranial electrode data. Journal of Neuroscience Methods, 281:40-48, 2017. doi:10.1016/j.jneumeth.2017.01.022.
• [14] J. Gu, H. Lu, W. Zuo, and C. Dong. Blind super-resolution with iterative kernel correction. In: Proc. 2019 IEEE/CVF Conf. Computer Vision and Pattern Recognition (CVPR), p. 1604-1613. Long Beach, CA, USA, 2019. doi:10.1109/CVPR.2019.00170.
• [15] M. P. Heinrich, M. Jenkinson, M. Bhushan, T. Matin, F. V. Gleeson, et al. MIND: Modality independent neighbourhood descriptor for multi-modal deformable registration. Medical Image Analysis, 16(7):1423-1435, 2012. Special Issue on the 2011 Conference on Medical Image Computing and Computer Assisted Intervention. doi:10.1016/j.media.2012.05.008.
• [16] X. Hu, H. Mu, X. Zhang, Z. Wang, T. Tan, et al. Meta-SR: A magnification-arbitrary network for super-resolution. In: Proc. 2019 IEEE/CVF Conf. Computer Vision and Pattern Recognition (CVPR), p. 1575-1584. Long Beach, CA, USA, 15-20 Jun 2019. doi:10.1109/CVPR.2019.00167.
• [17] G. Huang, Z. Liu, L. van der Maaten, and K. Q. Weinberger. Densely Connected Convolutional Networks. In: Proc. 2017 IEEE Conf. Computer Vision and Pattern Recognition (CVPR), p. 4700-4708. Honolulu, HI, USA, 21-26 Jul 2017. doi:10.1109/CVPR.2017.243.
• [18] C. M. Hyun, H. P. Kim, S. M. Lee, S. Lee, and J. K. Seo. Deep learning for under sampled MRI reconstruction. Physics in Medicine & Biology, 63(13):135007, 2018. doi:10.1088/1361-6560/aac71a.
• [19] P. Isola, J. Y. Zhu, T. Zhou, and A. A. Efros. Image-to-image translation with conditional adversarial networks. In: Proc. IEEE Conf. Computer Vision and Pattern Recognition (CVPR), pp. 5967-5976. Honolulu, HI, USA, 2017. doi:10.1109/CVPR.2017.632.
• [20] M. Jenkinson, P. Bannister, M. Brady, and S. Smith. Improved optimization for the robust and accurate linear registration and motion correction of brain images. NeuroImage, 17(2):825-841, 2002. doi:10.1006/nimg.2002.1132.
• [21] K. Jiang, Z. Wang, P. Yi, G. Wang, T. Lu, et al. Edge-enhanced GAN for remote sensing image super-resolution. IEEE Transactions on Geoscience and Remote Sensing, 57(8):5799-5812, 2019. doi:10.1109/TGRS.2019.2902431.
• [22] J. Johnson, A. Alahi, and F. F. Li. Perceptual losses for real-time style transfer and super-resolution. In: Proc. Eur. Conf. Computer Vision (ECCV), vol. 9906 of Lecture Notes in Computer Science, pp. 694-711. Springer, 2016. doi:10.1007/978-3-319-46475-6_43.
• [23] C. M. Kadipasaoglu, C. Morse, K. Pham, C. Donos, and N. Tandon. SAMCOR: A robust and precise coregistration algorithm for brain CT and MR imaging. Interdisciplinary Neurosurgery, 30:101637, 2022. doi:10.1016/j.inat.2022.101637.
• [24] K. Kataoka, Y. Shiraishi, Y. Takeda, S. Sakata, M. Matsumoto, et al. Aberrant PD-L1 expression through 3’-UTR disruption in multiple cancers. Nature, 534(7607):402-406, 2016. doi:10.1038/nature18294.
• [25] J. Kim, J. K. Lee, and K. M. Lee. Deeply-recursive convolutional network for image super-resolution. In: Proc. 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), p. 1637-1645. Las Vegas, NV, USA, 27 Jun 2016. doi:10.1109/CVPR.2016.181.
• [26] K. Kulkarni, S. Lohit, P. Turaga, R. Kerviche, and A. Ashok. ReconNet: Non-iterative reconstruction of images from compressively sensed measurements. In: Proc. IEEE Conference on Computer Vision and Pattern Recognition (CVPR), p. 449-458. Las Vegas, NV, USA, 27 Jun 2016. doi:10.1109/CVPR.2016.55.
• [27] C. Ledig, L. Theis, F. Huszár, J. Caballero, A. Cunningham, et al. Photo-realistic single image super-resolution using a generative adversarial network. In: Proc. 2017 IEEE Conference on Computer Vision and Pattern Recognition, pp. 105-114. Honolulu, HI, USA, 2017. doi:10.1109/CVPR.2017.19.
• [28] K. A. Lidke, B. Rieger, T. M. Jovin, and R. Heintzmann. Super-resolution by localization of quantum dots using blinking statistics. Optics Express, 13(18):7052-7062, 2005. doi:10.1364/OPEX.13.007052.
• [29] B. Lim, S. Son, H. Kim, S. Nah, and K. M. Lee. Enhanced deep residual networks for single image super-resolution. In: Proc. 2017 IEEE Conf. Computer Vision and Pattern Recognition Workshops (CVPRW), p. 1132-1140. Honolulu, HI, USA, 21-26 Jul 2017. doi:10.1109/CVPRW.2017.151.
• [30] C. Liu and D. Sun. On Bayesian adaptive video super resolution. IEEE Transactions on Pattern Analysis and Machine Intelligence, 36(2):346-360, 2014. doi:10.1109/TPAMI.2013.127.
• [31] D. Liu, Z. Wang, Y. Fan, X. Liu, Z. Wang, et al. Robust video super-resolution with learned temporal dynamics. In: Proc. 2017 IEEE Int. Conf. Computer Vision (ICCV), pp. 2526-2534. Venice, Italy, 22-29 Oct 2017. doi:10.1109/ICCV.2017.274.
• 32] D. Mahapatra, B. Bozorgtabar, and R. Garnavi. Image super-resolution using progressive generative adversarial networks for medical image analysis. Computerized Medical Imaging and Graphics, 71:30-39, 2019. doi:j.compmedimag.2018.10.005.
• [33] K. Malczewski. Image resolution enhancement of highly compressively sensed CT/PET signals. Algorithms, 13(5), 2020. doi:10.3390/a13050129.
• [34] K. Malczewski. Super-resolution with compressively sensed MR/PET signals at its input. Informatics in Medicine Unlocked, 18, 2020. doi:10.1016/j.imu.2020.100302.
• [35] K. Malczewski. Diffusion weighted imaging super-resolution algorithm for highly sparse raw data sequences. Sensors, 23(12):5698, 2023. doi:10.3390/s23125698.
• [36] K. Malczewski and R. Stasinski. High resolution MRI image reconstruction from a PROPELLER data set of samples. International Journal of Functional Informatics and Personalised Medicine, 1(3), 2008. doi:10.1504/IJFIPM.2008.021394.
• [37] A. Mousavi, A. B. Patel, and R. G. Baraniuk. A deep learning approach to structured signal recovery. In: Proc. 2015 53rd Annual Allerton Conference on Communication, Control, and Computing (Allerton). Monticello, IL, USA, 29 Sep - 02 Oct 2015.
• [38] K. Ning, Z. Zhang, K. Han, S. Han, and X. Zhang. Multi-frame super-resolution algorithm based on a WGAN. IEEE Access, 9:85839-85851, 2021. doi:10.1109/ACCESS.2021.3088128.
• [39] S. J. Pan and Q. Yang. A survey on transfer learning. IEEE Transactions on Knowledge and Data Engineering, 22(10):1345-1359, 2010. doi:10.1109/TKDE.2009.191.
• [40] H. Pedersen, S. Kozerke, S. Ringgaard, et al. k-t PCA: temporally constrained k-t BLAST reconstruction using principal component analysis. Magnetic Resonance in Medicine, 62(3):706-716, 2009. doi:10.1002/mrm.22052.
• [41] G. Qian, Y. Wang, J. Gu, C. Dong, W. Heidrich, et al. Rethinking learning-based demosaicing, denoising, and super-resolution pipeline. In: Proc. 2022 IEEE Int. Conf. Computational Photography (ICCP), pp. 1-12. Pasadena, CA, USA, 1-5 Aug 2022. doi:10.1109/ICCP54855.2022.9887682.
• [42] W. Shi, J. Caballero, F. Huszar, J. Totz, A. P. Aitken, et al. Real-time single image and video super-resolution using an efficient sub-pixel convolutional neural network. In: Proc. 2016 IEEE Conf. Computer Vision and Pattern Recognition (CVPR), p. 1874-1883. Las Vegas, NV, USA, 27-30 Jun 2016. doi:10.1109/CVPR.2016.207.
• [43] K. Simonyan and A. Zisserman. Very deep convolutional networks for large-scale image recognition. In: Proc. Int. Conf. Learning Representation (ICLR), 2015. Proceedings published in arXiv, see https://iclr.cc/archive/www/doku.php%3Fid=iclr2015:accepted-main.html. doi:10.48550/arXiv.1409.1556.
• [44] Y. Tai, J. Yang, and X. Liu. Image super-resolution via deep recursive residual network. Proc. 2017 IEEE Conf. Computer Vision and Pattern Recognition (CVPR), pp. 2790-2798, 21-26 Jul 2017. doi:10.1109/CVPR.2017.298.
• [45] C. Wachinger and N. Navab. Entropy and Laplacian images: Structural representations for multimodal registration. Medical Image Analysis, 16(1):1-17, 2012. doi:10.1016/j.media.2011.03.001.
• [46] A. W. A. Wahab, M. A. Bagiwa, M. Y. I. Idris, S. Khan, Z. Razak, et al. Passive video forgery detection techniques: A survey. In: Proc. 2014 10th Int. Conf. Information Assurance and Security (IAS), pp. 29-34. Okinawa, Japan, 28-30 Nov 2014. doi:10.1109/ISIAS.2014.7064616.
• [47] F. Yang, M. Ding, X. Zhang, W. Hou, and C. Zhong. Non-rigid multi-modal medical image registration by combining L-BFGS-B with cat swarm optimization. Information Sciences, 316:440-456, 2015. doi:10.1016/j.ins.2014.10.051.
• [48] Z. Zhang, D. Gao, X. Xie, and G. Shi. Dual-channel reconstruction network for image compressive sensing. Sensors, 19:2549, 2019. doi:10.3390/s19112549.
• [49] Z. Zhang, Z. Wang, Z. Lin, and H. Qi. Image super-resolution by neural texture transfer. In: Proc. 2019 IEEE/CVF Conf. Computer Vision and Pattern Recognition (CVPR), p. 7974-7983. Long Beach, CA, USA, 15-20 Jun 2019. doi:10.1109/CVPR.2019.00817.