Overview of the holographic-guided cardiovascular interventions and training - a perspective

Proniewska, Klaudia; Pregowska, Agnieszka; Walecki, Piotr; Dołęga-Dołęgowski, Damian; Ferrari, Roberto; Dudek, Dariusz

doi:10.1515/bams-2020-0043

Artykuł - szczegóły

Tytuł artykułu

Overview of the holographic-guided cardiovascular interventions and training - a perspective

Autorzy

Proniewska Klaudia , Pregowska Agnieszka , Walecki Piotr , Dołęga-Dołęgowski Damian , Ferrari Roberto , Dudek Dariusz

Wybrane pełne teksty z tego czasopisma

Identyfikatory

DOI

10.1515/bams-2020-0043

Warianty tytułu

Języki publikacji

Abstrakty

Immersive technologies, like Virtual Reality (VR), Augmented Reality (AR) and Mixed Reality (MR) have undergone technical evolutions over the last few decades. Their rapid development and dynamic changes enable their effective applications in medicine, in fields like imaging, preprocedural planning, treatment, operations planning, medical students training, and active support during therapeutic and rehabilitation procedures. Within this paper, a comprehensive analysis of VR/ AR/MR application in the medical industry and education is presented. We overview and discuss our previous experience with AR/MR and 3D visual environment and MR-based imaging systems in cardiology and interventional cardiology. Our research shows that using immersive technologies users can not only visualize the heart and its structure but also obtain quantitative feedback on their location. The MR-based imaging system proposed offers better visualization to interventionists and potentially helps users understand complex operational cases. The results obtained suggest that technology using VR/AR/MR can be successfully used in the teaching process of future doctors, both in aspects related to anatomy and clinical classes. Moreover, the system proposed provides a unique opportunity to break the boundaries, interact in the learning process, and exchange experiences inside the medical community.

Słowa kluczowe

augmented reality improving education process interaction intraprocedural visualization mixed reality preprocedural planning teaching

Wydawca

Uniwersytet Jagielloński - Collegium Medicum
De Gruyter

Czasopismo

Bio-Algorithms and Med-Systems

Rocznik

2020

Tom

Vol. 16, no. 3

Strony

art. no. 20200043

Opis fizyczny

Bibliogr. 65 poz., rys.

Twórcy

autor

Proniewska Klaudia

klaudia.proniewska@uj.edu.pl

Department of Bioinformatics and Telemedicine, Jagiellonian University Medical College, Lazarza 16, 31-530, Krakow, Poland. Phone: +48 12 619 96 83, Fax: +48 12 619 96 85

autor

Pregowska Agnieszka

Institute of Fundamental Technological Research, Polish Academy of Sciences, Warsaw, Poland

autor

Walecki Piotr

Department of Bioinformatics and Telemedicine, Jagiellonian University Medical College, Krakow, Poland

autor

Dołęga-Dołęgowski Damian

Department of Bioinformatics and Telemedicine, Jagiellonian University Medical College, Krakow, Poland

autor

Ferrari Roberto

Maria Cecilia Hospital, GVM Care & Research, Cotignola, Italy
Department of Medical Sciences, Ferrara University, Ferrara, Italy

autor

Dudek Dariusz

Maria Cecilia Hospital, GVM Care & Research, Cotignola, Italy
Jagiellonian University Medical College, Institute of Cardiology, Krakow, Poland

Bibliografia

1. De Paolis LT, Ricciardi F. Augmented visualisation in the treatment of the liver tumours with radiofrequency ablation. Comput Method Biomech Biomed Eng: Imag Vis 2018;6:396-404.
2. Eckert M, Volmerg JS, Friedrich CM. Augmented reality in medicine: systematic and bibliographic review. JMIR mHealth uHealth 2019; 7:e10967.
3. Minsky M. Telepresence. Omni 1980;45-52.
4. Sanchez-Vives MV, Slater M, From presence to consciousness through virtual reality. Nat Rev Neurosci 2005;6:32-9.
5. Slater M, Neyret S, Johnston T, Iruretagoyenal G, Álvarez de la Campa Crespo M, et al. An experimental study of a virtual reality counseling paradigm using embodied selfdialogue, Sci Rep 2019;9:1090321.
6. Azuma, RT A survey of augmented reality. Presence Teleop Virt Environ 1997;6:355-85.
7. Azuma RT, Baillot Y, Behringer R, Feiner S, Julier S, MacIntyre B. Recent advances in augmented reality, IEEE Comput Graph App 2001;21:34-47.
8. Rokhsaritalemi S, Sadeghi-Niaraki A, Choi SM. A review on mixed reality: current trends,challenges and prospects.ApplSci 2020;10:636.
9. Rosenberg B The use of virtual fixtures as perceptual overlays to enhance operator performance in remote environments. Technical Report AL-TR-0089. Wright-Patterson AFB OH: USAF Armstrong Laboratory; 1992.
10. Hönig W, Milanes C, Scara L, Phan T, Bolas M, Ayanian N. Mixed reality for robotics. In: 015 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS). Hamburg; 2015:5382-7 pp. In this issue.
11. Bin S, Masood S, Jung Y. Chapter Twenty - virtual and augmented reality in medicine. In: Dagan Feng D, editor. Biomedical Engineering, Academic Press Elsevier; 2020, pp. 673-86.
12. Szabó Z, Berg S, Sjökvist S, Gustafsson T, Carleberg P, Uppsäll M, et al. Real-time intraoperative visualization of myocardial circulation using augmented reality temperature display. Int J Cardiovasc Imag 2013;29:521-8.
13. Zou YB, Chen YM, Gao MK, Liu Q, Jiang SY, Lu JH, et al. Coronary heart disease preoperative gesture interactive diagnostic system based on augmented reality. J Med Syst 2017;41:126.
14. Hemanth JD, Kose U, Deperlioglu O, de Albuquerque VHC. An augmented reality-supported mobile application for diagnosis of heart diseases. J Supercomput 2020; 76:1242-67.
15. Chu MW, Moore J, Peters T, Bainbridge D, McCarty D, Guiraudon GM, et al. Augmented reality image guidance improves navigation for beating heart mitral valve repair. Innovations (Phila) 2012;7:274-81.
16. De Paolis LT, An augmented reality platform for preoperative surgical planning. J Interdiscip Res App Med 2019;3:19-24.
17. Oculus Rift. Available from: https://www.oculus.com/rift/?locale=pl_PL [Accessed 31 July 2020].
18. Botella C, Breton-Lopez J, Quero S, Banos RM, Garcia-Palacios A, Zaragoza I, et al. Treating cockroach phobia using a serious game on a mobile phone and augmented reality exposure: a single case study. Comput Hum Behav 2011;27:217-227.
19. Liao T, Humphreys L. Layar-ed places: using mobile augmented reality to tactically reengage, reproduce, and reappropriate public space. New Media Soc 2015:17:1418-1435.
20. Gerup J, Soerensen CB, Dieckmann P. Augmented reality and mixed reality for healthcare education beyond surgery: an integrative review. Int J Med Educ. 2020:11:1-18.
21. Appleby R. Desire in translation: white masculinity and TESOL. TESOL 2013;47:122-47.
22. Birt J, Stromberga Z, Cowling M, Moro C. Mobile mixed reality for experiential learning and simulation in medical and health sciences education, Information 2018:9: 2-14.
23. Zhu E, Hadadgar A, Masiello I, Zary N Augmented reality in healthcare education: an integrative review. PeerJ 2014;2:e469.
24. Microsoft HoloLens. Available from: https://www.microsoft.com/en-us/ [Accessed 31 July 2020].
25. Silva JNA, Southworth M, Raptis C, Silva J. Emerging applications of virtual reality in cardiovascular medicine. J Am Coll Cardiol 2018;3:420-30.
26. Liao T, Chang PF, Lee SY. Augmented reality in health and medicine: a review of augmented reality application for health professionals, procedures, and behavioral interventions. In: Kim J, Song H, editor. Technology and health promoting attitude and behavior change. Academic Press, Elsevier; 2020, pp. 109-28.
27. https://www.microsoft.com/en-us/hololens.
28. Opolski MP, Debski A, Borucki BA, Staruch AD, Kepka C, Rokicki JK, et al. Feasibility and safety of augmented-reality glass for computed tomography-assisted percutaneous revascularization of coronary chronic total occlusion: a single center prospective pilot study. J Cardiovasc Comput Tomograph 2017;11:489-96.
29. Belhaj Soulami R, Verhoye JP, Nguyen Duc H, Castro M, Auffret V, Anselmi A, et al. Computer-assisted transcatheter heart valve implantation in valve-in-valve procedures. Innovations (Phila) 2016;11:193-200.
30. Jang J, Tschabrunn CM, Barkagan M, Anter E, Menze B, Nezafat R. Three-dimensional holographic visualization of high-resolution myocardial scar on HoloLens. PLoS One 2018;13:e0205188.
31. Tsai MK, Liu PHE, Yau NJ. Using electronic maps and augmented reality-based training materials as escape guidelines for nuclear accidents: an explorative case study in Taiwan. Br J Educ Technol 2013;44:E18-21.
32. Rothbaum BO, Hodges LF, Kooper R, Opdyke D, Williford JS, North M. Effectiveness of computer-generated (Virtual-Reality) graded exposure in the treatment of acrophobia. Am J Psychiatr 1995;152: 626-8.
33. Riva G, Wiederhold BK, Mantovani F. Neuroscience of virtual reality: from virtual exposure to embodied medicine. Cyberpsychol Behav Soc Netw 2019;22:82-96.
34. Mendez A, Hussain T, Hosseinpour AR, Valverde I. Virtual reality for preoperative planning in large ventricular septal defects. Eur Heart J 2019;40:1092.
35. Knecht S, Brantner P, Cattin P, Tobler D, Kühne M, Sticherling C. State-of-the-art multimodality approach to assist ablations in complex anatomies-From 3D printing to virtual reality. Pacing Clin Electrophysiol 2019;42:101-3.
36. Ong CS, Krishnan A, Huang CY, Spevak P, Vricella L, Hibino N, et al. Role of virtual reality in congenital heart disease. Congenit Heart Dis 2018;13:357-61.
37. Sinkin JC, Rahman OF, Nahabedian MY. Google Glass in the operating room: the plastic surgeon’s perspective. Plast Reconstr Surg 2016;138:298-302.
38. Brewer ZE, Fann HC, Ogden DW, Burdon TA, Sheikh AY. Inheriting the learner’s view: a Google Glass-based wearable computing platform for improving surgical trainee performance, J Surg Educ 2016;73:682-8.
39. Tepper OM, Rudy HL, Lefkowitz A, Weimer KA, Marks SM, Stern CS, et al. Mixed reality with HoloLens: where virtual reality meets augmented reality in the operating room. Plast Reconstr Surg 2018;140:1066-70.
40. NM. Available from: https://www.nextmotion.net/.
41. Holohuman. Available from: https://3d4medical.com/apps/ holohuma [Accessed 31 July 2020].
42. HTC VIVE Pro. Available from https://www.vive.com/us/product/vive-virtual-reality-system/ [Accessed 31 July 2020].
43. Mcduff D, Hurter C, Gonzalez-Franco M. Pulse and vital sign measurement in mixed reality using a HoloLens. In: 23rd ACM symposium on virtual reality software and technology (VRST’17). Göteborg, Sweden; 2017, vol 34.
44. Hanna MG, Ahmed I, Nine J, Prajapati S, Pantanowitz L. Augmented reality technology using Microsoft HoloLens in anatomic pathology. Arch Pathol Lab Med 2018;142:638-44.
45. Digitaltrends. Available from: https://www.digitaltrends.com/virtual-reality/hololens-holoanatomy-award-jackson-holescience-media-awards/ [Accessed 31 July 2020].
46. Ruthberg JS, Quereshy HA, Ahmadmehrabi S, Trudeau S, Chaudry E, Hair B, et al. A multimodal multi-institutional solution to remote medical student education for otolaryngology during COVID-19. Otolaryngol-Head and Neck Surgery 2020;1-3.
47. Holoanatomy. Available from: https://www.microsoft.com/enus/p/holoanatomy/9nblggh4ntd3?activetab=pivot:overviewtab [Accessed 31 July 2020].
48. Keenan ID, Ben Awadh A. Integrating 3D visualisation technologies in undergraduate anatomy education. Adv Exp Med Biol 2019;1120:39-53.
49. Henssen DJHA, van den Heuvel L, De Jong G, Vorstenbosch MATM, van Cappellen van Walsum AM, Van den Hurk MM, et al. Neuroanatomy learning: augmented reality vs. Cross-sections. Anat Sci Educ 2019;13:353-65.
50. Foronda CL, Alfes CM, Dev P, Kleinheksel A, Nelson DA, O’Donnell JM, et al. Virtually nursing: emerging technologies in nursing education. Nurse Educat 2017;42:14-17.
51. Hauze SW, Hoyt HH, Frazee JP, Greiner PA. Marshall JM. Enhancing nursing education through affordable and realistic holographic mixed reality: the virtual standardized patient for clinical simulation. Adv Exp Med Biol 2019;1120: 1-13.
52. Evans L, Taubert M. State of the science: the doll is dead: simulation in palliative care education. BMJ Support Palliat Care 2019;9:117-19.
53. Southworth MK, Silva JR, Avari Silva JN. Use of extended realities in cardiology, Trends Cardiovasc Med 2020;30:143-8.
54. Hospital SCsHLPCs. Available from: https://www.stanfordchildrens.org/en/about/news/releases/2017/virtualreality-program [Accessed 31 July 2020].
55. Hospital SCsHLPCs. Available from: https://www.stanfordchildrens.org/en/innovation/virtual-reality/anxietyresearch [Accessed 31 July 2020].
56. Windows Blogs. Available from: https://blogs.windows.com/windowsexperience/2018/03/08/how-mixed-reality-ischanging-the-game-for-healthcare-from-performing-livesurgeries-to-delivering-ultrasounds-in-3d/[Accessed 31 July 2020].
57. Mahmood F, Mahmood E, Dorfman RG, Mitchell J, Mahmood FU, Jones SB, Matyal R Augmented reality and ultrasound education: initial experience. J Cardiothorac Vasc Anesth 2018; 32:1363-7.
58. https://www.dicomdirector.com/.
59. Proniewska K, Dołęga-Dołęgowski D, Dudek D. A holographic doctors’ assistant on the example of a wireless heart rate monitor. Bio Algorithm Med Syst 2018;14:1-5. UNSP 20180007.
60. https://azure.microsoft.com/en-us/services/kinect-dk/.
61. MRTK. Available from: https://docs.microsoft.com/en-us/windows/mixed-reality/mrtk-getting-started [Accessed 31 July 2020].
62. Spatial Anchors. Available from: https://azure.microsoft.com/en-us/services/spatial-anchors/ [Accessed 31 July 2020].
63. https://www.dicomdirector.com/.
64. https://azure.microsoft.com/en-us/services/kinect-dk/.
65. https://www.microsoft.com/en-us/hololens.

Uwagi

Opracowanie rekordu ze środków MNiSW, umowa Nr 461252 w ramach programu "Społeczna odpowiedzialność nauki" - moduł: Popularyzacja nauki i promocja sportu (2021).

Typ dokumentu

Bibliografia

Identyfikator YADDA

bwmeta1.element.baztech-baf18aab-773a-4f8f-a3f8-de3e8839741b