Augmented reality visualization for aiding biopsy procedure according to computed tomography based virtual plan

• Autorzy: Majak Marcin , Żuk Magdalena , Świątek-Najwer Ewelina , Popek Michał , Pietruski Piotr

Identyfikatory

DOI

10.37190/ABB-01811-2021-02

• Języki publikacji: EN

Abstrakty

EN

Purpose: The purpose of this study was to develop and verify an intraoperative module for supporting navigated biopsy procedures using optical see-through head-mounted display (HMD). Methods: Biopsy procedure including entry and endpoints of needle insertion was planned preoperatively having regard to the resection region segmentation and safety margin definition. Biopsy procedures were performed by two users using an intraoperative optical navigation module on a specially prepared brain phantom. Two visualization techniques were compared: an accurate augmented reality one, where a virtual plan is superimposed onto surgical field by using optical see-through HMD together with personalized calibration method and visualization on the external display. Results: Averaged errors from 24 trials using external display were 2.04 ± 0.83 mm for the first user and 2.69 ± 1.11 mm for the second one, while applying HMD 2.50 ± 0.93 mm (the first user) and 2.17 ± 0.82 mm (the second user), respectively. Conclusions: Proper usage of HMD visualization preceded by the personalized calibration allows the user to perform navigated biopsy procedure with comparable accuracy to its equivalent with the external display. Additionally, augmented reality visualization improves ergonomics and enables focusing on the surgical field without losing a direct line of sight with the field of view as it happens for external displays. However, ensuring high accuracy of augmented reality visualization still requires proper calibration and some user experience, which is challenging.

Słowa kluczowe

EN

biopsy; augmented reality; computer aided surgery; head mounted display; computer navigation

PL

biopsja; rzeczywistość rozszerzona; chirurgia wspomagana komputerowo; wyświetlacz nagłowny; nawigacja komputerowa

• Wydawca: Oficyna Wydawnicza Politechniki Wrocławskiej
• Czasopismo: Acta of Bioengineering and Biomechanics
• Rocznik: 2021
• Tom: Vol. 23, no. 2
• Strony: 81--89
• Opis fizyczny: Bibliogr. 30 poz., rys., tab.

Twórcy

autor

Majak Marcin

• Wrocław University of Science and Technology, Faculty of Mechanical Engineering, Wrocław, Poland

autor

Żuk Magdalena

• Wrocław University of Science and Technology, Faculty of Mechanical Engineering, Wrocław, Poland

autor

Świątek-Najwer Ewelina

• Wrocław University of Science and Technology, Faculty of Mechanical Engineering, Wrocław, Poland

autor

Popek Michał

• Wrocław University of Science and Technology, Faculty of Mechanical Engineering, Wrocław, Poland

autor

Pietruski Piotr

• Department of Plastic Surgery, Prof. W. Orlowski Memorial Hospital, Medical Centre of Postgraduate Education, Warsaw, Poland

Bibliografia

• [1] ANDREWS C.M., HENRY A.B., SORIANO I.M., SOUTHWORTH M.K., SILVA J.R., Registration Techniques for Clinical Applications of Three-Dimensional Augmented Reality Devices, IEEE Journal of Translational Engineering in Health and Medicine, 2020, 9, DOI: https://doi.org/10.1109/JTEHM.2020.3045642.
• [2] BADIALI G., CUTOLO F., CERCENELLI L., M. CARBONE M., D’AMATO R., FERRARI V., MARCHETTI C., The VOSTARS project: A new wearable hybrid video and optical see-through augmented reality surgical system for maxillofacial surgery, Int. J. Maxillofacial Surg., 2019, 48, DOI: https://doi.org/10.1016/ j.ijom.2019.03.472.
• [3] BADIALI G. et al., Review on Augmented Reality in Oral and Cranio-Maxillofacial Surgery: Toward “Surgery-Specific” Head-Up Displays, IEEE Access, 2020, 8, 59015–59028, DOI: 10.1109/ACCESS.2020.2973298.
• [4] BOSC R., FITOUSSI A., HERSANT B., DAO T.H., MENINGAUD J.P, Intraoperative augmented reality with heads-up displays in maxillofacial surgery: A systematic review of the literature and a classification of relevant technologies, Int. J. Oral Maxillofacial Surg., 2019, 48 (1), 132–139, DOI: 10.1016/ j.ijom.2018.09.010.
• [5] CARSE B., MEADOWS B., BOWERS R., ROWE P., Affordable clinical gait analysis: An assessment of the marker tracking accuracy of a new low-cost optical 3d motion analysis system, Physiotherapy, 2013, 99 (4), 347–351.
• [6] CHEN X., XU L., WANG Y., WANG H., WANG F., ZENG X., WANG Q., EGGER J., Development of a surgical navigation system based on augmented reality using an optical see-through headmounted display, Journal of Biomedical Informatics, 2015, 55, DOI: https://doi.org/10.1016/j.jbi.2015.04.003.
• [7] CUTOLO F. et al., Ambiguity-Free Optical–Inertial Tracking for Augmented Reality Headsets, Sensors, 2020, 20 (5), DOI: 10.3390/s20051444.
• [8] DE AMICI S., SANNA A., LAMBERTI F., PRALIO B., A wii remote-based infrared-optical tracking system, Entertainment Computing, 2010, 1 (3–4), 119.
• [9] GARRIDO-JURADO S., MUNOZ-SALINAS R., MADRID-CUEVAS F., MARIN-JIMENEZ M., Automatic generation and detection of highly reliable fiducial markers under occlusion, Pattern Recognition, 2014, 47 (6), 2280–2292.
• [10] GARCÍA-VÁZQUEZ V., VON HAXTHAUSEN F., JÄCKLE S., SCHUMANN C., KUHLEMANN I., BOUCHAGIAR J., HÖFER A.C., MATYSIAK F., HÜTTMANN G., GOLTZ J.P., KLEEMANN M., ERNST F., HORN M., Navigation and visualization with HoloLens in endovascular aortic repair, Innov. Surg. Sci., 2018, 3 (3), 167–177, DOI: 10.1515/iss-2018-2001.
• [11] GIL J.J., DÍAZ I., ACCINI F., Inferring material properties in robotic bone drilling processes, Acta Bioeng. Biomech., 2019, 21 (3), 109–118, DOI: 10.5277/ABB-01386-2019-02.
• [12] GRUBERT J., ITOH Y., MOSER K., SWAN J.E., A Survey of Calibration Methods for Optical See-Through Head-Mounted Displays, IEEE Transactions on Visualization and Computer Graphics, 2018, 24 (9), 2649–2662, DOI: 10.1109/ TVCG.2017.2754257.
• [13] ITOH Y., KLINKER G., Interaction-free calibration for optical seethrough head-mounted displays based on 3D eye localization, Proc. IEEE Symp. 3D User Interfaces, 2014, 75–82.
• [14] KUNZ C., GENTEN V., MEIßNER P., HEIN B., Metric-based evaluation of fiducial markers for medical procedures, Proc. SPIE 10951, Medical Imaging 2019: Image-Guided Procedures, Robotic Interventions, and Modeling, 109512O, 2019, DOI: https://doi.org/10.1117/12.2511720.
• [15] LIN M.A., SIU A.F., BAE J.H., CUTKOSKY M.R., DANIEL B.L., HoloNeedle: Augmented reality guidance system for needle placement investigating the advantages of three-dimensional needle shape reconstruction, IEEE Robot. Autom. Lett., 2018, 3 (4), 4156–4162, DOI: 10.1109/LRA.2018.2863381
• [16] MAJAK M., ŻUK M., ŚWIĄTEK-NAJWER E., POPEK M., PIETRUSKI P., Biopsy procedure applied in MentorEye molecular surgical navigation system, Lecture Notes in Computational Vision and Biomechanics, 2018, 27, 338–344.
• [17] MAKIBUCHI N., KATO H., YONEYAMA A., Vision-based robust calibration for optical see-through head-mounted displays, Proc. IEEE Int. Conf. Image Process., 2013, 2177–2181.
• [18] MCKNIGHT R.R., PEAN C.A., BUCK J.S. et al., Virtual Reality and Augmented Reality – Translating Surgical Training into Surgical Technique, Curr. Rev. Musculoskelet. Med., 2020, 13, 663–674, https://doi.org/10.1007/s12178-020-09667-3
• [19] DE OLIVEIRA M.E., DEBARBA H.G., LÄDERMANN A., CHAGUÉ S., CHARBONNIER C., A hand-eye calibration method for augmented reality applied to computer-assisted orthopedic surgery, Int. J. Med. Robot., 2019, 15 (2), DOI: 10.1002/rcs.1969.
• [20] PIETRUSKI P., MAJAK M., ŚWIĄTEK-NAJWER E., ŻUK M., POPEK M., MAZUREK M., ŚWIECKA M., JAWOROWSKI J., Navigation-guided fibula free flap for mandibular reconstruction: A proof of concept study, Journal of Plastic, Reconstructive and Aesthetic Surgery, 2019, 72 (4), DOI: 10.1016/j.bjps.2019.01.026.
• [21] PIETRUSKI P., MAJAK M., ŚWIĄTEK-NAJWER E., ŻUK M., POPEK M., JAWOROWSKI J., MAZUREK M., Supporting fibula free flap harvest with augmented reality: A proof-of-concept study, The Laryngoscope, 2019, 130 (5), 1173–1179, https:// doi.org/10.1002/lary.28090
• [22] PIETRUSKI P. et. al., Supporting mandibular resection with intraoperative navigation utilizing augmented reality technology– A proof of concept study, Journal of Cranio-Maxillofacial Surgery, 2019, 47 (6), DOI: https://doi.org/10.1016/ j.jcms.2019.03.004.
• [23] QIAN L., BARTHEL A., JOHNSON A., OSGOOD G., KAZANZIDES P., NAVAB N., FUERST B., Comparison of optical see-through head-mounted displays for surgical interventions with objectanchored 2D-display, Int. J. Comput. Assist. Radiol. Surg., 2017, 12 (6), DOI: 10.1007/s11548-017-1564-y.
• [24] RAHMAN R., WOOD M.E., QIAN L., PRICE C.L., JOHNSON A.A., OSGOOD G.M., Head-Mounted Display Use in Surgery: A Systematic Review, Surgical Innovation, 2020, 27 (1), 88–100, DOI: 10.1177/1553350619871787.
• [25] SAKAI D., JOYCE K., SUGIMOTO M. et al., Augmented, virtual and mixed reality in spinal surgery: A real-world experience, Journal of Orthopaedic Surgery, 2020, DOI: 10.1177/ 2309499020952698.
• [26] TUCERYAN M., GENC Y., NAVAB N., Single-Point Active Alignment Method (SPAAM) for Optical See-Through HMD Calibration for Augmented Reality, Teleoperators and Virtual Environments, 2002, 11, 259–276.
• [27] WACKER F.K., VOGT S.K., KHAMENE A., JESBERGER J.A., NOUR S.G., ELGORT D.R., SAUER F., DUERK J.L., LEWIN J.S., An augmented reality system for MR image-guided needle biopsy: initial results in a swine model, Radiology, 2006, 238 (2), 497–504.
• [28] WANG H., WANG F., XU L., CHEN X., WANG Q., Precision insertion of percutaneous sacroiliac screws using a novel augmented reality-based navigation system: a pilot study, International Orthopaedics, 2016, 40, 1941–1947.
• [29] WANG J., SHEN Y., YANG S., A practical marker-less image registration method for augmented reality oral and maxillofacial surgery, Int. J. Comput. Assist. Radiol. Surg., 2019, 14 (5), 763–773, DOI: 10.1007/s11548-019-01921-5.
• [30] ŻUK M., MAJAK M., ŚWIĄTEK-NAJWER E., POPEK M., KULAS Z., Evaluation of calibration procedure for stereoscopic visualization using optical See-Through Head Mounted Displays for a complex oncological treatment, Lecture Notes in Computational Vision and Biomechanics, 2018, 27, 354–359.

Uwagi

Opracowanie rekordu ze środków MNiSW, umowa nr SONP/SP/546092/2022 w ramach programu "Społeczna odpowiedzialność nauki" - moduł: Popularyzacja nauki i promocja sportu (2024).