Experience with augmented reality technology in the surgical treatment of a patient with encapsulated metal foreign bodies of lower extremities

Abstract

This paper presents the experience of using augmented reality (AR) technology in the surgical treatment of a patient with encapsulated foreign bodies in the lower extremities. The study aims to test a previously developed algorithm for implementing AR technology in surgery and assess its effectiveness in treating patients with encapsulated foreign bodies. The study was conducted by a multidisciplinary team, comprising the Department of Naval Surgery of the S.M. Kirov Military Medical Academy and the Technical University named after Peter the Great. The medical and technical aspects involved an AR hardware and software package, including a personal computer and Microsoft Hololens II AR glasses. An invasive fixation system was also developed, incorporating a threaded pin used as an X-ray contrast mark with a seat for attaching special markers. The clinical part of the study involved observing and subsequently removing encapsulated foreign bodies in a patient who received inpatient treatment at the clinic of the Department of Naval Surgery in October 2022. Overall, AR technology demonstrated the potential for performing minimally invasive removal of encapsulated foreign bodies in limbs. The detailed visualization provided by AR allowed for determining optimal operative access, the volume of the upcoming operation, and the exact position, skeletotopy, and syntopia of the foreign body during the preoperative stage. Consequently, various intervention options could be digitally simulated. The use of AR technology facilitated intraoperative navigation, enhancing the safety and efficiency of the operation. Thus, AR technology has proven to be a minimally invasive, safe, and effective tool in surgical treatment. However, several unresolved issues require further research.

Keywords

augmented reality / soft tissue foreign bodies / intraoperative navigation / digital modeling / multidisciplinary team / lower extremity gunshot wounds / preoperative planning / combined trauma

References

[1] Nechaev EhA, Gritsanov AI, Fomin NF, Minullin IP. Minno-vzryvnaya travma. Saint Petersburg; Al'd, 1994. 488 p. (In Russ.).

[2] Нечаев Э.А., Грицанов А.И., Фомин Н.Ф., Минуллин И.П. Минно-взрывная травма. Санкт-Петербург: Альд, 1994. 488 с.

[3] El'skii VN, Klimovitskii VG, Pasternak VN, et al. Kontseptsiya travmaticheskoi bolezni na sovremennom ehtape i aspekty prognozirovaniya ee iskhodov. Archives of clinical and experimental medicine. 2003;12(1):87–92. (In Russ.).

[4] Ельский В.Н., Климовицкий В.Г., Пастернак В.Н., и др. Концепция травматической болезни на современном этапе и аспекты прогнозирования ее исходов // Архив клинической и экспериментальной медицины. 2003. Т. 12, № 1. С. 87–92.

[5] Eckert M, Volmerg JS, Friedrich CM. Augmented reality in medicine: systematic and bibliographic review. JMIR Mhealth Uhealth. 2019;7(4):e10967. DOI: 10.2196/10967

[6] Eckert M., Volmerg J.S., Friedrich C.M. Augmented reality in medicine: systematic and bibliographic review // JMIR Mhealth Uhealth. 2019. Vol. 7, No. 4. ID e10967. DOI: 10.2196/10967

[7] González Izard S, Sánchez Torres R, Alonso Plaza Ó, et al. Nextmed: automatic imaging segmentation, 3D reconstruction, and 3D model visualization platform using augmented and virtual reality. Sensors (Basel). 2020;20(10):2962. DOI: 10.3390/s20102962

[8] González Izard S., Sánchez Torres R., Alonso Plaza Ó., et al. Nextmed: automatic imaging segmentation, 3D reconstruction, and 3D model visualization platform using augmented and virtual reality // Sensors (Basel). 2020. Vol. 20, No. 10. ID 2962. DOI: 10.3390/s20102962

[9] Ivanov VM, Krivtsov AM, Strelkov SV, et al. Practical application of augmented/mixed reality technologies in surgery of abdominal cancer patients. J Imaging. 2022;8(7):183. DOI: 10.3390/jimaging8070183

[10] Ivanov V.M., Krivtsov A.M., Strelkov S.V., et al. Practical application of augmented/mixed reality technologies in surgery of abdominal cancer patients // J Imaging. 2022. Vol. 8, No. 7. ID 183. DOI: 10.3390/jimaging8070183

[11] Kotiv BN, Budko IA, Ivanov IA, Trosko IU. Artificial intelligence using for medical diagnosis via implementation of expert systems. Bulletin of the Russian Military Medical Academy. 2021;23(1): 215–224. (In Russ.). DOI: 10.17816/brmma63657

[12] Котив Б.Н., Будько И.А., Иванов И.А., Тросько И.У. Использование искусственного интеллекта для медицинской диагностики с помощью реализации экспертной системы // Вестник Российской военно-медицинской академии. 2021. Т. 23, № 1. С. 215–224. DOI: 10.17816/brmma63657

[13] Thomas DJ. Augmented reality in surgery: The computer-aided medicine revolution. Int J Surg. 2016;36(A):25. DOI: 10.1016/j.ijsu.2016.10.003

[14] Thomas D.J. Augmented reality in surgery: The computer-aided medicine revolution // Int J Surg. 2016. Vol. 36, No. A. Р. 25. DOI: 10.1016/j.ijsu.2016.10.003

[15] Lysenko AV, Razumova AYa, Yaremenko AI, et al. Primary results of using augmented reality technology for various maxillofacial pathologies. Meditsinskii vestnik MVD. 2022;117(2):7–10. (In Russ.). DOI: 10.52341/20738080_2022_117_2_7

[16] Лысенко А.В., Разумова А.Я., Яременко А.И., и др. Первичные результаты применения технологии дополненной реальности при различных патологиях в челюстно-лицевой области // Медицинский вестник МВД. 2022. Т. 117, № 2. С. 7–10. DOI: 10.52341/20738080_2022_117_2_7

[17] Leuze C, Zoellner A, Schmidt AR, et al. Augmented reality visualization tool for the future of tactical combat casualty care. J Trauma Acute Care Surg. 2021;91(2S):S40–S45. DOI: 10.1097/TA.0000000000003263

[18] Leuze C., Zoellner A., Schmidt A.R., et al. Augmented reality visualization tool for the future of tactical combat casualty care // J Trauma Acute Care Surg. 2021. Vol. 91, No. 2S. Р. S40–S45. DOI: 10.1097/TA.0000000000003263

[19] Ivanov VM, Krivtsov AM, Strelkov SV, et al. Intraoperative use of mixed reality technology in median neck and branchial cyst excision. Future Internet. 2021;13(8):214. DOI: 10.3390/fi13080214

[20] Ivanov V.M., Krivtsov A.M., Strelkov S.V., et al. Intraoperative use of mixed reality technology in median neck and branchial cyst excision // Future Internet. 2021. Vol. 13, No. 8. ID 214. DOI: 10.3390/fi13080214

[21] Kasimov RR, Zavrazhnov AA, Zavrazhnov AI, et al. Clinical and epidemiological characteristics severe injuries in military personnel in peacetime. Emergency medical care. 2022;23(2):4–13. (In Russ.). DOI: 10.24884/2072-6716-2022-23-2-4-13

[22] Касимов Р.Р., Завражнов А.А., Махновский А.И., и др. Клинико-эпидемиологическая характеристика тяжелой травмы у военнослужащих в мирное время // Скорая медицинская помощь. 2022. Т. 23, № 2. С. 4–13. DOI: 10.24884/2072-6716-2022-23-2-4-13

[23] Zawy Alsofy S, Nakamura M, Suleiman A, et al. Cerebral anatomy detection and surgical planning in patients with anterior skull base meningiomas using a virtual reality technique. J Clin Med. 2021;10(4):681. DOI: 10.3390/jcm10040681

[24] Zawy Alsofy S., Nakamura M., Suleiman A., et al. Cerebral anatomy detection and surgical planning in patients with anterior skull base meningiomas using a virtual reality technique // J Clin Med. 2021. Vol. 10, No. 4. ID 681. DOI: 10.3390/jcm10040681

[25] Certificate of registration of the computer program RUS 2021668401, 15.11.2021. Appl. № 2021667400/29.10.2021. Agafonova MV, Gaivoronskii IV, Dorozhkin RV, et al. Programma vizualizatsii protsessov funktsional'noi anatomii tsentral'noi nervnoi sistemy v rezhimakh AR/VR. (In Russ.).

[26] Свидетельство о регистрации программы для ЭВМ 2021668401, 15.11.2021. Заявка № 2021667400/29.10.2021. Агафонова М.В., Гайворонский И.В., Дорожкин Р.В., и др. Программа визуализации процессов функциональной анатомии центральной нервной системы в режимах AR/VR.

[27] Lysenko A, Razumova A, Yaremenko A, et al. The first clinical use of augmented reality to treat salivary stones. Med Case Rep Dent. 2020;5960421. DOI: 10.1155/2020/5960421

[28] Lysenko A., Razumova A., Yaremenko A., et al. The first clinical use of augmented reality to treat salivary stones // Med Case Rep Dent. 2020. ID 5960421. DOI: 10.1155/2020/5960421

[29] Grebenkov VG, Rumyantsev VN, Ivanov VM, et al. Perioperative augmented reality technology in surgical treatment of locally advanced recurrent rectal cancer. Pirogov Russian Journal of Surgery. 2022;(12-2):44–53. (In Russ.). DOI: 10.17116/hirurgia202212244

[30] Гребеньков В.Г., Румянцев В.Н., Иванов В.М., и др. Периоперационное применение технологии дополненной реальности в хирургическом лечении больного местнораспространенным локорегионарным рецидивом рака прямой кишки // Хирургия. Журнал им. Н.И. Пирогова. 2022. № 12-2. С. 44–53. DOI: 10.17116/hirurgia202212244

[31] Shapovalov VM, Gladkov RV. Explosive damage in peacetime: epidemiology, pathogenesis and main clinical manifestations. Medicо-Biological and Socio-Psychological Problems of Safety in Emergency Situations. 2014;(3):5–16. (In Russ.). DOI: 10.25016/2541-7487-2014-0-3-5-16

[32] Шаповалов В.М., Гладков Р.В. Взрывные повреждения мирного времени: эпидемиология, патогенез и основные клинические проявления // Медикобиологические и социально-психологические проблемы безопасности в чрезвычайных ситуациях. 2014. № 3. С. 5–16. DOI: 10.25016/2541-7487-2014-0-3-5-16

RIGHTS & PERMISSIONS

Eco-Vector