Advancements in needle visualization enhancement and localization methods in ultrasound: a literature review

Abstract

Ultrasound guidance plays a central role in numerous minimally invasive procedures involving percutaneous needle insertion, ensuring safe and accurate needle placement. However, it encounters two primary challenges: (1) aligning the needle with the ultrasound beam and (2) visualizing the needle even when correctly aligned. In this review, we offer a concise overview of the physics foundation underlying these challenges and explore various approaches addressing specific challenges, with a focus on software-based solutions. We further distinguish between hardware-based and software-based solutions, placing a stronger emphasis on the latter. The incorporation of artificial intelligence into these methods to enhance needle visualization and localization is briefly discussed. We identify state-of-the-art needle detection methods, showcasing submillimeter precision in tip localization and orientation. Additionally, we provide insights into potential future directions, aiming to facilitate the translation of these advanced methods into the clinic. This article serves as a comprehensive guide, offering insights into challenges, evolving solutions, and prospective research directions to effectively address these issues.

Keywords

Needle enhancement / segmentation / localization / ultrasound / biopsy / machine learning / deep learning

References

[1] Barr L,Roque PJ.Basic ultrasound-guided procedures.Crit Care Clin2014;30:275-304

[2] Tirado A,Henningsen C,Chiles K.Ultrasound-guided procedures in the emergency department-needle guidance and localization.Emerg Med Clin North Am2013;31:87-115

[3] Campbell AS.Ultrasound-Guided Procedures. In: Kuzmiak CM, editor. Interventional breast procedures. Cham: Springer; 2019. pp. 21-33.

[4] Huntoon MA.Imaging in Interventional Pain Management. In: Narouze SN, editor. Atlas of ultrasound-guided procedures in interventional pain management. New York: Springer; 2011. pp. 3-12.

[5] Albrecht E.Advances in regional anaesthesia and acute pain management: a narrative review.Anaesthesia2020;75:e101-10

[6] Souza D,Halaszynski TM.Ultrasound technical aspects: how to improve needle visibility. In: Narouze SN, editor. Atlas of ultrasound-guided procedures in interventional pain management. New York: Springer; 2018. pp. 27-55.

[7] Marhofer P.Ultrasound-guided regional anesthesia: current concepts and future trends.Anesth Analg2007;104:1265-9

[8] Chapman GA,Bodenham AR.Visualisation of needle position using ultrasonography.Anaesthesia2006;61:148-58

[9] Scholten HJ,Mihajlovic N,A Bouwman R.Improving needle tip identification during ultrasound-guided procedures in anaesthetic practice.Anaesthesia2017;72:889-904

[10] Yang H,Kolen AF.Medical instrument detection in ultrasound: a review.Artif Intell Rev2023;56:4363-402

[11] Beigi P,Ng GC.Enhancement of needle visualization and localization in ultrasound.Int J Comput Assist Radiol Surg2021;16:169-78

[12] Chin KJ,Chan VWS.Needle visualization in ultrasound-guided regional anesthesia: challenges and solutions.Reg Anesth Pain Med2008;33:532-44

[13] Peng C,Chen M.Recent Advances in tracking devices for biomedical ultrasound imaging applications.Micromachines2022;13:1855 PMCID:PMC9695235

[14] Aldrich JE.Basic physics of ultrasound imaging.Crit Care Med2007;35:S131-7

[15] Obstetric Anesthesia. Needling techniques. Available from: https://pie.med.utoronto.ca/OBAnesthesia/OBAnesthesia_content/OBA_needling_module.html. [Last accessed on 25 Jul 2024]

[16] Cao L,Wei T.Comparison between the long-axis in-plane and short-axis out-of-plane approaches for ultrasound-guided arterial cannulation: a meta-analysis and systematic review.BMC Anesthesiol2023;23:120 PMCID:PMC10100252

[17] Miller AG.Review of ultrasound-guided radial artery catheter placement.Respir Care2016;61:383-8

[18] Chan V.Basics of ultrasound: pitfalls and limitations. In: Narouze SN, editor. Atlas of ultrasound-guided procedures in interventional pain management. New York: Springer; 2018. pp. 11-5.

[19] Elsharkawy H,Kolli S,Soliman ML.The Infiniti Plus ultrasound needle guidance system improves needle visualization during the placement of spinal anesthesia.Korean J Anesthesiol2016;69:417-9 PMCID:PMC4967642

[20] Collins GB,Young J.A comparison of free-hand vs laser-guided long-axis ultrasound techniques in novice users.Br J Radiol2013;86:20130026 PMCID:PMC3755392

[21] Maecken T,Wolf B,Litz RJ.Ultrasound-guided catheterisation of the subclavian vein: freehand vs needle-guided technique.Anaesthesia2015;70:1242-9

[22] Antico M,Wu L.Ultrasound guidance in minimally invasive robotic procedures.Med Image Anal2019;54:149-67

[23] Hemmerling TM,Wehbe M,Zaouter C.First robotic ultrasound-guided nerve blocks in humans using the magellan system.Anesth Analg2013;116:491-4

[24] Kojcev R,Zettinig O.Dual-robot ultrasound-guided needle placement: closing the planning-imaging-action loop.Int J Comput Assist Radiol Surg2016;11:1173-81

[25] Brattain LJ,Gjesteby LA.AI-enabled, ultrasound-guided handheld robotic device for femoral vascular access.Biosensors2021;11:522 PMCID:PMC8699246

[26] Choquet O.Case report: Three-dimensional high-resolution ultrasound-guided nerve blocks: a new panoramic vision of local anesthetic spread and perineural catheter tip location.Anesth Analg2013;116:1176-81

[27] Beigi P,Rasoulian A,Dube A.Three-dimensional ultrasound-guided real-time midline epidural needle placement with epiguide: a prospective feasibility study.Ultrasound Med Biol2017;43:375-9

[28] Gebhard RE,Meeks R.Three-dimensional ultrasound imaging.Curr Opin Anesthesiol2015;28:583-7

[29] Hopkins RE.In-vitro visualization of biopsy needles with ultrasound: a comparative study of standard and echogenic needles using an ultrasound phantom.Clin Radiol2001;56:499-502

[30] van de Berg NJ, Sánchez-Margallo JA, van Dijke AP, Langø T, van den Dobbelsteen JJ. A methodical quantification of needle visibility and echogenicity in ultrasound images.Ultrasound Med Biol2019;45:998-1009

[31] Brookes J,Armstrong K.Comparative evaluation of the visibility and block characteristics of a stimulating needle and catheter vs an echogenic needle and catheter for sciatic nerve block with a low-frequency ultrasound probe.Br J Anaesth2015;115:912-9

[32] Hovgesen CH,Vilmann P.Echogenic surface enhancements for improving needle visualization in ultrasound: a PRISMA systematic review.J Ultrasound Med2022;41:311-25

[33] Klein SM,Reach J,Smith SW.Piezoelectric vibrating needle and catheter for enhancing ultrasound-guided peripheral nerve blocks.Anesth Analg2007;105:1858-60

[34] Orlando N,Barker K.A power Doppler ultrasound method for improving intraoperative tip localization for visually obstructed needles in interstitial prostate brachytherapy.Med Phys2023;50:2649-61

[35] Daoud MI,Zayadeen AR.Accurate needle localization using two-dimensional power doppler and b-mode ultrasound image analyses: a feasibility study.Sensors2018;18:3475 PMCID:PMC6209937

[36] Harmat A,Salcudean SE.Needle tip localization using stylet vibration [Journal Article].Ultrasound Med Biol2006;32:1339-48

[37] Stolka PJ,Rendina M,Hager GD.Needle guidance using handheld stereo vision and projection for ultrasound-based interventions.Med Image Comput Comput Assist Interv2014;17:684-91 PMCID:PMC5488249

[38] Najafi M,Rohling R.Single-camera closed-form real-time needle tracking for ultrasound-guided needle insertion.Ultrasound Med Biol2015;41:2663-76

[39] Gallo C,Meagher E.Computer-assisted needle navigation for pediatric internal jugular central venous cannulation: a feasibility study.J Vasc Access2020;21:931-7 PMCID:PMC7606770

[40] Xia W,West SJ.In-plane ultrasonic needle tracking using a fiber-optic hydrophone.Med Phys2015;42:5983-91 PMCID:PMC5207301

[41] Baker C,Lin FY.Intraoperative needle tip tracking with an integrated fibre-optic ultrasound sensor.Sensors2022;22:9035 PMCID:PMC9739176

[42] Kåsine T,Rosseland LA.Needle tip tracking for ultrasound-guided peripheral nerve block procedures - an observer blinded, randomised, controlled, crossover study on a phantom model.Acta Anaesthesiol Scand2019;63:1055-62 PMCID:PMC6767137

[43] Watanabe K,Lefor AK.Photoacoustic needle improves needle tip visibility during deep peripheral nerve block.Sci Rep2021;11:8432 PMCID:PMC8055898

[44] Nakazawa H,Lefor AK.Use of a photoacoustic needle improves needle tip recognition in a video recording of simulated ultrasound-guided vascular access: a pilot study.J Vasc Access2024;25:922-7 PMCID:PMC11075403

[45] Tielens LKP,Lerou JGC,Bruhn J.Ultrasound-guided needle handling using a guidance positioning system in a phantom.Anaesthesia2014;69:24-31

[46] Fevre MC,Picard J.Reduced variability and execution time to reach a target with a needle GPS system: comparison between physicians, residents and nurse anaesthetists.Anaesth Crit Care Pain Med2018;37:55-60

[47] Niazi AU,Jin R.Real-time ultrasound-guided spinal anesthesia using the SonixGPS ultrasound guidance system: a feasibility study.Acta Anaesthesiol Scand2014;58:875-81

[48] Hakime A,De Carvalho EG,Auperin A.Electromagnetic-tracked biopsy under ultrasound guidance: preliminary results.Cardiovasc Intervent Radiol2012;35:898-905

[49] Swenson JD,Pace NL,Loose EC.Evaluation of a new needle guidance system for ultrasound: results of a prospective, randomized, blinded study.Reg Anesth Pain Med2016;41:356-61

[50] Johnson AN,Halmann N,Owen CA.Ultrasound-guided needle technique accuracy: prospective comparison of passive magnetic tracking versus unassisted echogenic needle localization.Reg Anesth Pain Med2017;42:223-32 PMCID:PMC5318157

[51] Cheung S.Enhancement of needle visibility in ultrasound-guided percutaneous procedures.Ultrasound Med Biol2004;30:617-24

[52] Beigi P,Rohling R.Automatic detection of a hand-held needle in ultrasound via phased-based analysis of the tremor motion.Proc SPIE2016;9786:166-71

[53] Mwikirize C,Hacihaliloglu I.Enhancement of needle tip and shaft from 2d ultrasound using signal transmission maps. In: Ourselin S, Joskowicz L, Sabuncu MR, Unal G, Wells W, editors. Medical image computing and computer-assisted intervention - MICCAI 2016. Cham: Springer; 2016. pp. 362-9.

[54] Mwikirize C,Hacihaliloglu I.Learning needle tip localization from digital subtraction in 2D ultrasound.Int J Comput Assist Radiol Surg2019;14:1017-26

[55] Beigi P,Salcudean SE.CASPER: computer-aided segmentation of imperceptible motion-a learning-based tracking of an invisible needle in ultrasound.Int J Comput Assist Radiol Surg2017;12:1857-66

[56] Pourtaherian A,Kusters L.Medical instrument detection in 3-dimensional ultrasound data volumes.IEEE Trans Med Imaging2017;36:1664-75

[57] Ayvali E.Optical flow-based tracking of needles and needle-tip localization using circular hough transform in ultrasound images.Ann Biomed Eng2015;43:1828-40 PMCID:PMC4465878

[58] Pourtaherian A,de With,Korsten HHM.Benchmarking of state-of-the-art needle detection algorithms in 3D ultrasound data volumes.Proc SPIE2015;9415:577-84

[59] Beigi P,Salcudean SE.Spectral analysis of the tremor motion for needle detection in curvilinear ultrasound via spatiotemporal linear sampling.Int J Comput Assist Radiol Surg2016;11:1183-92

[60] Mwikirize C,Hacihaliloglu I.Local phase-based learning for needle detection and localization in 3D ultrasound. In: Cardoso MJ, et al., editors. Computer assisted and robotic endoscopy and clinical image-based procedures. Cham: Springer; 2017. pp. 108-15.

[61] Zhao Y,Liebgott H.Automatic needle detection and tracking in 3D ultrasound using an ROI-based RANSAC and Kalman method.Ultrason Imaging2013;35:283-306

[62] Tang C,Omisore OM,Xia Z.A real-time needle tracking algorithm with first-frame linear structure removing in 2D ultrasound-guided prostate therapy. In: 2019 IEEE International Conference on Robotics and Biomimetics (ROBIO); 2019 Dec 6-8; Dali, China. IEEE; 2020. pp. 1240-5.

[63] Hacihaliloglu I,Ng G,Salcudean S.Projection-based phase features for localization of a needle tip in 2D curvilinear ultrasound. In: Navab N, Hornegger J, Wells WM, Frangi A, editors. Medical image computing and computer-assisted intervention - MICCAI 2015. Cham: Springer; 2015. pp. 347-54.

[64] Agarwal N,Gupta A.Real-time needle tip localization in 2D ultrasound images using kalman filter. In: 2019 IEEE/ASME International Conference on Advanced Intelligent Mechatronics (AIM); 2019 Jul 8-12; Hong Kong, China. IEEE; 2019. pp. 1008-12.

[65] Mwikirize C,Hacihaliloglu I.Single shot needle tip localization in 2D ultrasound. In: Shen D, et al., editors. Medical image computing and computer assisted intervention - MICCAI 2019. Cham: Springer; 2019. pp. 637-45.

[66] Mwikirize C,Imanirakiza S,Nosher JL.Time-aware deep neural networks for needle tip localization in 2D ultrasound.Int J Comput Assist Radiol Surg2021;16:819-27

[67] Hatt CR,Parthasarathy V.Enhanced needle localization in ultrasound using beam steering and learning-based segmentation.Comput Med Imaging Graph2015;41:46-54

[68] Younes H,Troccaz J.Automatic needle localization in 3D ultrasound images for brachytherapy. In: 2018 IEEE 15th International Symposium on Biomedical Imaging (ISBI 2018); 2018 Apr 4-7; Washington, DC, USA. IEEE; 2018. pp. 1203-7.

[69] Beigi P,Salcudean T,Ng GC.Detection of an invisible needle in ultrasound using a probabilistic SVM and time-domain features.Ultrasonics2017;78:18-22

[70] Geraldes AA.A neural network approach for flexible needle tracking in ultrasound images using Kalman filter. In: 5th IEEE RAS/EMBS International Conference on Biomedical Robotics and Biomechatronics; 2014 Aug 12-15; Sao Paulo, Brazil. IEEE; 2014. pp. 70-5.

[71] Pourtaherian A,Zinger S.Improving needle detection in 3d ultrasound using orthogonal-plane convolutional networks. In: Descoteaux M, et al., editors. Medical image computing and computer-assisted intervention - MICCAI 2017. Cham: Springer; 2017. pp. 610-8.

[72] Pourtaherian A. Robust needle detection and visualization for 3D ultrasound image-guided interventions. Available from: https://research.tue.nl/en/publications/robust-needle-detection-and-visualization-for-3d-ultrasound-image. [Last accessed on 25 Jul 2024]

[73] Arif M,van Walsum T.Automatic needle detection and real-time Bi-planar needle visualization during 3D ultrasound scanning of the liver.Med Image Anal2019;53:104-10

[74] Lee JY,Woh JR.Ultrasound needle segmentation and trajectory prediction using excitation network.Int J Comput Assist Radiol Surg2020;15:437-43

[75] Zhang Y,Tian Z.Multi-needle detection in 3d ultrasound images using unsupervised order-graph regularized sparse dictionary learning.IEEE Trans Med Imaging2020;39:2302-15 PMCID:PMC7370243

[76] Gao J,Liu GD.Robust needle localization and enhancement algorithm for ultrasound by deep learning and beam steering methods.J Comput Sci Technol2021;36:334-46

[77] Wijata A,Pyciński B.An automatic biopsy needle detection and segmentation on ultrasound images using a convolutional neural network.Ultrason Imaging2021;43:262-72

[78] Chen S,Li Z,Cao Q.Automatic and accurate needle detection in 2D ultrasound during robot-assisted needle insertion process.Int J Comput Assist Radiol Surg2022;17:295-303

[79] Zhang Y,Lei Y.Automatic multi-needle localization in ultrasound images using large margin mask RCNN for ultrasound-guided prostate brachytherapy.Phys Med Biol2020;65:205003

[80] Zhang Y,Lei Y.Multi-needle detection in ultrasound image using max-margin mask R-CNN.Proc SPIE2021;11602:264-69

[81] Andersén C,Thunberg P.Deep learning-based digitization of prostate brachytherapy needles in ultrasound images.Med Phys2020;47:6414-20 PMCID:PMC7821271

[82] Zhang Y,He X.Ultrasound multi-needle detection using deep attention U-Net with TV regularizations.Proc SPIE2021;11598:599-604

[83] Mwikirize C,Hacihaliloglu I.Convolution neural networks for real-time needle detection and localization in 2D ultrasound.Int J Comput Assist Radiol Surg2018;13:647-57

[84] Wang R,Liu X.Robust tip localization under continuous spatial and temporal constraints during 2D ultrasound-guided needle puncture.Int J Comput Assist Radiol Surg2023;18:2233-42

[85] Rubin J,Thodiyil AO. Efficient video-based deep learning for ultrasound guided needle insertion. 2021. Available from: https://openreview.net/forum?id=dVUHL5QhDhL. [Last accessed on 25 Jul 2024]

[86] Wang R,Liu X.TipDet: A multi-keyframe motion-aware framework for tip detection during ultrasound-guided interventions.Comput Methods Programs Biomed2024;247:108109

[87] Che H,Chen Y.Improving needle tip tracking and detection in ultrasound-based navigation system using deep learning-enabled approach.IEEE J Biomed Health Inform2024;28:2930-42

[88] Zade AAT,Majedi H,Ahmadian A.Spatiotemporal analysis of speckle dynamics to track invisible needle in ultrasound sequences using convolutional neural networks: a phantom study.Int J Comput Assist Radiol Surg2023;18:1373-82

[89] Gillies DJ,Gyacskov I.Deep learning segmentation of general interventional tools in two-dimensional ultrasound images.Med Phys2020;47:4956-70

[90] Pourtaherian A,Ghazvinian Zanjani F.Localization of partially visible needles in 3D ultrasound using dilated CNNs. In: 2018 IEEE International Ultrasonics Symposium (IUS); 2018 Oct 22-25; Kobe, Japan. IEEE; 2019. pp. 1-4.

[91] Yan W,Chen J,Tang RSY.Learning-based needle tip tracking in 2D ultrasound by fusing visual tracking and motion prediction.Med Image Anal2023;88:102847

[92] Zhang Y,Lei Y.Weakly supervised multi-needle detection in 3D ultrasound images with bidirectional convolutional sparse coding.Proc SPIE2020;11319:229-36

[93] Maier-Hein L,Godau P.Metrics reloaded: recommendations for image analysis validation.Nat Methods2024;21:195-212

[94] Reinke A,Baumgartner M.Understanding metric-related pitfalls in image analysis validation.Nat Methods2024;21:182-94 PMCID:PMC11181963

[95] Rodriguez-Molares A,Lovstakken L.Specular beamforming.IEEE Trans Ultrason Ferroelectr Freq Control2017;64:1285-97

[96] Cai Q,Chen M.Inertial measurement unit-assisted ultrasonic tracking system for ultrasound probe localization.IEEE Trans Ultrason Ferroelectr Freq Control2023;70:920-29