Rapid 3D reconstruction in fetal ultrasound imaging using artificial intelligence and medical 3D printing

Abstract

Congenital heart disease (CHD) has been one of the most serious problems in newborns. For fetal heart health care, 3D modeling and printing technology have been adopted in the diagnosis of CHD during antenatal care. However, the development of 3D printing techniques and their clinical applications have been hindered by the manual processing of ultrasound (US) volume data in clinical practice. To overcome this problem, we present an interactive semi-automatic method based on deep learning that uses manual processing results from expert sonographers for training. The accuracy, interpretability, and variability of the performances were evaluated on the validation set. The results demonstrated that compared with a physician with less than 3 years of experience, a better Faster- region-based convolutional neural network-based threshold was achieved using our proposed fetal heart reconstruction technique (FRT), with enhanced performance based on the outflow tract view and three-vessel view. No significant difference was found among the clinical parameters, in proportion, measured from the model rebuilt using FRT and US volume data. Furthermore, the reconstruction time of the fetal heart blood pool model was reduced from approximately 5 h to 5 min. Our results indicate that deep learning has the ability to process US data accurately, representing an important step towards the reconstruction of the fetal heart digital model, which is critical for advancing clinical diagnosis and treatment of CHD during pregnancy.

Keywords

3D printing technology / Congenital heart disease / Deep learning / Reconstruction of ultrasound imaging data

Funding

This work was supported by the Key Research and Development Project of Shaanxi Province (2021LLRH-08).

Conflict of interest

The authors declare that they have no competing financial interests or personal relationships that could have appeared to influence the work reported in this paper.

References

[1] Benjamin EJ, Muntner P, Alonso A, et al. Heart disease and stroke statistics-2019 update: a report from the American Heart Association. Circulation. 2019; 139(10):e56-e528. doi: 10.1161/cir.0000000000000659

[2] International Society of Ultrasound in Obstetrics and Gynecology, Carvalho JS, Allan LD, et al. ISUOG practice guidelines (updated): sonographic screening examination of the fetal heart. Ultrasound Obstet Gynecol. 2013; 41(3):348-59. doi: 10.1002/uog.12403

[3] Jantarasaengaram S, Vairojanavong K. Eleven fetal echocardiographic planes using 4-dimensional ultrasound with spatio-temporal image correlation (STIC): a logical approach to fetal heart volume analysis. Cardiovasc Ultrasound. 2010;8:41. doi: 10.1186/1476-7120-8-41

[4] Huang J, Shi H, Chen Q, et al. Three-dimensional printed model fabrication and effectiveness evaluation in fetuses with congenital heart disease or with a normal heart. J Ultrasound Med. 2021; 40(1):15-28. doi: 10.1002/jum.15366

[5] Fu Y, Lei Y, Wang T, Curran WJ, Liu T, Yang X. Deep learning in medical image registration: a review. Phys Med Biol. 2020; 65(20):20tr01. doi: 10.1088/1361-6560/ab843e

[6] Madani A, Ong JR, Tibrewal A, Mofrad MRK. Deep echocardiography: data-efficient supervised and semi-supervised deep learning towards automated diagnosis of cardiac disease. NPJ Digit Med. 2018;1:59. doi: 10.1038/s41746-018-0065-x

[7] Ackland DC, Robinson D, Redhead M, Lee PVS, Moskaljuk A, Dimitroulis G. A personalized 3D-printed prosthetic joint replacement for the human temporomandibular joint: from implant design to implantation. J Mech Behav Biomed Mater. 2017; 69:404-411. doi: 10.1016/j.jmbbm.2017.01.048

[8] Diment LE, Thompson MS, Bergmann JHM. Clinical efficacy and effectiveness of 3D printing: a systematic review. BMJ Open. 2017; 7(12):e016891. doi: 10.1136/bmjopen-2017-016891

[9] Hosny A, Dilley JD, Kelil T, et al. Pre-procedural fit-testing of TAVR valves using parametric modeling and 3D printing. J Cardiovasc Comput Tomogr. 2019; 13(1):21-30. doi: 10.1016/j.jcct.2018.09.007

[10] Costello JP, Olivieri LJ, Su L, et al. Incorporating three-dimensional printing into a simulation-based congenital heart disease and critical care training curriculum for resident physicians. Congenit Heart Dis. 2015; 10(2):185-90. doi: 10.1111/chd.12238

[11] Ng WL, Goh GL, Goh GD, Ten JSJ, Yeong WY. Progress and opportunities for machine learning in materials and processes of additive manufacturing. Adv Mater. 2024; 36(34):e2310006. doi: 10.1002/adma.202310006

[12] Jin L, Zhai X, Wang K, et al. Big data, machine learning, and digital twin assisted additive manufacturing: a review. Mater Des. 2024;244:113086. doi: 10.1016/j.matdes.2024.113086

[13] Ren S, He K, Girshick R, Sun J. Faster R-CNN: towards real-time object detection with region proposal networks. IEEE Trans Pattern Anal Mach Intell. 2017; 39(6):1137-1149. doi: 10.1109/TPAMI.2016.2577031

[14] Bell S, Zitnick CL, Bala K, Girshick R. Inside-outside net: detecting objects in context with skip pooling and recurrent neural networks. Proc CVPR IEEE. 2016:2874-2883. doi: 10.1109/Cvpr.2016.314

[15] Ardila D, Kiraly AP, Bharadwaj S, et al. End-to-end lung cancer screening with three-dimensional deep learning on low-dose chest computed tomography. Nat Med. 2019; 25(6):954-961. doi: 10.1038/s41591-019-0447-x

[16] Poplin R, Varadarajan AV, Blumer K, et al. Prediction of cardiovascular risk factors from retinal fundus photographs via deep learning. Nat Biomed Eng. 2018; 2(3):158-164. doi: 10.1038/s41551-018-0195-0

[17] Esteva A, Kuprel B, Novoa RA, et al. Dermatologist-level classification of skin cancer with deep neural networks. Nature. 2017; 542(7639):115-118. doi: 10.1038/nature21056

[18] Coudray N, Ocampo PS, Sakellaropoulos T, et al. Classification and mutation prediction from non-small cell lung cancer histopathology images using deep learning. Nat Med. 2018; 24(10):1559-1567. doi: 10.1038/s41591-018-0177-5

[19] Simonyan K, Zisserman A. Very deep convolutional networks for large-scale image recognition. arXiv. 2015. doi: 10.48550/arXiv.1409.1556

[20] Abadi M, Agarwal A, Barham P, et al. TensorFlow: large-scale machine learning on heterogeneous distributed systems. arXiv. 2016. doi: 10.48550/arXiv.1603.04467

[21] Bradski G, Kaehler A. Learning OpenCV: Computer Vision with the OpenCV Library. Cambridge: O’Reilly; 2008. doi: 10.1109/MRA.2009.933612

[22] Bishop KC, Kuller JA, Boyd BK, Rhee EH, Miller S, Barker P. Ultrasound examination of the fetal heart. Obstet Gynecol Surv. 2017; 72(1):54-61. doi: 10.1097/OGX.0000000000000394

[23] Liu S, Wang Y, Yang X, et al. Deep learning in medical ultrasound analysis: a review. Engineering. 2019; 5(2):261-275. doi: 10.1016/j.eng.2018.11.020

[24] Ouyang D, He B, Ghorbani A, et al. Video-based AI for beat-to-beat assessment of cardiac function. Nature. 2020; 580(7802):252-256. doi: 10.1038/s41586-020-2145-8

[25] Campanella G, Hanna MG, Geneslaw L, et al. Clinical-grade computational pathology using weakly supervised deep learning on whole slide images. Nat Med. 2019; 25(8):1301-1309. doi: 10.1038/s41591-019-0508-1

[26] Litjens G, Kooi T, Bejnordi BE, et al. A survey on deep learning in medical image analysis. Med Image Anal. 2017; 42:60-88. doi: 10.1016/j.media.2017.07.005

[27] Luijten B, Cohen R, de Bruijn FJ, et al. Adaptive ultrasound beamforming using deep learning. IEEE Trans Med Imaging. 2020; 39(12):3967-3978. doi: 10.1109/TMI.2020.3008537

[28] Liaw CY, Guvendiren M. Current and emerging applications of 3D printing in medicine. Biofabrication. 2017; 9(2):024102. doi: 10.1088/1758-5090/aa7279