HDCAR: A 3D-2D registration network for abdominal aortic vessels based on CTA vessel models and DSA images

Abstract

Multimodal image registration is a crucial prerequisite for the automation and intelligence of interventional surgical medical robots. In endovascular aneurysm repair, due to limitations in imaging principles and hemodynamic effects, single-frame DSA images often fail to provide a complete representation of the vascular structure. This is particularly true for blood vessels that run parallel to the X-ray beam, as they are difficult to visualize in the DSA images. To address this issue, this study proposes an abdominal aortic vessel registration network, HDCAR, based on preoperative CTA 3D vascular models and intraoperative DSA images, aiming to enhance vascular completeness and spatial consistency in intraoperative imaging. The HDCAR network integrates multiple optimization modules to improve registration accuracy and robustness. First, the K-Sample module is employed to filter DSA images, enhancing the uniformity of intra-vascular structures and improving contrast between vessels and surrounding tissues. Second, depth information is incorporated to strengthen cross-dimensional spatial feature fusion, thereby optimizing the alignment between preoperative 3D models and intraoperative 2D images. Additionally, the network utilizes a dual-rectangular-window-based cross-attention mechanism and the RankC module to enhance both global contextual relationships and local feature representations. The ASPP module is further employed to extract multi-scale feature information, improving the model’s ability to capture vascular structures. Finally, a two-stage hybrid loss function is applied to optimize network parameters, ensuring precise and stable image registration. Experimental results demonstrate that the HDCAR network achieves high-precision vascular registration across multi-modal images, significantly improving the completeness and accuracy of intraoperative vascular imaging. This provides more precise imaging support for endovascular aneurysm repair procedures and holds great potential for clinical applications.

Keywords

Image registration / Medical robotics / Abdominal aortic aneurysm / Deep learning

CRediT authorship contribution statement

Bo Zhang: Writing – original draft, Visualization, Validation, Software, Methodology, Investigation. Shiqi Liu: Supervision, Resources, Formal analysis. Xiaoliang Xie: Supervision, Resources. Xiaohu Zhou: Supervision, Formal analysis. Zengguang Hou: Writing – review & editing, Supervision. Meng Song: Validation. Xiyao Ma: Visualization. Kang Li: Data curation. Zhichao Lai: Data curation. Bao Liu: Data curation.

Ethics approval

This study was performed in line with the principles of the Declaration of Helsinki. Approval was granted by the Peking Union Medical College Hospital’s Ethics Committee (I-25YSB0629). All data was anonymized and securely stored.

Declaration of competing interest

The authors declare that they have no known competing financial interests or personal relationships that could have appeared to influence the work reported in this paper.

Acknowledgments

This paper was supported in part by National Key Research and Development Program of China (2024YFF1206902); in part by the Development Project of National Major Scientific Research Instrument (82327801); in part by the National Natural Science Foundation of China (62303463); in part by the Beijing Natural Science Foundation (L246047, L232137, Z241100009024031); in part by the National High Level Hospital Clinical Research Funding (2022-PUMCH-B-125).

References

[1] T. Vercauteren, X. Pennec, A. Perchant, N. Ayache, Diffeomorphic demons: Efficient non-parametric image registration, Neuroimage 45 (1) (2009) S61-S72, https://doi.org/10.1016/j.neuroimage.2008.10.040.

[2] A.U. SIFTflow, Non-rigid multi-modal registration of coronary, Pattern Recognition and Image Analysis: 5th Iberian Conference, IbPRIA 2011, Las Palmas de Gran Canaria, Spain, June 8–10, 2011. Proceedings, vol. 6669, Springer Science & Business Media (2011), p. 159, 10.1007/978-3-642-21257-4_20.

[3] C.J. Schulz, M. Schmitt, D. Böckler, P. Geisbüsch, Fusion imaging to support endovascular aneurysm repair using 3D-3D registration, J. Endovasc. Ther. 23 (5) (2016) 791-799, https://doi.org/10.1177/1526602816660327.

[4] B.D. De Vos, F.F. Berendsen, M.A. Viergever, M. Staring, I. Išgum, End-to-end unsupervised deformable image registration with a convolutional neural network, International Workshop on Deep Learning in Medical Image Analysis, Springer, 2017, pp. 204-212, https://doi.org/10.1007/978-3-319-67558-9_24.

[5] D. Mahapatra, B. Antony, S. Sedai, R. Garnavi, Deformable medical image registration using generative adversarial networks, 2018 IEEE 15th International Symposium on Biomedical Imaging, ISBI 2018, IEEE, 2018, pp. 1449-1453, https://doi.org/10.1109/ISBI.2018.8363845.

[6] Y. Fu, Y. Lei, T. Wang, W.J. Curran, T. Liu, X. Yang, Deep learning in medical image registration: a review, Phys. Med. Biol. 65 (20) (2020) 20TR01, https://doi.org/10.1088/1361-6560/ab843e.

[7] G. Balakrishnan, A. Zhao, M.R. Sabuncu, J. Guttag, A.V. Dalca, An unsupervised learning model for deformable medical image registration, Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, 2018, pp. 9252-9260, https://doi.org/10.1109/CVPR.2018.00964.

[8] G. Balakrishnan, A. Zhao, M.R. Sabuncu, J. Guttag, A.V. Dalca, Voxelmorph: a learning framework for deformable medical image registration, IEEE Trans. Med. Imaging 38 (8) (2019) 1788-1800, https://doi.org/10.1109/TMI.2019.2897538.

[9] K.A. Eppenhof, J.P. Pluim, Pulmonary CT registration through supervised learning with convolutional neural networks, IEEE Trans. Med. Imaging 38 (5) (2018) 1097-1105, https://doi.org/10.1109/TMI.2018.2878316.

[10] K. López-Linares, I. García, A. García, C. Cortes, G. Piella, I. Macía, J. Noailly, M.A. Gonzalez Ballester, Image-based 3D characterization of abdominal aortic aneurysm deformation after endovascular aneurysm repair, Front. Bioeng. Biotechnol. 7 (2019) 267, https://doi.org/10.3389/fbioe.2019.00267.

[11] R. Azad, A. Kazerouni, M. Heidari, E.K. Aghdam, A. Molaei, Y. Jia, A. Jose, R. Roy, D. Merhof, Advances in medical image analysis with vision transformers: A comprehensive review, Med. Image Anal. 91 (2024) 103000, https://doi.org/10.1016/j.media.2023.103000.

[12] T. Duan, W. Chen, M. Ruan, X. Zhang, S. Shen, W. Gu, Unsupervised deep learning-based medical image registration: a survey, Phys. Med. Biol. 70 (2) (2025) 02TR01, https://doi.org/10.1088/1361-6560/ad9e69.

[13] Z. Wang, H. Wang, Y. Wang, Pyramid attention network for medical image registration, 2024 IEEE International Symposium on Biomedical Imaging, ISBI, IEEE, 2024, pp. 1-5, https://doi.org/10.1109/ISBI56570.2024.10635156.

[14] B. Hou, A. Alansary, S. McDonagh, A. Davidson, M. Rutherford, J.V. Hajnal, D. Rueckert, B. Glocker, B. Kainz, Predicting slice-to-volume transformation in presence of arbitrary subject motion, Medical Image Computing and Computer-Assisted Intervention- MICCAI 2017: 20th International Conference, Quebec City, QC, Canada, September 11–13, 2017, Proceedings, Part II 20, Springer, 2017, pp. 296-304, https://doi.org/10.1007/978-3-319-66185-8_34.

[15] Z. Chen, Z. Xu, W. Yi, X. Yang, W. Hou, M. Ding, O. Granichin, Real-time and multimodal brain slice-to-volume registration using CNN, Expert Syst. Appl. 133 (2019) 86-96, https://doi.org/1010.1016/j.eswa.2019.05.016.

[16] Z. Chen, Z. Xu, Q. Gui, X. Yang, Q. Cheng, W. Hou, M. Ding, Self-learning based medical image representation for rigid real-time and multimodal slice-to-volume registration, Inform. Sci. 541 (2020) 502-515, https://doi.org/1010.1016/j.ins.2020.06.072.

[17] A. Leroy, M. Lerousseau, T. Henry, A. Cafaro, N. Paragios, V. Grégoire, E. Deutsch, End-to-end multi-slice-to-volume concurrent registration and multimodal generation, International Conference on Medical Image Computing and Computer-Assisted Intervention, Springer, 2022, pp. 152-162, https://doi.org/10.1007/978-3-031-16446-0_15.

[18] J. Xu, D. Moyer, P.E. Grant, P. Golland, J.E. Iglesias, E. Adalsteinsson, SVoRT: Iterative transformer for slice-to-volume registration in fetal brain MRI, International Conference on Medical Image Computing and Computer-Assisted Intervention, Springer, 2022, pp. 3-13, https://doi.org/10.1007/978-3-031-16446-0_1.

[19] Q. Wu, H. Jiang, L. Luo, J. Li, Y. Ding, J. Xie, J. Yang, Diff-reg: Diffusion model in doubly stochastic matrix space for registration problem, European Conference on Computer Vision, Springer, 2024, pp. 160-178, https://doi.org/10.1007/978-3-031-73650-6_10.

[20] M. Li, Z. Qin, Z. Gao, R. Yi, C. Zhu, Y. Guo, K. Xu, 2D3d-matr: 2d-3d matching transformer for detection-free registration between images and point clouds, Proceedings of the IEEE/CVF International Conference on Computer Vision, 2023, pp. 14128-14138, https://doi.org/10.1109/ICCV51070.2023.01299.

[21] V. Gopalakrishnan, N. Dey, P. Golland, Intraoperative 2d/3d image registration via differentiable x-ray rendering, Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2024, pp. 11662-11672, https://doi.org/10.1109/CVPR52733.2024.01108.

[22] L.-C. Chen, G. Papandreou, I. Kokkinos, K. Murphy, A.L. Yuille, Deeplab: Semantic image segmentation with deep convolutional nets, atrous convolution, and fully connected crfs, IEEE Trans. Pattern Anal. Mach. Intell. 40 (4) (2017) 834-848, https://doi.org/10.1109/TPAMI.2017.2699184.

[23] J. Wang, M. Zhang, Deepflash: An efficient network for learning-based medical image registration, Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2020, pp. 4444-4452, https://doi.org/10.1109/CVPR42600.2020.00450.

[24] Y. Liao, B. Lei, Y. Zhuang, X. Xu, M. Zhang, DCP++: A new registration method for brain MRI, 2024 IEEE International Conference on Medical Artificial Intelligence (MedAI), IEEE, 2024, pp. 441-448, https://doi.org/10.1109/MedAI62885.2024.00065.

[25] Z. Chen, Y. Zheng, J.C. Gee, TransMatch: a transformer-based multilevel dual-stream feature matching network for unsupervised deformable image registration, IEEE Trans. Med. Imaging 43 (1) (2023) 15-27, https://doi.org/10.1109/TMI.2023.3288136.

[26] W. Tang, F. He, Y. Liu, Y. Duan, MATR: Multimodal medical image fusion via multiscale adaptive transformer, IEEE Trans. Image Process. 31 (2022) 5134-5149, https://doi.org/1010.1109/TIP.2022.3193288.