Multi-domain abdomen image alignment based on multi-scale diffeomorphic jointed network

Abstract

Recently, the generative adversarial network (GAN) has been extensively applied to the cross-modality conversion of medical images and has shown outstanding performance than other image conversion algorithms. Hence, we propose a novel GAN-based multi-domain registration method named multiscale diffeomorphic jointed network of registration and synthesis (MDJRS-Net). The deviation of the generator of the GAN-based approach affects the alignment phase, so a joint training strategy is introduced to improve the performance of the generator, which feedbacks the structural loss contained in the deformation field. Meanwhile, the nature of diffeomorphism can enable the network to generate deformation fields with more anatomical properties. The average dice score (Dice) is improved by 1.95% for the computer tomography venous (CTV) to magnetic resonance imaging (MRI) registration task and by 1.92% for the CTV to computer tomography plain (CTP) task compared with the other methods.

References

[1] LiuF, CaiJ, HuoY, et al.. JSSR: a joint synthesis, segmentation, and registration system for 3D multi-modal image alignment of large-scale pathological CT scans[C], 2020, Berlin, Heidelberg, Springer-Verlag: 257-274

[2] HuijskensS C, VanD I W E M, VisserJ, et al.. Abdominal organ position variation in children during image-guided radiotherapy[J]. Radiation oncology, 2018, 13(1):1-9

[3] ZhangY, JiangF, ShenR. Region-based face alignment with convolution neural network cascade[C], 2017, Berlin, Heidelberg, Springer-Verlag: 300-309

[4] JaderbergM, SimonyanK, ZissermanA. Spatial transformer networks[J]. Advances in neural information processing systems, 2015, 28: 2017-2025

[5] CaoX, YangJ, GaoY, et al.. Dual-core steered non-rigid registration for multi-modal images via bi-directional image synthesis[J]. Medical image analysis, 2017, 41: 18-31

[6] FanJ, CaoX, WangQ, et al.. Adversarial learning for mono- or multi-domain registration[J]. Medical image analysis, 2019, 58: 101545

[7] DubostF, DeB M, NardinM, et al.. Multi-atlas image registration of clinical data with automated quality assessment using ventricle segmentation[J]. Medical image analysis, 2020, 63: 101698

[8] BlendowskiM, HansenL, HeinrichM P. Weakly-supervised learning of multi-modal features for regularised iterative descent in 3D image registration[J]. Medical image analysis, 2021, 67: 101822

[9] TangT, GuanQ, WuY. Support vector machine incremental learning triggered by wrongly predicted samples[J]. Optoelectronics letters, 2018, 14(3):232-235

[10] VercauterenT, PennecX, PerchantA, et al.. Diffeomorphic demons: efficient non-parametric image registration[J]. Neuroimage, 2009, 45(1):S61-S72

[11] BegM F, MillerM I, TrouvéA, et al.. Computing large deformation metric mappings via geodesic flows of diffeomorphisms[J]. International journal of computer vision, 2005, 61(2):139-157

[12] GoodfellowI, Pouget-AbadieJ, MirzaM, et al.. Generative adversarial nets[J]. Advances in neural information processing systems, 2014, 27: 2672-2680

[13] ZhuJ Y, ParkT, IsolaP, et al.. Unpaired image-to-image translation using cycle-consistent adversarial networks[C], 2017, New York, IEEE: 2223-2232

[14] LiuM Y, BreuelT, KautzJ. Unsupervised image-to-image translation networks[J]. Advances in neural information processing systems, 2017, 30: 700-708

[15] QinC, ShiB, LiaoR, et al.. Unsupervised deformable registration for multi-modal images via disentangled representations[C], 2019, Berlin, Heidelberg, Springer-Verlag: 249-261

[16] WeiD, AhmadS, HuoJ, et al.. SLIR: synthesis, localization, inpainting, and registration for image-guided thermal ablation of liver tumors[J]. Medical image analysis, 2020, 65: 101763

[17] XuZ, LuoJ, YanJ, et al.. Adversarial uni-and multi-modal stream networks for multimodal image registration[C], 2020, Berlin, Heidelberg, Springer-Verlag: 222-232

[18] ZhouB, AugenfeldZ, ChapiroJ, et al.. Anatomy-guided multimodal registration by learning segmentation without ground truth: application to intra-procedural CBCT/MR liver segmentation and registration[J]. Medical image analysis, 2021, 71: 102041

[19] BalakrishnanG, ZhaoA, SabuncuM R, et al.. Voxelmorph: a learning framework for deformable medical image registration[J]. IEEE transactions on medical imaging, 2019, 38(8):1788-1800

[20] ArarM, GingerY, DanonD, et al.. Unsupervised multi-modal image registration via geometry preserving image-to-image translation[C], 2020, New York, IEEE: 13410-13419

[21] ZhaoS, DongY, ChangE I, et al.. Recursive cascaded networks for unsupervised medical image registration[C], 2019, New York, IEEE: 10600-10610

[22] KimB, KimD H, ParkS H, et al.. Cyclemorph: cycle consistent unsupervised deformable image registration[J]. Medical image analysis, 2021, 71: 102036

[23] HeuselM, RamsauerH, UnterthinerT, et al.. GANs trained by a two time-scale update rule converge to a local nash equilibrium[J]. Advances in neural information processing systems, 2017, 30: 6626-6637

[24] ChenY, LuZ, YangX H, et al.. Multi-domain abdomen image alignment based on joint network of registration and synthesis[C], 2021, Berlin, Heidelberg, Springer-Verlag: 334-344

[25] SzegedyC, VanhouckeV, IoffeS, et al.. Rethinking the inception architecture for computer vision[C], 2016, New York, IEEE: 2818-2826