Embracing the overlooked: harnessing feature disentanglement for cross-domain learning

Abstract

Distributional shift between domains poses great challenges to modern machine learning algorithms. The domain generalization (DG) signifies a popular line targeting this issue, where these methods intend to uncover universal patterns across disparate distributions. Noted, the crucial challenge behind DG is the existence of irrelevant domain features, and most prior works overlook this information. Motivated by this, we propose a novel Contrastive-based Disentanglement method for Domain Generalization (CDDG), to effectively utilize the disentangled features to exploit the over-looked domain-specific features, and thus facilitate the extraction of the desired cross-domain category features for DG tasks. Specifically, CDDG learns to decouple inherent mutually exclusive features by leveraging them in the latent space, thus making the learning discriminative. Extensive experiments conducted on various benchmark datasets, including PACS, VLCS, Office-Home, TerraIncognita and DomainNet, have demonstrated the superiority of our method compared to other state-of-the-art approaches. Furthermore, visualization evaluations confirm the potential of our method in achieving effective feature disentanglement.

Graphical abstract

Keywords

computer vision / domain generalization / feature disentanglement

References

[1] Gulrajani I, Lopez-Paz D. In search of lost domain generalization. In: Proceedings of the 9th International Conference on Learning Representations. 2021

[2] Li D, Yang Y, Song Y Z, Hospedales T M. Deeper, broader and artier domain generalization. In: Proceedings of the IEEE International Conference on Computer Vision. 2017, 5543−5551

[3] Li Y F, Liang D M . Safe semi-supervised learning: a brief introduction. Frontiers of Computer Science, 2019, 13( 4): 669–676

[4] Zhou K, Liu Z, Qiao Y, Xiang T, Loy C C . Domain generalization: a survey. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2023, 45( 4): 4396–4415

[5] Wang J, Lan C, Liu C, Ouyang Y, Qin T, Lu W, Chen Y, Zeng W, Yu P S . Generalizing to unseen domains: a survey on domain generalization. IEEE Transactions on Knowledge and Data Engineering, 2023, 35( 8): 8052–8072

[6] Jia B B, Liu J Y, Hang J Y, Zhang M L . Learning label-specific features for decomposition-based multi-class classification. Frontiers of Computer Science, 2023, 17( 6): 176348

[7] Wang Y, Li H, Chau L P, Kot A C. Variational disentanglement for domain generalization. 2021, arXiv preprint arXiv: 2109.05826

[8] Cho J, Nam G, Kim S, Yang H, Kwak S. PromptStyler: Prompt-driven style generation for source-free domain generalization. In: Proceedings of IEEE/CVF International Conference on Computer Vision. 2023, 15656−15666

[9] Tian Y, Sun C, Poole B, Krishnan D, Schmid C, Isola P. What makes for good views for contrastive learning? In: Proceedings of the 34th International Conference on Neural Information Processing Systems. 2020, 573

[10] Yeh C H, Hong C Y, Hsu Y C, Liu T L, Chen Y, LeCun Y. Decoupled contrastive learning. In: Proceedings of the 17th European Conference on Computer Vision. 2022, 668−684

[11] Honarvar Nazari N, Kovashka A. Domain generalization using shape representation. In: Proceedings of the European Conference on Computer Vision. 2020, 666−670

[12] Zakharov S, Kehl W, Ilic S. DeceptionNet: network-driven domain randomization. In: Proceedings of the IEEE/CVF International Conference on Computer Vision. 2019, 532−541

[13] Somavarapu N, Ma C Y, Kira Z. Frustratingly simple domain generalization via image stylization. 2020, arXiv preprint arXiv: 2006.11207

[14] Zhou K, Yang Y, Qiao Y, Xiang T. Domain generalization with mixstyle. In: Proceedings of the 9th International Conference on Learning Representations. 2021

[15] Liu M, Pan J, Yan Z, Zuo W, Zhang L . Adaptive network combination for single-image reflection removal: a domain generalization perspective. Frontiers of Computer Science, 2025, 19( 1): 191703

[16] Nguyen T, Do K, Duong B, Nguyen T. Domain generalisation via risk distribution matching. In: Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision. 2024, 2778−2787

[17] Eastwood C, Robey A, Singh S, von Kugelgen J, Hassani H, Pappas G J, Schölkopf B. Probable domain generalization via quantile risk minimization. In: Proceedings of the 36th International Conference on Neural Information Processing Systems. 2022, 1261

[18] Wu K, Jia F, Han Y . Domain-specific feature elimination: multi-source domain adaptation for image classification. Frontiers of Computer Science, 2023, 17( 4): 174705

[19] Wang P, Zhang Z, Lei Z, Zhang L. Sharpness-aware gradient matching for domain generalization. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. 2023, 3769−3778

[20] Li S Y, Zhao S J, Cao Z T, Huang S J, Chen S . Robust domain adaptation with noisy and shifted label distribution. Frontiers of Computer Science, 2025, 19( 3): 193310

[21] Wang Y, Zhang W, Zhang M L. Partial label causal representation learning for instance-dependent supervision and domain generalization. In: Proceedings of the AAAI Conference on Artificial Intelligence. 2025, 21366−21374

[22] Li H, Pan S J, Wang S, Kot A C. Domain generalization with adversarial feature learning. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. 2018, 5400−5409

[23] Li H, Wang Y F, Wan R, Wang S, Li T Q, Kot A C. Domain generalization for medical imaging classification with linear-dependency regularization. In: Proceedings of the 34th International Conference on Neural Information Processing Systems. 2020, 262

[24] Zhang A, Wang H, Wang X, Chua T S. Disentangling masked autoencoders for unsupervised domain generalization. In: Proceedings of the 18th European Conference on Computer Vision. 2025, 126−151

[25] Makino T, Park J W, Tagasovska N, Kudo T, Coelho P, Huetter J C, Yao H, Hoeckendorf B, Leote A C, Ra S, Richmond D, Cho K, Regev A, Lopez R. Supervised contrastive block disentanglement. 2025, arXiv preprint arXiv: 2502.07281

[26] Cha J, Chun S, Lee K, Cho H C, Park S, Lee Y, Park S. SWAD: domain generalization by seeking flat minima. In: Proceedings of the 35th International Conference on Neural Information Processing Systems. 2021, 1716

[27] Li B, Shen Y, Yang J, Wang Y, Ren J, Che T, Zhang J, Liu Z. Sparse mixture-of-experts are domain generalizable learners. In: Proceedings of the 11th International Conference on Learning Representations. 2023

[28] Kim D, Yoo Y, Park S, Kim J, Lee J. SelfReg: self-supervised contrastive regularization for domain generalization. In: Proceedings of the IEEE/CVF International Conference on Computer Vision. 2021, 9599−9608

[29] Harary S, Schwartz E, Arbelle A, Staar P, Abu-Hussein S, Amrani E, Herzig R, Alfassy A, Giryes R, Kuehne H, Katabi D, Saenko K, Feris R, Karlinsky L. Unsupervised domain generalization by learning a bridge across domains. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. 2022, 5270−5280

[30] Chen T, Kornblith S, Norouzi M, Hinton G E. A simple framework for contrastive learning of visual representations. In: Proceedings of the 37th International Conference on Machine Learning. 2020, 1597−1607

[31] He K, Fan H, Wu Y, Xie S, Girshick R. Momentum contrast for unsupervised visual representation learning. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. 2020, 9726−9735

[32] Chen X, Fan H, Girshick R, He K. Improved baselines with momentum contrastive learning. 2020, arXiv preprint arXiv: 2003.04297

[33] Wang T, Isola P. Understanding contrastive representation learning through alignment and uniformity on the hypersphere. In: Proceedings of the 37th International Conference on Machine Learning. 2020, 9929−9939

[34] Yao X, Bai Y, Zhang X, Zhang Y, Sun Q, Chen R, Li R, Yu B. PCL: proxy-based contrastive learning for domain generalization. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. 2022, 7087−7097

[35] Jeon S, Hong K, Lee P, Lee J, Byun H. Feature stylization and domain-aware contrastive learning for domain generalization. In: Proceedings of the 29th ACM International Conference on Multimedia. 2021, 22−31

[36] Triantafillou E, Larochelle H, Zemel R S, Dumoulin V. Learning a universal template for few-shot dataset generalization. In: Proceedings of the 38th International Conference on Machine Learning. 2021, 10424−10433

[37] Lee S, Bae J, Kim H Y. Decompose, adjust, compose: effective normalization by playing with frequency for domain generalization. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. 2023, 11776−11785

[38] Qiao F, Zhao L, Peng X. Learning to learn single domain generalization. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. 2020, 12553−12562

[39] Jo S Y, Yoon S W. POEM: polarization of embeddings for domain-invariant representations. In: Proceedings of the 37th AAAI Conference on Artificial Intelligence. 2023, 8150−8158

[40] Khosla P, Teterwak P, Wang C, Sarna A, Tian Y, Isola P, Maschinot A, Liu C, Krishnan D. Supervised contrastive learning. In: Proceedings of the 34th International Conference on Neural Information Processing Systems. 2020, 1567

[41] Fang C, Xu Y, Rockmore D N. Unbiased metric learning: on the utilization of multiple datasets and web images for softening bias. In: Proceedings of the IEEE International Conference on Computer Vision. 2013, 1657−1664

[42] Venkateswara H, Eusebio J, Chakraborty S, Panchanathan S. Deep hashing network for unsupervised domain adaptation. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. 2017, 5385−5394

[43] Beery S, Van Horn G, Perona P. Recognition in terra incognita. In: Proceedings of the 15th European Conference on Computer Vision. 2018, 472−489

[44] Peng X, Bai Q, Xia X, Huang Z, Saenko K, Wang B. Moment matching for multi-source domain adaptation. In: Proceedings of the IEEE/CVF International Conference on Computer Vision. 2019, 1406−1415

[45] He K, Zhang X, Ren S, Sun J. Deep residual learning for image recognition. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. 2016, 770−778

[46] Vapnik V N . An overview of statistical learning theory. IEEE Transactions on Neural Networks, 1999, 10( 5): 988–999

[47] Sagawa S, Koh P W, Hashimoto T B, Liang P. Distributionally robust neural networks for group shifts: on the importance of regularization for worst-case generalization. 2019, arXiv preprint arXiv: 1911.08731

[48] Li D, Yang Y, Song Y Z, Hospedales T. Learning to generalize: meta-learning for domain generalization. In: Proceedings of the 32nd AAAI Conference on Artificial Intelligence. 2018, 427

[49] Yan S, Song H, Li N, Zou L, Ren L. Improve unsupervised domain adaptation with mixup training. 2020, arXiv preprint arXiv: 2001.00677

[50] Zhang M, Marklund H, Dhawan N, Gupta A, Levine S, Finn C. Adaptive risk minimization: learning to adapt to domain shift. In: Proceedings of the 35th International Conference on Neural Information Processing Systems. 2021, 1812

[51] Krueger D, Caballero E, Jacobsen J H, Zhang A, Binas J, Zhang D, Le Priol R, Courville A C. Out-of-distribution generalization via risk extrapolation (REx). In: Proceedings of the 38th International Conference on Machine Learning. 2021, 5815−5826

[52] Nam H, Lee H, Park J, Yoon W, Yoo D. Reducing domain gap by reducing style bias. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. 2021, 8686−8695

[53] Arjovsky M, Bottou L, Gulrajani I, Lopez-Paz D. Invariant risk minimization. 2019, arXiv preprint arXiv: 1907.02893

[54] Ganin Y, Ustinova E, Ajakan H, Germain P, Larochelle H, Laviolette F, Marchand M, Lempitsky V . Domain-adversarial training of neural networks. The Journal of Machine Learning Research, 2016, 17( 1): 2096–2030

[55] Li Y, Tian X, Gong M, Liu Y, Liu T, Zhang K, Tao D. Deep domain generalization via conditional invariant adversarial networks. In: Proceedings of the 15th European Conference on Computer Vision. 2018, 647−663

[56] Sun B, Saenko K. Deep CORAL: correlation alignment for deep domain adaptation. In: Proceedings of the European Conference on Computer Vision. 2016, 443−450

[57] Huang Z, Wang H, Xing E P, Huang D. Self-challenging improves cross-domain generalization. In: Proceedings of the 16th European Conference on Computer Vision. 2020, 124−140

[58] Cha J, Lee K, Park S, Chun S. Domain generalization by mutual-information regularization with pre-trained models. In: Proceedings of the 17th European Conference on Computer Vision. 2022, 440−457

[59] van der Maaten L, Hinton G. Visualizing data using t-SNE. Journal of Machine Learning Research, 2008, 9: 2579−2605

[60] Selvaraju R R, Cogswell M, Das A, Vedantam R, Parikh D, Batra D. Grad-CAM: visual explanations from deep networks via gradient-based localization. In: Proceedings of the IEEE International Conference on Computer Vision. 2017, 618−626

RIGHTS & PERMISSIONS

Higher Education Press