A comprehensive survey of federated transfer learning: challenges, methods and applications

Abstract

Federated learning (FL) is a novel distributed machine learning paradigm that enables participants to collaboratively train a centralized model with privacy preservation by eliminating the requirement of data sharing. In practice, FL often involves multiple participants and requires the third party to aggregate global information to guide the update of the target participant. Therefore, many FL methods do not work well due to the training and test data of each participant may not be sampled from the same feature space and the same underlying distribution. Meanwhile, the differences in their local devices (system heterogeneity), the continuous influx of online data (incremental data), and labeled data scarcity may further influence the performance of these methods. To solve this problem, federated transfer learning (FTL), which integrates transfer learning (TL) into FL, has attracted the attention of numerous researchers. However, since FL enables a continuous share of knowledge among participants with each communication round while not allowing local data to be accessed by other participants, FTL faces many unique challenges that are not present in TL. In this survey, we focus on categorizing and reviewing the current progress on federated transfer learning, and outlining corresponding solutions and applications. Furthermore, the common setting of FTL scenarios, available datasets, and significant related research are summarized in this survey.

Graphical abstract

Keywords

federated transfer learning / federated learning / transfer learning / survey

Wei Guo is a PhD student at the Institute of Artificial Intelligence, Beihang University, China. She received his MSc degree from the School of Electronics and Computer Science at Southampton University, UK. Her research interests primarily lie in federated learning and transfer learning

Fuzhen Zhuang received the BE degree from the College of Computer Science, Chongqing University, China in 2006, and the PhD degree in computer science from the Institute of Computing Technology, Chinese Academy of Sciences, China in 2011. He is currently a full professor with the Institute of Artificial Intelligence, Beihang University, China. He has published more than 140 papers in some prestigious refereed journals and conference proceedings. His research interests include transfer learning, machine learning, data mining, multitask learning, knowledge graph, and recommendation systems. He is a senior member of the CCF. He was a recipient of the Distinguished Dissertation Award of CAAI in 2013

Xiao Zhang is an associate professor at the School of Computer Science and Technology, Shandong University, China. His research interests include distributed learning, federated learning, edge intelligence, and data mining. He has published more than 20 papers in prestigious refereed journals and conference proceedings, such as IEEE TKDE, TMC, UBICOMP, SIGKDD, SIGIR, IJCAI, ACM CIKM, and IEEE ICDM

Yiqi Tong is a PhD student at the School of Computer Science and Engineering, Beihang University, China. He received his MSc degree from the School of Informatics at Xiamen University, China. His research interests primarily lie in natural language processing and recommendation systems

Jin Dong is the General Director of Beijing Academy of Blockchain and Edge Computing (BABEC), Director of National Blockchain Technology Innovation Center, Director of Beijing Advanced Innovation Center for Future Blockchain and Privacy Computing. He has been dedicated to technical research in the fields of blockchain, privacy computing, chip design, etc. The team led by him developed the first of kind high performance hardware-software integrated blockchain system - ChainMaker around the globe, aiming to break through the performance and security bottlenecks of large-scale blockchain applications. This has been widely adopted by a variety of key economic and industrial applications in China. Jin Dong received his PhD degree from Tsinghua University, China and has filed more than forty US patents

References

[1] McMahan B, Moore E, Ramage D, Hampson S, Arcas B A Y. Communication-efficient learning of deep networks from decentralized data. In: Proceedings of the 20th International Conference on Artificial Intelligence and Statistics. 2017, 1273−1282

[2] Feng S, Li B, Yu H, Liu Y, Yang Q . Semi-supervised federated heterogeneous transfer learning. Knowledge-Based Systems, 2022, 252: 109384

[3] Zhang C, Xie Y, Bai H, Yu B, Li W, Gao Y . A survey on federated learning. Knowledge-Based Systems, 2021, 216: 106775

[4] Gao L, Fu H, Li L, Chen Y, Xu M, Xu C Z. FedDC: Federated learning with non-IID data via local drift decoupling and correction. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. 2022, 10102−10111

[5] Shi Y, Zhang Y, Xiao Y, Niu L . Optimization strategies for client drift in federated learning: a review. Procedia Computer Science, 2022, 214: 1168–1173

[6] Liu Y, Kang Y, Zou T, Pu Y, He Y, Ye X, Ouyang Y, Zhang Y Q, Yang Q. Vertical federated learning: concepts, advances and challenges. 2022, arXiv preprint arXiv: 2211.12814

[7] Zhuang F, Qi Z, Duan K, Xi D, Zhu Y, Zhu H, Xiong H, He Q . A comprehensive survey on transfer learning. Proceedings of the IEEE, 2021, 109( 1): 43–76

[8] Yang Q, Liu Y, Chen T, Tong Y . Federated machine learning: concept and applications. ACM Transactions on Intelligent Systems and Technology, 2019, 10( 2): 12

[9] Rahman K M J, Ahmed F, Akhter N, Hasan M, Amin R, Aziz K E, Islam A K M M, Mukta M S H, Islam A K M N . Challenges, applications and design aspects of federated learning: a survey. IEEE Access, 2021, 9: 124682–124700

[10] Liu J, Huang J, Zhou Y, Li X, Ji S, Xiong H, Dou D . From distributed machine learning to federated learning: a survey. Knowledge and Information Systems, 2022, 64( 4): 885–917

[11] Li L, Fan Y, Lin K Y. A survey on federated learning. In: Proceedings of the16th IEEE International Conference on Control & Automation (ICCA). 2020, 791−796

[12] Zhan Y, Zhang J, Hong Z, Wu L, Li P, Guo S . A survey of incentive mechanism design for federated learning. IEEE Transactions on Emerging Topics in Computing, 2022, 10( 2): 1035–1044

[13] Yin X, Zhu Y, Hu J . A comprehensive survey of privacy-preserving federated learning: A taxonomy, review, and future directions. ACM Computing Surveys, 2022, 54( 6): 131

[14] Lyu L, Yu H, Yang Q. Threats to federated learning: a survey. 2020, arXiv preprint arXiv: 2003.02133

[15] Pfitzner B, Steckhan N, Arnrich B . Federated learning in a medical context: a systematic literature review. ACM Transactions on Internet Technology, 2021, 21( 2): 50

[16] Nguyen D C, Pham Q V, Pathirana P N, Ding M, Seneviratne A, Lin Z, Dobre O, Hwang W J . Federated learning for smart healthcare: a survey. ACM Computing Surveys, 2023, 55( 3): 60

[17] Lim W Y B, Luong N C, Hoang D T, Jiao Y, Liang Y C, Yang Q, Niyato D, Miao C . Federated learning in mobile edge networks: a comprehensive survey. IEEE Communications Surveys & Tutorials, 2020, 22( 3): 2031–2063

[18] Nguyen D C, Ding M, Pathirana P N, Seneviratne A, Li J, Poor H V . Federated learning for internet of things: a comprehensive survey. IEEE Communications Surveys & Tutorials, 2021, 23( 3): 1622–1658

[19] Zhu H, Xu J, Liu S, Jin Y . Federated learning on non-IID data: a survey. Neurocomputing, 2021, 465: 371–390

[20] Tan A Z, Yu H, Cui L, Yang Q . Towards personalized federated learning. IEEE Transactions on Neural Networks and Learning Systems, 2023, 34( 12): 9587–9603

[21] Pan S J, Yang Q . A survey on transfer learning. IEEE Transactions on Knowledge and Data Engineering, 2010, 22( 10): 1345–1359

[22] Konečný J, McMahan H B, Ramage D, Richtárik P. Federated optimization: distributed machine learning for on-device intelligence. 2016, arXiv preprint arXiv: 1610.02527

[23] Long M, Wang J, Sun J, Yu P S . Domain invariant transfer kernel learning. IEEE Transactions on Knowledge and Data Engineering, 2015, 27( 6): 1519–1532

[24] Long M, Cao Y, Wang J, Jordan M I. Learning transferable features with deep adaptation networks. In: Proceedings of the 32nd International Conference on Machine Learning. 2015, 97−105

[25] Bengio Y. Deep learning of representations for unsupervised and transfer learning. In: Proceedings of 2011 International Conference on Unsupervised and Transfer Learning Workshop. 2011, 17−37

[26] Raina R, Battle A, Lee H, Packer B, Ng A Y. Self-taught learning: transfer learning from unlabeled data. In: Proceedings of the 24th International Conference on Machine Learning. 2007, 759−766

[27] Younis R, Fisichella M . FLY-SMOTE: re-balancing the non-IID IoT edge devices data in federated learning system. IEEE Access, 2022, 10: 65092–65102

[28] Wu Q, Chen X, Zhou Z, Zhang J . FedHome: cloud-edge based personalized federated learning for in-home health monitoring. IEEE Transactions on Mobile Computing, 2022, 21( 8): 2818–2832

[29] Jeong E, Oh S, Kim H, Park J, Bennis M, Kim S L. Communication-efficient on-device machine learning: Federated distillation and augmentation under non-IID private data. 2018, arXiv preprint arXiv: 1811.11479

[30] Li A, Zhang L, Tan J, Qin Y, Wang J, Li X Y. Sample-level data selection for federated learning. In: Proceedings of the IEEE Conference on Computer Communications. 2021, 1−10

[31] Chen D, Hu J, Tan V J, Wei X, Wu E. Elastic aggregation for federated optimization. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. 2023, 12187−12197

[32] Chen H Y, Chao W L. Fedbe: Making Bayesian model ensemble applicable to federated learning. In: Proceedings of the 9th International Conference on Learning Representations. 2021

[33] Seo H, Park J, Oh S, Bennis M, Kim S L. Federated knowledge distillation. 2020, arXiv preprint arXiv: 2011.02367

[34] Gong B, Shi Y, Sha F, Grauman K. Geodesic flow kernel for unsupervised domain adaptation. In: Proceedings of 2012 IEEE Conference on Computer Vision and Pattern Recognition. 2012, 2066−2073

[35] Peng X, Bai Q, Xia X, Huang Z, Saenko K, Wang B. Moment matching for multi-source domain adaptation. In: Proceedings of the IEEE/CVF International Conference on Computer Vision. 2019, 1406−1415

[36] Kuznetsova A, Rom H, Alldrin N, Uijlings J, Krasin I, Pont-Tuset J, Kamali S, Popov S, Malloci M, Kolesnikov A, Duerig T, Ferrari V . The open images dataset V4: Unified image classification, object detection, and visual relationship detection at scale. International Journal of Computer Vision, 2020, 128( 7): 1956–1981

[37] Cassara P, Gotta A, Valerio L . Federated feature selection for cyber-physical systems of systems. IEEE Transactions on Vehicular Technology, 2022, 71( 9): 9937–9950

[38] Darlow L N, Crowley E J, Antoniou A, Storkey A J. CINIC-10 is not ImageNet or CIFAR-10. 2018, arXiv preprint arXiv: 1810.03505

[39] Liu Z, Luo P, Wang X, Tang X. Deep learning face attributes in the wild. In: Proceedings of the IEEE International Conference on Computer Vision. 2015, 3730−3738

[40] Reddi S J, Charles Z, Zaheer M, Garrett Z, Rush K, Konečný J, Kumar S, McMahan H B. Adaptive federated optimization. In: Proceedings of the 9th International Conference on Learning Representations. 2021

[41] Liu Q, Yang H, Dou Q, Heng P A. Federated semi-supervised medical image classification via inter-client relation matching. In: Proceedings of the 24th International Conference. 2021, 325−335

[42] Sattler F, Müller K R, Samek W . Clustered federated learning: model-agnostic distributed multitask optimization under privacy constraints. IEEE Transactions on Neural Networks and Learning Systems, 2021, 32( 8): 3710–3722

[43] Qayyum A, Ahmad K, Ahsan M A, Al-Fuqaha A, Qadir J . Collaborative federated learning for healthcare: multi-modal covid-19 diagnosis at the edge. IEEE Open Journal of the Computer Society, 2022, 3: 172–184

[44] Mansour Y, Mohri M, Ro J, Suresh A T. Three approaches for personalization with applications to federated learning. 2020, arXiv preprint arXiv: 2002.10619

[45] Huang L, Shea A L, Qian H, Masurkar A, Deng H, Liu D . Patient clustering improves efficiency of federated machine learning to predict mortality and hospital stay time using distributed electronic medical records. Journal of Biomedical Informatics, 2019, 99: 103291

[46] Ouyang X, Xie Z, Zhou J, Xing G, Huang J . ClusterFL: a clustering-based federated learning system for human activity recognition. ACM Transactions on Sensor Networks, 2023, 19( 1): 17

[47] Duan M, Liu D, Chen X, Liu R, Tan Y, Liang L . Self-balancing federated learning with global imbalanced data in mobile systems. IEEE Transactions on Parallel and Distributed Systems, 2021, 32( 1): 59–71

[48] Zhang X, Mavromatics A, Vafeas A, Nejabati R, Simeonidou D . Federated feature selection for horizontal federated learning in IoT networks. IEEE Internet of Things Journal, 2023, 10( 11): 10095–10112

[49] Hu Y, Zhang Y, Gao X, Gong D, Song X, Guo Y, Wang J . A federated feature selection algorithm based on particle swarm optimization under privacy protection. Knowledge-Based Systems, 2023, 260: 110122

[50] Jeong W, Yoon J, Yang E, Hwang S J. Federated semi-supervised learning with inter-client consistency & disjoint learning. In: Proceedings of the 9th International Conference on Learning Representations. 2021

[51] Li S, Zhou T, Tian X, Tao D. Learning to collaborate in decentralized learning of personalized models. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. 2022, 9756−9765

[52] Qi T, Wu F, Lyu L, Huang Y, Xie X. FedSampling: a better sampling strategy for federated learning. In: Proceedings of the 32nd International Joint Conference on Artificial Intelligence. 2023, 4154−4162

[53] Chen M, Yang Z, Saad W, Yin C, Poor H V, Cui S . A joint learning and communications framework for federated learning over wireless networks. IEEE Transactions on Wireless Communications, 2021, 20( 1): 269–283

[54] Deng Y, Lyu F, Ren J, Wu H, Zhou Y, Zhang Y, Shen X . AUCTION: automated and quality-aware client selection framework for efficient federated learning. IEEE Transactions on Parallel and Distributed Systems, 2022, 33( 8): 1996–2009

[55] Yang H H, Arafa A, Quek T Q S, Poor H V. Age-based scheduling policy for federated learning in mobile edge networks. In: Proceedings of 2020 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP). 2020, 8743−8747

[56] Su L, Zhou R, Wang N, Fang G, Li Z. An online learning approach for client selection in federated edge learning under budget constraint. In: Proceedings of the 51st International Conference on Parallel Processing. 2022, 72

[57] Wu C, Wu F, Lyu L, Huang Y, Xie X . Communication-efficient federated learning via knowledge distillation. Nature Communications, 2022, 13( 1): 2032

[58] Tuor T, Wang S, Ko B J, Liu C, Leung K K. Data selection for federated learning with relevant and irrelevant data at clients. 2020, arXiv preprint arXiv: 2001.08300

[59] Duan M, Liu D, Ji X, Liu R, Liang L, Chen X, Tan Y. FedGroup: Efficient clustered federated learning via decomposed data-driven measure. 2020, arXiv preprint arXiv: 2010.06870

[60] Nagalapatti L, Narayanam R. Game of gradients: mitigating irrelevant clients in federated learning. In: Proceedings of the 35th AAAI Conference on Artificial Intelligence. 2021, 9046−9054

[61] Yoon T, Shin S, Hwang S J, Yang E. FedMix: approximation of Mixup under mean augmented federated learning. In: Proceedings of the 9th International Conference on Learning Representations. 2021

[62] Hao W, El-Khamy M, Lee J, Zhang J, Liang K J, Chen C, Carin L. Towards fair federated learning with zero-shot data augmentation. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops (CVPRW). 2021, 3305−3314

[63] Liang P P, Liu T, Liu Z, Allen N B, Auerbach R P, Brent D, Salakhutdinov R, Morency L P. Think locally, act globally: federated learning with local and global representations. 2020, arXiv preprint arXiv: 2001.01523

[64] Briggs C, Fan Z, Andras P. Federated learning with hierarchical clustering of local updates to improve training on non-IID data. In: Proceedings of 2020 International Joint Conference on Neural Networks (IJCNN). 2020, 1−9

[65] Liu B, Guo Y, Chen X. PFA: privacy-preserving federated adaptation for effective model personalization. In: Proceedings of the Web Conference 2021. 2021, 923−934

[66] Tan Y, Long G, Liu L, Zhou T, Lu Q, Jiang J, Zhang C. FedProto: federated prototype learning across heterogeneous clients. In: Proceedings of the 36th AAAI Conference on Artificial Intelligence. 2022, 8432−8440

[67] Shen T, Zhang J, Jia X, Zhang F, Huang G, Zhou P, Kuang K, Wu F, Wu C. Federated mutual learning. 2020, arXiv preprint arXiv: 2006.16765

[68] Arivazhagan M G, Aggarwal V, Singh A K, Choudhary S. Federated learning with personalization layers. 2019, arXiv preprint arXiv: 1912.00818

[69] Fallah A, Mokhtari A, Ozdaglar A. Personalized federated learning with theoretical guarantees: a model-agnostic meta-learning approach. In: Proceedings of the 34th International Conference on Neural Information Processing Systems. 2020, 300

[70] Li T, Sahu A K, Talwalkar A, Smith V . Federated learning: challenges, methods, and future directions. IEEE Signal Processing Magazine, 2020, 37( 3): 50–60

[71] Deng Y, Kamani M M, Mahdavi M. Adaptive personalized federated learning. 2020, arXiv preprint arXiv: 2003.13461

[72] Dinh C T, Tran N H, Nguyen T D. Personalized federated learning with Moreau envelopes. In: Proceedings of the 34th International Conference on Neural Information Processing Systems. 2020, 1796

[73] Hanzely F, Richtárik P. Federated learning of a mixture of global and local models. 2020, arXiv preprint arXiv: 2002.05516

[74] Dinh C T, Tran N H, Nguyen T D, Bao W, Zomaya A Y, Zhou B B. Federated learning with proximal stochastic variance reduced gradient algorithms. In: Proceedings of the 49th International Conference on Parallel Processing. 2020, 48

[75] Hanzely F, Hanzely S, Horváth S, Richtárik P. Lower bounds and optimal algorithms for personalized federated learning. In: Proceedings of the 34th International Conference on Neural Information Processing Systems. 2020, 194

[76] Li T, Hu S, Beirami A, Smith V. Ditto: fair and robust federated learning through personalization. In: Proceedings of the 38th International Conference on Machine Learning. 2021, 6357−6368

[77] Karimireddy S P, Kale S, Mohri M, Reddi S J, Stich S U, Suresh A T. SCAFFOLD: stochastic controlled averaging for federated learning. In: Proceedings of the 37th International Conference on Machine Learning. 2020, 476

[78] Ma Z, Zhao M, Cai X, Jia Z . Fast-convergent federated learning with class-weighted aggregation. Journal of Systems Architecture, 2021, 117: 102125

[79] Collins L, Hassani H, Mokhtari A, Shakkottai S. Exploiting shared representations for personalized federated learning. In: Proceedings of the 38th International Conference on Machine Learning. 2021, 2089−2099

[80] Oh J, Kim S, Yun S Y. FedBABU: towards enhanced representation for federated image classification. 2021, arXiv preprint arXiv: 2106.06042

[81] Jang J, Ha H, Jung D, Yoon S. FedClassAvg: local representation learning for personalized federated learning on heterogeneous neural networks. In: Proceedings of the 51st International Conference on Parallel Processing. 2022, 76

[82] Zhuang W, Gan X, Wen Y, Zhang S, Yi S. Collaborative unsupervised visual representation learning from decentralized data. In: Proceedings of the IEEE/CVF International Conference on Computer Vision. 2021, 4892−4901

[83] Qu Z, Li X, Han X, Duan R, Shen C, Chen L. How to prevent the poor performance clients for personalized federated learning? In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. 2023, 12167−12176

[84] Wang K, He Q, Chen F, Chen C, Huang F, Jin H, Yang Y. FlexiFed: personalized federated learning for edge clients with heterogeneous model architectures. In: Proceedings of the ACM Web Conference 2023. 2023, 2979−2990

[85] Li A, Sun J, Zeng X, Zhang M, Li H, Chen Y. FedMask: joint computation and communication-efficient personalized federated learning via heterogeneous masking. In: Proceedings of the 19th ACM Conference on Embedded Networked Sensor Systems. 2021, 42−55

[86] Yang Z, Sun Q. Personalized heterogeneity-aware federated search towards better accuracy and energy efficiency. In: Proceedings of the 41st IEEE/ACM International Conference on Computer Aided Design. 2022, 1−9

[87] Diao E, Ding J, Tarokh V. HeteroFL: computation and communication efficient federated learning for heterogeneous clients. In: Proceedings of the 9th International Conference on Learning Representations. 2020

[88] Zeng H, Zhou T, Guo Y, Cai Z, Liu F. FedCav: contribution-aware model aggregation on distributed heterogeneous data in federated learning. In: Proceedings of the 50th International Conference on Parallel Processing. 2021, 75

[89] Zhu J, Ma X, Blaschko M B. Confidence-aware personalized federated learning via variational expectation maximization. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. 2023, 24542−24551

[90] Lin X, Chen H, Xu Y, Xu C, Gui X, Deng Y, Wang Y. Federated learning with positive and unlabeled data. In: Proceedings of the 39th International Conference on Machine Learning. 2022, 13344−13355

[91] Beilharz J, Pfftzner B, Schmid R, Geppert P, Arnrich B, Polze A. Implicit model specialization through dag-based decentralized federated learning. In: Proceedings of the 22nd International Middleware Conference. 2021, 310−322

[92] Zhang M, Sapra K, Fidler S, Yeung S, Álvarez J M. Personalized federated learning with first order model optimization. In: Proceedings of the 9th International Conference on Learning Representations. 2021

[93] Liu J, Wu J, Chen J, Hu M, Zhou Y, Wu D. FedDWA: personalized federated learning with dynamic weight adjustment. In: Proceedings of the 32nd International Joint Conference on Artificial Intelligence. 2023, 3993−4001

[94] Wang H, Kaplan Z, Niu D, Li B. Optimizing federated learning on Non-IID data with reinforcement learning. In: Proceedings of the IEEE Conference on Computer Communications. 2020, 1698−1707

[95] Nishio T, Yonetani R. Client selection for federated learning with heterogeneous resources in mobile edge. In: Proceedings of 2019 IEEE International Conference on Communications (ICC). 2019, 1−7

[96] Wang L, Wang W, Li B. CMFL: mitigating communication overhead for federated learning. In: Proceedings of the 39th International Conference on Distributed Computing Systems (ICDCS). 2019, 954−964

[97] Xia W, Quek T Q S, Guo K, Wen W, Yang H H, Zhu H . Multi-armed bandit-based client scheduling for federated learning. IEEE Transactions on Wireless Communications, 2020, 19( 11): 7108–7123

[98] Yang M, Wang X, Zhu H, Wang H, Qian H. Federated learning with class imbalance reduction. In: Proceedings of the 29th European Signal Processing Conference (EUSIPCO). 2021, 2174−2178

[99] Chai Z, Ali A, Zawad S, Truex S, Anwar A, Baracaldo N, Zhou Y, Ludwig H, Yan F, Cheng Y. TiFL: a tier-based federated learning system. In: Proceedings of the 29th International Symposium on High-Performance Parallel and Distributed Computing. 2020, 125−136

[100] Li L, Duan M, Liu D, Zhang Y, Ren A, Chen X, Tan Y, Wang C. FedSAE: a novel self-adaptive federated learning framework in heterogeneous systems. In: Proceedings of the International Joint Conference on Neural Networks. 2021, 1−10

[101] Cox B, Chen L Y, Decouchant J. Aergia: leveraging heterogeneity in federated learning systems. In: Proceedings of the 23rd ACM/IFIP International Middleware Conference. 2022, 107−120

[102] Dong J, Wang L, Fang Z, Sun G, Xu S, Wang X, Zhu Q. Federated class-incremental learning. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. 2022, 10154−10163

[103] Makhija D, Han X, Ho N, Ghosh J. Architecture agnostic federated learning for neural networks. In: Proceedings of the 39th International Conference on Machine Learning. 2022, 14860−14870

[104] Huang Y, Chu L, Zhou Z, Wang L, Liu J, Pei J, Zhang Y. Personalized cross-silo federated learning on non-IID data. In: Proceedings of the 35th AAAI Conference on Artificial Intelligence. 2021, 7865−7873

[105] Zhu S, Qi Q, Zhuang Z, Wang J, Sun H, Liao J. FedNKD: a dependable federated learning using fine-tuned random noise and knowledge distillation. In: Proceedings of 2022 International Conference on Multimedia Retrieval. 2022, 185−193

[106] Chai Z, Chen Y, Anwar A, Zhao L, Cheng Y, Rangwala H. FedAT: a high-performance and communication-efficient federated learning system with asynchronous tiers. In: Proceedings of the International Conference for High Performance Computing, Networking, Storage and Analysis. 2021, 1−17

[107] Hu C, Liang H H, Han X M, Liu B A, Cheng D Z, Wang D. Spread: decentralized model aggregation for scalable federated learning. In: Proceedings of the 51st International Conference on Parallel Processing. 2022, 75

[108] Zhang X, Li Y, Li W, Guo K, Shao Y. Personalized federated learning via variational Bayesian inference. In: Proceedings of the 39th International Conference on Machine Learning. 2022, 26293−26310

[109] Nguyen N H, Le Nguyen P, Nguyen T D, Nguyen T T, Nguyen D L, Nguyen T H, Pham H H, Truong T N. FedDRL: deep reinforcement learning-based adaptive aggregation for non-IID data in federated learning. In: Proceedings of the 51st International Conference on Parallel Processing. 2022, 73

[110] Yoon J, Jeong W, Lee G, Yang E, Hwang S J. Federated continual learning with weighted inter-client transfer. In: Proceedings of the 38th International Conference on Machine Learning. 2021, 12073−12086

[111] Qu Z, Duan R, Chen L, Xu J, Lu Z, Liu Y . Context-Aware online client selection for hierarchical federated learning. IEEE Transactions on Parallel and Distributed Systems, 2022, 33( 12): 4353–4367

[112] Marfoq O, Neglia G, Vidal R, Kameni L. Personalized federated learning through local memorization. In: Proceedings of the 39th International Conference on Machine Learning. 2022, 15070−15092

[113] Huang W, Ye M, Du B, Gao X. Few-shot model agnostic federated learning. In: Proceedings of the 30th ACM International Conference on Multimedia. 2022, 7309−7316

[114] Itahara S, Nishio T, Koda Y, Morikura M, Yamamoto K . Distillation-based semi-supervised federated learning for communication-efficient collaborative training with non-IID private data. IEEE Transactions on Mobile Computing, 2023, 22( 1): 191–205

[115] Zhang J, Guo S, Guo J, Zeng D, Zhou J, Zomaya A Y . Towards data-independent knowledge transfer in model-heterogeneous federated learning. IEEE Transactions on Computers, 2023, 72( 10): 2888–2901

[116] Zhuang W, Wen Y, Zhang S. Divergence-aware federated self-supervised learning. In: Proceedings of the 10th ACM International Conference on Multimedia. 2022

[117] Wang Y, Xu H, Ali W, Li M, Zhou X, Shao J . FedFTHA: a fine-tuning and head aggregation method in federated learning. IEEE Internet of Things Journal, 2023, 10( 14): 12749–12762

[118] Gong X, Sharma A, Karanam S, Wu Z, Chen T, Doermann D, Innanje A. Preserving privacy in federated learning with ensemble cross-domain knowledge distillation. In: Proceedings of the 36th AAAI Conference on Artificial Intelligence. 2022, 11891−11899

[119] Lin T, Kong L, Stich S U, Jaggi M. Ensemble distillation for robust model fusion in federated learning. In: Proceedings of the 34th International Conference on Neural Information Processing Systems. 2020, 198

[120] Lin H, Lou J, Xiong L, Shahabi C. SemiFed: semi-supervised federated learning with consistency and pseudo-labeling. 2021, arXiv preprint arXiv: 2108.09412

[121] Lubana E S, Tang C I, Kawsar F, Dick R P, Mathur A. Orchestra: unsupervised federated learning via globally consistent clustering. In: Proceedings of the 39th International Conference on Machine Learning. 2022, 14461−14484

[122] Liang X, Lin Y, Fu H, Zhu L, Li X. RSCfed: random sampling consensus federated semi-supervised learning. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. 2022, 10144−10153

[123] Li M, Li Q, Wang Y. Class balanced adaptive pseudo labeling for federated semi-supervised learning. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. 2023, 16292−16301

[124] Zhang L, Wu D, Yuan X. FedZKT: zero-shot knowledge transfer towards resource-constrained federated learning with heterogeneous on-device models. In: Proceedings of the 42nd International Conference on Distributed Computing Systems (ICDCS). 2022, 928−938

[125] Zhou T, Konukoglu E. FedFA: federated feature augmentation. In: Proceedings of the 11th International Conference on Learning Representations. 2023

[126] Yue K, Jin R, Pilgrim R, Wong C W, Baron D, Dai H. Neural tangent kernel empowered federated learning. In: Proceedings of the 39th International Conference on Machine Learning. 2022, 25783−25803

[127] Long G, Xie M, Shen T, Zhou T, Wang X, Jiang J . Multi-center federated learning: clients clustering for better personalization. World Wide Web, 2023, 26( 1): 481–500

[128] Li Q, He B, Song D. Model-contrastive federated learning. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. 2021, 10708−10717

[129] Chen H Y, Chao W L. On bridging generic and personalized federated learning for image classification. In: Proceedings of the 10th International Conference on Learning Representations. 2022

[130] Yao X, Sun L. Continual local training for better initialization of federated models. In: Proceedings of 2020 IEEE International Conference on Image Processing (ICIP). 2020, 1736−1740

[131] Wang J, Liu Q, Liang H, Joshi G, Poor H V. Tackling the objective inconsistency problem in heterogeneous federated optimization. In: Proceedings of the 34th International Conference on Neural Information Processing Systems. 2020, 638

[132] Yu S, Nguyen P, Abebe W, Qian W, Anwar A, Jannesari A. SPATL: salient parameter aggregation and transfer learning for heterogeneous federated learning. In: Proceedings of the International Conference for High Performance Computing, Networking, Storage and Analysis. 2022, 1−14

[133] Liu C, Yang Y, Cai X, Ding Y, Lu H. Completely heterogeneous federated learning. 2022, arXiv preprint arXiv: 2210.15865

[134] Ilhan F, Su G, Liu L. ScaleFL: Resource-adaptive federated learning with heterogeneous clients. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. 2023, 24532−24541

[135] Liu C, Lou C, Wang R, Xi A Y, Shen L, Yan J. Deep neural network fusion via graph matching with applications to model ensemble and federated learning. In: Proceedings of the 39th International Conference on Machine Learning. 2022, 13857−13869

[136] Cho Y J, Wang J, Joshi G. Client selection in federated learning: convergence analysis and power-of-choice selection strategies. 2020, arXiv preprint arXiv: 2010.01243

[137] Huang T, Lin W, Wu W, He L, Li K, Zomaya A Y . An efficiency-boosting client selection scheme for federated learning with fairness guarantee. IEEE Transactions on Parallel and Distributed Systems, 2021, 32( 7): 1552–1564

[138] Wang H, Li Y, Xu W, Li R, Zhan Y, Zeng Z. DaFKD: domain-aware federated knowledge distillation. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. 2023, 20412−20421

[139] Gong X, Song L, Vedula R, Sharma A, Zheng M, Planche B, Innanje A, Chen T, Yuan J, Doermann D, Wu Z Y . Federated learning with privacy-preserving ensemble attention distillation. IEEE Transactions on Medical Imaging, 2023, 42( 7): 2057–2067

[140] Li Q, He B, Song D. Practical one-shot federated learning for cross-silo setting. In: Proceedings of the 30th International Joint Conference on Artificial Intelligence. 2021, 1484−1490

[141] Sattler F, Marban A, Rischke R, Samek W. Communication-efficient federated distillation. 2020, arXiv preprint arXiv: 2012.00632

[142] Zhu Z, Hong J, Zhou J. Data-free knowledge distillation for heterogeneous federated learning. In: Proceedings of the 38th International Conference on Machine Learning. 2021, 12878−12889

[143] Zhang L, Shen L, Ding L, Tao D, Duan L Y. Fine-tuning global model via data-free knowledge distillation for non-IID federated learning. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. 2022, 10164−10173

[144] Yang Y, Yang R, Peng H, Li Y, Li T, Liao Y, Zhou P. FedACK: federated adversarial contrastive knowledge distillation for cross-lingual and cross-model social bot detection. In: Proceedings of the ACM Web Conference 2023. 2023, 1314−1323

[145] Li G, Hu Y, Zhang M, Liu J, Yin Q, Peng Y, Dou D. FedHiSyn: a hierarchical synchronous federated learning framework for resource and data heterogeneity. In: Proceedings of the 51st International Conference on Parallel Processing. 2022, 8

[146] Han S, Park S, Wu F, Kim S, Wu C, Xie X, Cha M. FedX: unsupervised federated learning with cross knowledge distillation. In: Proceedings of the 17th European Conference on Computer Vision. 2022, 691−707

[147] Li C, Zeng X, Zhang M, Cao Z. PyramidFL: a fine-grained client selection framework for efficient federated learning. In: Proceedings of the 28th Annual International Conference on Mobile Computing and Networking. 2022, 158−171

[148] Zhang S, Li Z, Chen Q, Zheng W, Leng J, Guo M. Dubhe: towards data unbiasedness with homomorphic encryption in federated learning client selection. In: Proceedings of the 50th International Conference on Parallel Processing. 2021, 83

[149] Chen H, Frikha A, Krompass D, Gu J, Tresp V. FRAug: tackling federated learning with non-IID features via representation augmentation. In: Proceedings of the IEEE/CVF International Conference on Computer Vision. 2023, 4826−4836

[150] Liu Q, Chen C, Qin J, Dou Q, Heng P A. FedDG: federated domain generalization on medical image segmentation via episodic learning in continuous frequency space. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. 2021, 1013−1023

[151] Li X, Jiang M, Zhang X, Kamp M, Dou Q. FedBN: federated learning on non-IID features via local batch normalization. In: Proceedings of the 9th International Conference on Learning Representations. 2021

[152] Yang D, Xu Z, Li W, Myronenko A, Roth H R, Harmon S, Xu S, Turkbey B, Turkbey E, Wang X, Zhu W, Carrafiello G, Patella F, Cariati M, Obinata H, Mori H, Tamura K, An P, Wood B J, Xu D . Federated semi-supervised learning for COVID region segmentation in chest CT using multi-national data from China, Italy, Japan. Medical Image Analysis, 2021, 70: 101992

[153] Wang H, Zhao H, Wang Y, Yu T, Gu J, Gao J. FedKC: federated knowledge composition for multilingual natural language understanding. In: Proceedings of the ACM Web Conference 2022. 2022, 1839−1850

[154] Wang K, Mathews R, Kiddon C, Eichner H, Beaufays F, Ramage D. Federated evaluation of on-device personalization. 2019, arXiv preprint arXiv: 1910.10252

[155] Pillutla K, Malik K, Mohamed A, Rabbat M G, Sanjabi M, Xiao L. Federated learning with partial model personalization. In: Proceedings of the 39th International Conference on Machine Learning. 2022, 17716−17758

[156] Li A, Sun J, Li P, Pu Y, Li H, Chen Y. Hermes: an efficient federated learning framework for heterogeneous mobile clients. In: Proceedings of the 27th Annual International Conference on Mobile Computing and Networking. 2021, 420−437

[157] Zhuang W, Wen Y, Zhang S. Joint optimization in edge-cloud continuum for federated unsupervised person re-identification. In: Proceedings of the 29th ACM International Conference on Multimedia. 2021, 433−441

[158] Zhang R, Xu Q, Yao J, Zhang Y, Tian Q, Wang Y. Federated domain generalization with generalization adjustment. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. 2023, 3954−3963

[159] Ruan Y, Joe-Wong C. FedSoft: soft clustered federated learning with proximal local updating. In: Proceedings of the 36th AAAI Conference on Artificial Intelligence. 2022, 8124−8131

[160] Xie H, Xiong L, Yang C. Federated node classification over graphs with latent link-type heterogeneity. In: Proceedings of the ACM Web Conference 2023. 2023, 556−566

[161] Donahue K, Kleinberg J M. Optimality and stability in federated learning: a game-theoretic approach. In: Proceedings of the International Conference on Neural Information Processing Systems. 2021, 1287−1298

[162] Dai Z, Low B K H, Jaillet P. Federated Bayesian optimization via Thompson sampling. In: Proceedings of the 34th International Conference on Neural Information Processing Systems. 2020, 812

[163] Li D, Wang J. FedMD: Heterogenous federated learning via model distillation. 2019, arXiv preprint arXiv: 1910.03581

[164] Li Y, Zhou W, Wang H, Mi H, Hospedales T M. FedH2L: federated learning with model and statistical heterogeneity. 2021, arXiv preprint arXiv: 2101.11296

[165] Wu Y, Kang Y, Luo J, He Y, Fan L, Pan R, Yang Q. FedCG: leverage conditional GAN for protecting privacy and maintaining competitive performance in federated learning. In: Proceedings of the 31st International Joint Conference on Artificial Intelligence. 2022, 2334−2340

[166] Niu Z, Wang H, Sun H, Ouyang S, Chen Y W, Lin L. MCKD: Mutually collaborative knowledge distillation for federated domain adaptation and generalization. In: Proceedings of 2023 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP). 2023, 1−5

[167] Huang W, Ye M, Shi Z, Li H, Du B. Rethinking federated learning with domain shift: a prototype view. In: Proceedings of 2023 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR). 2023, 16312−16322

[168] Wang H, Yurochkin M, Sun Y, Papailiopoulos D S, Khazaeni Y. Federated learning with matched averaging. In: Proceedings of the 8th International Conference on Learning Representations. 2020

[169] Ghosh A, Chung J, Yin D, Ramchandran K. An efficient framework for clustered federated learning. In: Proceedings of the 34th International Conference on Neural Information Processing Systems. 2020, 1643

[170] Tu L, Ouyang X, Zhou J, He Y, Xing G. FedDL: federated learning via dynamic layer sharing for human activity recognition. In: Proceedings of the 19th ACM Conference on Embedded Networked Sensor Systems. 2021, 15−28

[171] Donahue K, Kleinberg J. Model-sharing games: Analyzing federated learning under voluntary participation. In: Proceedings of the 35th AAAI Conference on Artificial Intelligence. 2021, 5303−5311

[172] Gao D, Liu Y, Huang A, Ju C, Yu H, Yang Q. Privacy-preserving heterogeneous federated transfer learning. In: Proceedings of 2019 IEEE International Conference on Big Data (Big Data). 2019, 2552−2559

[173] Kang Y, He Y, Luo J, Fan T, Liu Y, Yang Q. Privacy-preserving federated adversarial domain adaptation over feature groups for interpretability. IEEE Transactions on Big Data, 2022, doi: 10.1109/TBDATA.2022.3188292

[174] Fu R, Wu Y, Xu Q, Zhang M . FEAST: a communication-efficient federated feature selection framework for relational data. Proceedings of the ACM on Management of Data, 2023, 1( 1): 107

[175] Banerjee S, Elmroth E, Bhuyan M. Fed-FiS: a novel information-theoretic federated feature selection for learning stability. In: Proceedings of the 28th International Conference on Neural Information Processing. 2021, 480−487

[176] Wu Z, Li Q, He B . Practical vertical federated learning with unsupervised representation learning. IEEE Transactions on Big Data, 2022,

[177] He Y, Kang Y, Zhao X, Luo J, Fan L, Han Y, Yang Q. A hybrid self-supervised learning framework for vertical federated learning. 2022, arXiv preprint arXiv: 2208.08934

[178] Feng S, Yu H. Multi-participant multi-class vertical federated learning. 2020, arXiv preprint arXiv: 2001.11154

[179] Feng S . Vertical federated learning-based feature selection with non-overlapping sample utilization. Expert Systems with Applications, 2022, 208: 118097

[180] Jiang J, Burkhalter L, Fu F, Ding B, Du B, Hithnawi A, Li B, Zhang C. VF-PS: how to select important participants in vertical federated learning, efficiently and securely? In: Proceedings of the 36th International Conference on Neural Information Processing Systems. 2022, 152

[181] Castiglia T, Zhou Y, Wang S, Kadhe S, Baracaldo N, Patterson S. LESS-VFL: Communication-efficient feature selection for vertical federated learning. In: Proceedings of the 40th International Conference on Machine Learning. 2023, 3757−3781

[182] Kairouz P, McMahan H B, Avent B, Bellet A, Bennis M, . . Advances and open problems in federated learning. Foundations and Trends® in Machine Learning, 2021, 14( 1−2): 1–210

[183] Chai Z, Fayyaz H, Fayyaz Z, Anwar A, Zhou Y, Baracaldo N, Ludwig H, Cheng Y. Towards taming the resource and data heterogeneity in federated learning. In: Proceedings of 2019 USENIX Conference on Operational Machine Learning. 2019, 19−21

[184] Ye M, Fang X, Du B, Yuen P C, Tao D . Heterogeneous federated learning: state-of-the-art and research challenges. ACM Computing Surveys, 2024, 56( 3): 79

[185] Schlegel R, Kumar S, Rosnes E, Amat A G I . CodedPaddedFL and CodedSecAgg: Straggler mitigation and secure aggregation in federated learning. IEEE Transactions on Communications, 2023, 71( 4): 2013–2027

[186] You C, Xiang J, Su K, Zhang X, Dong S, Onofrey J, Staib L, Duncan J S. Incremental learning meets transfer learning: application to multi-site prostate MRI segmentation. In: Proceedings of the 3rd MICCAI Workshop on Distributed, Collaborative, and Federated Learning, and Affordable AI and Healthcare for Resource Diverse Global Health. 2022, 3−16

[187] Chen Y, Qin X, Wang J, Yu C, Gao W . FedHealth: a federated transfer learning framework for wearable healthcare. IEEE Intelligent Systems, 2020, 35( 4): 83–93

[188] Ditzler G, Polikar R . Incremental learning of concept drift from streaming imbalanced data. IEEE Transactions on Knowledge and Data Engineering, 2013, 25( 10): 2283–2301

[189] Elwell R, Polikar R . Incremental learning of concept drift in nonstationary environments. IEEE Transactions on Neural Networks, 2011, 22( 10): 1517–1531

[190] Tan D S, Lin Y X, Hua K L . Incremental learning of multi-domain image-to-image translations. IEEE Transactions on Circuits and Systems for Video Technology, 2021, 31( 4): 1526–1539

[191] Huang Y, Bert C, Gomaa A, Fietkau R, Maier A, Putz F. A survey of incremental transfer learning: combining peer-to-peer federated learning and domain incremental learning for multicenter collaboration. 2023, arXiv preprint arXiv: 2309.17192

[192] Tang J, Lin K Y, Li L . Using domain adaptation for incremental SVM classification of drift data. Mathematics, 2022, 10( 19): 3579

[193] Alam S, Liu L, Yan M, Zhang M. FedRolex: model-heterogeneous federated learning with rolling sub-model extraction. In: Proceedings of the 36th International Conference on Neural Information Processing Systems. 2022, 2152

[194] Yu F, Zhang W, Qin Z, Xu Z, Wang D, Liu C, Tian Z, Chen X. Heterogeneous federated learning. 2020, arXiv preprint arXiv: 2008.06767

[195] Jin Y, Wei X, Liu Y, Yang Q. Towards utilizing unlabeled data in federated learning: a survey and prospective. 2020, arXiv preprint arXiv: 2002.11545

[196] Diao E, Ding J, Tarokh V. SemiFL: semi-supervised federated learning for unlabeled clients with alternate training. In: Proceedings of the 36th International Conference on Neural Information Processing Systems. 2022, 1299

[197] Shin J, Li Y, Liu Y, Lee S J. FedBalancer: data and pace control for efficient federated learning on heterogeneous clients. In: Proceedings of the 20th Annual International Conference on Mobile Systems, Applications and Services. 2022, 436−449

[198] Pilla L L. Optimal task assignment for heterogeneous federated learning devices. In: Proceedings of 2021 IEEE International Parallel and Distributed Processing Symposium (IPDPS). 2021, 661−670

[199] Zhang F, Kuang K, You Z, Shen T, Xiao J, Zhang Y, Wu C, Zhuang Y, Li X. Federated unsupervised representation learning. 2020, arXiv preprint arXiv: 2010.08982

[200] Liao Y, Ma L, Zhou B, Zhao X, Xie F . DraftFed: a draft-based personalized federated learning approach for heterogeneous convolutional neural networks. IEEE Transactions on Mobile Computing, 2024, 23( 5): 3938–3949

[201] Liu Y, Guo S, Zhang J, Zhou Q, Wang Y, Zhao X. Feature correlation-guided knowledge transfer for federated self-supervised learning. 2022, arXiv preprint arXiv: 2211.07364

[202] Li T, Sahu A K, Zaheer M, Sanjabi M, Talwalkar A, Smith V. Federated optimization in heterogeneous networks. In: Proceedings of the Machine Learning and Systems. 2020, 429−450

[203] Duan J H, Li W, Zou D, Li R, Lu S. Federated learning with data-agnostic distribution fusion. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. 2023, 8074−8083

[204] Liu X, Xi W, Li W, Xu D, Bai G, Zhao J . Co-MDA: federated multisource domain adaptation on black-box models. IEEE Transactions on Circuits and Systems for Video Technology, 2023, 33( 12): 7658–7670

[205] Yoon J, Park G, Jeong W, Hwang S J. Bitwidth heterogeneous federated learning with progressive weight dequantization. In: Proceedings of the 39th International Conference on Machine Learning. 2022, 25552−25565

[206] Dai Z, Low B K H, Jaillet P. Differentially private federated Bayesian optimization with distributed exploration. In: Proceedings of the International Conference on Neural Information Processing Systems. 2021, 9125−9139

[207] Zhu H, Wang X, Jin Y . Federated many-task Bayesian optimization. IEEE Transactions on Evolutionary Computation, 2023,

[208] Chawla N V, Bowyer K W, Hall L O, Kegelmeyer W P . SMOTE: synthetic minority over-sampling technique. Journal of Artificial Intelligence Research, 2002, 16: 321–357

[209] He H, Bai Y, Garcia E A, Li S. ADASYN: Adaptive synthetic sampling approach for imbalanced learning. In: Proceedings of 2008 IEEE International Joint Conference on Neural Networks (IEEE World Congress on Computational Intelligence). 2008, 1322−1328

[210] Han H, Wang W Y, Mao B H. Borderline-smote: a new over-sampling method in imbalanced data sets learning. In: Proceedings of the International Conference on Intelligent Computing. 2005, 878−887

[211] Kubat M, Matwin S. Addressing the curse of imbalanced training sets: one-sided selection. In: Proceedings of the 14th International Conference on Machine Learning. 1997, 179−186

[212] Yen S J, Lee Y S . Cluster-based under-sampling approaches for imbalanced data distributions. Expert Systems with Applications, 2009, 36( 3): 5718–5727

[213] Tsai C F, Lin W C, Hu Y H, Yao G T . Under-sampling class imbalanced datasets by combining clustering analysis and instance selection. Information Sciences, 2019, 477: 47–54

[214] Zhang L, Li Y, Xiao X, Li X Y, Wang J, Zhou A, Li Q. CrowdBuy: privacy-friendly image dataset purchasing via crowdsourcing. In: Proceedings of the IEEE Conference on Computer Communications. 2018, 2735−2743

[215] Li A, Zhang L, Qian J, Xiao X, Li X Y, Xie Y. TODQA: efficient task-oriented data quality assessment. In: Proceedings of the 15th International Conference on Mobile Ad-Hoc and Sensor Networks (MSN). 2019, 81−88

[216] Katharopoulos A, Fleuret F. Not all samples are created equal: deep learning with importance sampling. In: Proceedings of the 35th International Conference on Machine Learning. 2018, 2530−2539

[217] Alain G, Lamb A, Sankar C, Courville A, Bengio Y. Variance reduction in SGD by distributed importance sampling. 2015, arXiv preprint arXiv: 1511.06481

[218] Loshchilov I, Hutter F. Online batch selection for faster training of neural networks. 2015, arXiv preprint arXiv: 1511.06343

[219] Schaul T, Quan J, Antonoglou I, Silver D. Prioritized experience replay. In: Proceedings of the 4th International Conference on Learning Representations. 2016

[220] Wu C Y, Manmatha R, Smola A J, Krahenbuhl P. Sampling matters in deep embedding learning. In: Proceedings of the IEEE International Conference on Computer Vision. 2017, 2859−2867

[221] Li K, Xiao C . CBFL: a communication-efficient federated learning framework from data redundancy perspective. IEEE Systems Journal, 2022, 16( 4): 5572–5583

[222] Duan L, Tsang I W, Xu D, Chua T S. Domain adaptation from multiple sources via auxiliary classifiers. In: Proceedings of the 26th Annual International Conference on Machine Learning. 2009, 289−296

[223] Duan L, Xu D, Tsang I W H . Domain adaptation from multiple sources: a domain-dependent regularization approach. IEEE Transactions on Neural Networks and Learning Systems, 2012, 23( 3): 504–518

[224] Zhuang F, Luo P, Xiong H, Xiong Y, He Q, Shi Z . Cross-domain learning from multiple sources: a consensus regularization perspective. IEEE Transactions on Knowledge and Data Engineering, 2010, 22( 12): 1664–1678

[225] Luo P, Zhuang F, Xiong H, Xiong Y, He Q. Transfer learning from multiple source domains via consensus regularization. In: Proceedings of the 17th ACM Conference on Information and Knowledge Management. 2008, 103−112

[226] Kirkpatrick J, Pascanu R, Rabinowitz N, Veness J, Desjardins G, Rusu A A, Milan K, Quan J, Ramalho T, Grabska-Barwinska A, Hassabis D, Clopath C, Kumaran D, Hadsell R . Overcoming catastrophic forgetting in neural networks. Proceedings of the National Academy of Sciences of the United States of America, 2017, 114( 13): 3521–3526

[227] Yu H, Zhang N, Deng S, Yuan Z, Jia Y, Chen H. The devil is the classifier: investigating long tail relation classification with decoupling analysis. 2020, arXiv preprint arXiv: 2009.07022

[228] Kang B, Xie S, Rohrbach M, Yan Z, Gordo A, Feng J, Kalantidis Y. Decoupling representation and classifier for long-tailed recognition. In: Proceedings of the 8th International Conference on Learning Representations. 2020

[229] Yosinski J, Clune J, Bengio Y, Lipson H. How transferable are features in deep neural networks? In: Proceedings of the 27th International Conference on Neural Information Processing Systems. 2014, 3320−3328

[230] Devlin J, Chang M W, Lee K, Toutanova K. BERT: pre-training of deep bidirectional transformers for language understanding. In: Proceedings of 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies. 2019, 4171−4186

[231] Li X, Huang K, Yang W, Wang S, Zhang Z. On the convergence of FedAvg on non-IID data. In: Proceedings of the 8th International Conference on Learning Representations. 2020

[232] Yamada Y, Lindenbaum O, Negahban S, Kluger Y. Feature selection using stochastic gates. In: Proceedings of the 37th International Conference on Machine Learning. 2020, 987

[233] Zhou K, Yang Y, Qiao Y, Xiang T. Domain generalization with mixstyle. In: Proceedings of the 9th International Conference on Learning Representations. 2021

[234] Li Q, Huang J, Hu J, Gong S. Feature-distribution perturbation and calibration for generalized Reid. In: Proceedings of 2024 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP). 2024, 2880−2884

[235] Huang X, Belongie S. Arbitrary style transfer in real-time with adaptive instance normalization. In: Proceedings of the IEEE International Conference on Computer Vision. 2017, 1510−1519

[236] Han B, Yao Q, Yu X, Niu G, Xu M, Hu W, Tsang I W, Sugiyama M. Co-teaching: Robust training of deep neural networks with extremely noisy labels. In: Proceedings of the 32nd International Conference on Neural Information Processing Systems. 2018, 8536−8546

[237] Liu Y, Kang Y, Xing C, Chen T, Yang Q . A secure federated transfer learning framework. IEEE Intelligent Systems, 2020, 35( 4): 70–82

[238] Yang H, He H, Zhang W, Cao X . FedSteg: a federated transfer learning framework for secure image steganalysis. IEEE Transactions on Network Science and Engineering, 2021, 8( 2): 1084–1094

[239] Li Y, Chen C Y, Wasserman W W . Deep feature selection: theory and application to identify enhancers and promoters. Journal of Computational Biology, 2016, 23( 5): 322–336

[240] Louizos C, Welling M, Kingma D P. Learning sparse neural networks through L₀ regularization. In: Proceedings of the 6th International Conference on Learning Representations. 2018

[241] Chang H, Shejwalkar V, Shokri R, Houmansadr A. Cronus: robust and heterogeneous collaborative learning with black-box knowledge transfer. 2019, arXiv preprint arXiv: 1912.11279

[242] Van Berlo B, Saeed A, Ozcelebi T. Towards federated unsupervised representation learning. In: Proceedings of the 3rd ACM International Workshop on Edge Systems, Analytics and Networking. 2020, 31−36

[243] Liu J, Zhao P, Zhuang F, Liu Y, Sheng V S, Xu J, Zhou X, Xiong H. Exploiting aesthetic preference in deep cross networks for cross-domain recommendation. In: Proceedings of the Web Conference 2020. 2020, 2768−2774

[244] Li P, Tuzhilin A. DDTCDR: deep dual transfer cross domain recommendation. In: Proceedings of the 13th International Conference on Web Search and Data Mining. 2020, 331−339

[245] Ammad-Ud-Din M, Ivannikova E, Khan S A, Oyomno W, Fu Q, Tan K E, Flanagan A. Federated collaborative filtering for privacy-preserving personalized recommendation system. 2019, arXiv preprint arXiv: 1901.09888

[246] Minto L, Haller M, Livshits B, Haddadi H. Stronger privacy for federated collaborative filtering with implicit feedback. In: Proceedings of the 15th ACM Conference on Recommender Systems. 2021, 342−350

[247] Chai D, Wang L, Chen K, Yang Q . Secure federated matrix factorization. IEEE Intelligent Systems, 2021, 36( 5): 11–20

[248] Du Y, Zhou D, Xie Y, Shi J, Gong M . Federated matrix factorization for privacy-preserving recommender systems. Applied Soft Computing, 2021, 111: 107700

[249] Li Z, Ding B, Zhang C, Li N, Zhou J . Federated matrix factorization with privacy guarantee. Proceedings of the VLDB Endowment, 2021, 15( 4): 900–913

[250] Wu C, Wu F, Cao Y, Huang Y, Xie X. FedGNN: federated graph neural network for privacy-preserving recommendation. 2021, arXiv preprint arXiv: 2102.04925

[251] Zhang C, Long G, Zhou T, Yan P, Zhang Z, Zhang C, Yang B. Dual personalization on federated recommendation. In: Proceedings of the 32nd International Joint Conference on Artificial Intelligence. 2023, 4558−4566

[252] Wu J, Liu Q, Huang Z, Ning Y, Wang H, Chen E, Yi J, Zhou B. Hierarchical personalized federated learning for user modeling. In: Proceedings of the Web Conference 2021. 2021, 957−968

[253] Wu M, Li L, Chang T, Rigall E, Wang X, Xu C Z. FedCDR: federated cross-domain recommendation for privacy-preserving rating prediction. In: Proceedings of the 31st ACM International Conference on Information & Knowledge Management. 2022, 2179−2188

[254] Luo S, Xiao Y, Song L. Personalized federated recommendation via joint representation learning, user clustering, and model adaptation. In: Proceedings of the 31st ACM International Conference on Information & Knowledge Management. 2022, 4289−4293

[255] Kaissis G A, Makowski M R, Rückert D, Braren R F . Secure, privacy-preserving and federated machine learning in medical imaging. Nature Machine Intelligence, 2020, 2( 6): 305–311

[256] Sui D, Chen Y, Zhao J, Jia Y, Xie Y, Sun W. FedED: federated learning via ensemble distillation for medical relation extraction. In: Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing (EMNLP). 2020, 2118−2128

[257] Silva S, Gutman B A, Romero E, Thompson P M, Altmann A, Lorenzi M. Federated learning in distributed medical databases: meta-analysis of large-scale subcortical brain data. In: Proceedings of the 16th IEEE international symposium on biomedical imaging (ISBI 2019). 2019, 270−274

[258] Jin H, Dai X, Xiao J, Li B, Li H, Zhang Y . Cross-cluster federated learning and blockchain for internet of medical things. IEEE Internet of Things Journal, 2021, 8( 21): 15776–15784

[259] Xia Y, Yang D, Li W, Myronenko A, Xu D, Obinata H, Mori H, An P, Harmon S, Turkbey E, Turkbey B, Wood B, Patella F, Stellato E, Carrafiello G, Ierardi A, Yuille A, Roth H. Auto-FedAvg: learnable federated averaging for multi-institutional medical image segmentation. 2021, arXiv preprint arXiv: 2104.10195

[260] Jiang M, Roth H R, Li W, Yang D, Zhao C, Nath V, Xu D, Dou Q, Xu Z. Fair federated medical image segmentation via client contribution estimation. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. 2023, 16302−16311

[261] Acar D A E, Zhao Y, Navarro R M, Mattina M, Whatmough P N, Saligrama V. Federated learning based on dynamic regularization. In: Proceedings of the 9th International Conference on Learning Representations. 2021

[262] Chen Z, Yang C, Zhu M, Peng Z, Yuan Y . Personalized retrogress-resilient federated learning toward imbalanced medical data. IEEE Transactions on Medical Imaging, 2022, 41( 12): 3663–3674

[263] Xu A, Li W, Guo P, Yang D, Roth H, Hatamizadeh A, Zhao C, Xu D, Huang H, Xu Z. Closing the generalization gap of cross-silo federated medical image segmentation. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. 2022, 20834−20843

[264] Jiang M, Yang H, Cheng C, Dou Q . IOP-FL: Inside-outside personalization for federated medical image segmentation. IEEE Transactions on Medical Imaging, 2023, 42( 7): 2106–2117

[265] Wang J, Jin Y, Stoyanov D, Wang L . FedDP: dual personalization in federated medical image segmentation. IEEE Transactions on Medical Imaging, 2024, 43( 1): 297–308

[266] Adnan M, Kalra S, Cresswell J C, Taylor G W, Tizhoosh H R . Federated learning and differential privacy for medical image analysis. Scientific Reports, 2022, 12( 1): 1953

[267] Wu Y, Zeng D, Wang Z, Shi Y, Hu J. Federated contrastive learning for volumetric medical image segmentation. In: Proceedings of the 24th International Conference on Medical Image Computing and Computer Assisted Intervention. 2021, 367−377

[268] Li Y, Wen G . Research and practice of financial credit risk management based on federated learning. Engineering Letters, 2023, 31( 1):

[269] Xu Z, Cheng J, Cheng L, Xu X, Bilal M . Mses credit risk assessment model based on federated learning and feature selection. Computers, Materials & Continua, 2023, 75( 3): 5573–5595

[270] Lee C M, Delgado Fernández J, Potenciano Menci S, Rieger A, Fridgen G. Federated learning for credit risk assessment. In: Proceedings of the 56th Hawaii International Conference on System Sciences. 2023, 386−395

[271] Yang W, Zhang Y, Ye K, Li L, Xu C Z. FFD: a federated learning based method for credit card fraud detection. In: Proceedings of the 8th International Congress on Big Data. 2019, 18−32

[272] Wang Z, Xiao J, Wang L, Yao J . A novel federated learning approach with knowledge transfer for credit scoring. Decision Support Systems, 2024, 177: 114084

[273] Pourroostaei Ardakani S, Du N, Lin C, Yang J C, Bi Z, Chen L . A federated learning-enabled predictive analysis to forecast stock market trends. Journal of Ambient Intelligence and Humanized Computing, 2023, 14( 4): 4529–4535

[274] Yan Y, Yang G, Gao Y, Zang C, Chen J, Wang Q. Multi-participant vertical federated learning based time series prediction. In: Proceedings of the 8th International Conference on Computing and Artificial Intelligence. 2022, 165−171

[275] Shaheen M, Farooq M S, Umer T . Reduction in data imbalance for client-side training in federated learning for the prediction of stock market prices. Journal of Sensor and Actuator Networks, 2024, 13( 1): 1

[276] Myalil D, Rajan M A, Apte M, Lodha S. Robust collaborative fraudulent transaction detection using federated learning. In: Proceedings of the 2021 20th IEEE International Conference on Machine Learning and Applications (ICMLA). 2021, 373−378

[277] Abadi A, Doyle B, Gini F, Guinamard K, Murakonda S K, Liddell J, Mellor P, Murdoch S J, Naseri M, Page H, Theodorakopoulos G, Weller S. Starlit: Privacy-preserving federated learning to enhance financial fraud detection. 2024, arXiv preprint arXiv: 2401.10765

[278] Liu Y, Yu J J Q, Kang J, Niyato D, Zhang S . Privacy-preserving traffic flow prediction: a federated learning approach. IEEE Internet of Things Journal, 2020, 7( 8): 7751–7763

[279] Zhang C, Dang S, Shihada B, Alouini M S. Dual attention-based federated learning for wireless traffic prediction. In: Proceedings of the IEEE Conference on Computer Communications. 2021, 1−10

[280] Zeng T, Guo J, Kim K J, Parsons K, Orlik P, Di Cairano S, Saad W. Multi-task federated learning for traffic prediction and its application to route planning. In: Proceedings of 2021 IEEE Intelligent Vehicles Symposium (IV). 2021, 451−457

[281] Zhang C, Cui L, Yu S, Yu J J Q . A communication-efficient federated learning scheme for IoT-based traffic forecasting. IEEE Internet of Things Journal, 2022, 9( 14): 11918–11931

[282] Qi T, Chen L, Li G, Li Y, Wang C . FedAGCN: a traffic flow prediction framework based on federated learning and asynchronous graph convolutional network. Applied Soft Computing, 2023, 138: 110175

[283] Phyu H P, Stanica R, Naboulsi D . Multi-slice privacy-aware traffic forecasting at ran level: a scalable federated-learning approach. IEEE Transactions on Network and Service Management, 2023, 20( 4): 5038–5052

[284] Xia M, Jin D, Chen J . Short-term traffic flow prediction based on graph convolutional networks and federated learning. IEEE Transactions on Intelligent Transportation Systems, 2023, 24( 1): 1191–1203

[285] Zhang L, Zhang C, Shihada B . Efficient wireless traffic prediction at the edge: a federated meta-learning approach. IEEE Communications Letters, 2022, 26( 7): 1573–1577

[286] Hu S, Ye Y, Hu Q, Liu X, Cao S, Yang H H, Shen Y, Angeloudis P, Parada L, Wu C . A federated learning-based framework for ride-sourcing traffic demand prediction. IEEE Transactions on Vehicular Technology, 2023, 72( 11): 14002–14015

[287] Huo J T, XU Y W, Huo Z S, Xiao L M, He Z X . Research on key technologies of edge cache in virtual data space across WAN. Frontiers of Computer Science, 2023, 17( 1): 171102

[288] Li H Z, Jin H, Zheng L, Huang Y, Liao X F . ReCSA: a dedicated sort accelerator using ReRAM-based content addressable memory. Frontiers of Computer Science, 2023, 17( 2): 172103

[289] Jia J, Liu Y, Zhang G Z, Gao Y L, Qian D P . Software approaches for resilience of high performance computing systems: a survey. Frontiers of Computer Science, 2023, 17( 4): 174105

[290] Guo J Y, Zhang L, ROMERO HUNG J, Li C, Zhao J R, Guo M Y . FPGA sharing in the cloud: a comprehensive analysis. Frontiers of Computer Science, 2023, 17( 5): 175106

Acknowledgements

The research work was supported by the National Key R&D Program of China (No. 2021ZD0113602), the National Natural Science Foundation of China (Grant Nos. 62176014 and 62202273), and the Fundamental Research Funds for the Central Universities.

Competing interests

The authors declare that they have no competing interests or financial conflicts to disclose.

Open Access

This article is licensed under a Creative Commons Attribution 4.0 International License, which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons licence, and indicate if changes were made.

The images or other third party material in this article are included in the article’s Creative Commons licence, unless indicated otherwise in a credit line to the material. If material is not included in the article’s Creative Commons licence and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder.

To view a copy of this licence, visit http://creativecommons.org/licenses/by/4.0/.

RIGHTS & PERMISSIONS

2024 The Author(s) 2024. This article is published with open access at link.springer.com and journal.hep.com.cn