Personalized federated learning for semantic communication with collaborative fine-tuning✩

Maochuan Wu; Juan Li; Jing Xu; Bing Chen; Kun Zhu

doi:10.1016/j.dcan.2025.08.005

›› 2026, Vol. 12 ›› Issue (2) :306 -318. DOI: 10.1016/j.dcan.2025.08.005

Regular Papers

research-article

Personalized federated learning for semantic communication with collaborative fine-tuning^✩

Maochuan Wu ^a
, Juan Li ^a^,^b^,^*
, Jing Xu ^a^,^b
, Bing Chen ^a
, Kun Zhu ^a

Author information +

^a College of Computer Science and Technology, Nanjing University of Aeronautics and Astronautics, Nanjing, China

^b Ministry Key Laboratory for Safety-Critical Software Development and Verification (Nanjing University of Aeronautics and Astronautics), Nanjing, China

^* College of Computer Science and Technology, Nanjing University of Aeronautics and Astronautics, Nanjing, China. E-mail addresses: 15366662916@163.com (M. Wu), juanli@nuaa.edu.cn (J. Li), 162010303@nuaa.edu.cn (J. Xu), cb_china@nuaa.edu.cn (B. Chen), zhukun@nuaa.edu.cn (K. Zhu).

Maochuan Wu received the M.Eng. degree in Electronic and Communication Engi-neering from Nanjing University of Aeronautics and Astronautics, Nanjing, China, in 2018. He is currently pursuing the D.Eng. degree at the same university. His research interests include networked unmanned systems.

Juan Li received her B.Eng. Degree from the Department of Computer Science and Technology at Anhui University, Hefei, China, in 2014 and Ph.D. Degree from the Depart-ment of Computer Science and Engineering at Shanghai Jiao Tong University, Shanghai, China, in 2020. She is now an associate professor at Nanjing University of Aeronautics and Astronautics. Her research interests include federated learning, mechanism design, and edge computing.

Jing Xu is working toward the graduate degree with the College of Computer Science and Technology, Nanjing University of Aeronautics and Astronautics, Nanjing, China. Her research interests include federated learning and mechanism design.

Bing Chen received the Ph.D. degree from the Nanjing University of Aeronautics and Astronautics, Nanjing, China, in 2008, where he is a Professor with the College of Com-puter Science and Technology, and the Collaborative Innovation Center of Novel Software Technology and Industrialization. His research interests include computer networks and security, grid and cloud computing, and Internet of Things.

Kun Zhu received the Ph.D. degree from the School of Computer Engineering, Nanyang Technological University, Singapore, in 2012. He is currently a Professor with the College of Computer Science and Technology, Nanjing University of Aeronautics and Astronautics, and the Collaborative Innovation Center of Novel Software Technology and Industrialization, Nanjing, China. His research interests include resource allocation in 5G, wireless virtualization, and selforganizing networks.

Show less

History +

PDF

Abstract

Semantic Communication (SemCom) is a promising paradigm for future 6G networks, where communication performance hinges on the eﬀectiveness of SemCom models, particularly the source-channel encoder and decoder. However, training these models faces significant challenges. Firstly, the privacy-sensitive nature of communication data discourages users from uploading data to centralized servers. Secondly, heterogeneous local data distributions and diverse communication counterparts of diﬀerent users necessitate personalized SemCom models. Specifically, a user’s encoder must align with its receivers’ decoders and the transmitted data distribution, while its decoder must adapt to the user’s transmitters and received data distribution. To address these challenges, we propose FineFed, a personalized federated learning method with collaborative fine-tuning. Initially, a unified global model is trained distributively via federated learning, eliminating data uploads. Subsequently, users iteratively fine-tune encoders and decoders collaboratively, achieving SemCom model personalization. For encoder fine-tuning, decoders are fixed and shared with transmitters to address distributed loss calculation issues. Each encoder is fine-tuned using the idea of multi-task learning, treating communication with each receiver as a separate task. Then, encoders are fixed. A user shares its decoder with its own transmitters. These transmitters collaboratively fine-tune the user’s decoder by the idea of federated multi-task learning. Experimental results demonstrate that FineFed improves the average performance of federated SemCom models by 1%-7%, bringing it closer to the performance of centrally-trained models.

Keywords

Semantic communication / Federated learning / Fine-tuning / Personalized federated learning

Cite this article

Download citation ▾

Maochuan Wu, Juan Li, Jing Xu, Bing Chen, Kun Zhu. Personalized federated learning for semantic communication with collaborative fine-tuning^✩. , 2026, 12(2): 306-318 DOI:10.1016/j.dcan.2025.08.005

登录浏览全文

4963

注册一个新账户忘记密码

CRediT authorship contribution statement

Maochuan Wu: Writing-original draft, Validation, Methodology. Juan Li: Writing-review & editing, Supervision, Funding acquisition, Conceptualization. Jing Xu: Visualization, Validation, Data curation. Bing Chen: Writing-review & editing, Funding acquisition. Kun Zhu: Conceptualization.

Declaration of competing interest

The authors declare the following financial interests/personal rela-tionships which may be considered as potential competing interests: Juan Li reports financial support was provided by National Natural Science Foundation of China. Juan Li reports financial support was provided by Natural Science Foundation of Jiangsu Province. Juan Li reports financial support was provided by China Postdoctoral Science Foundation. Juan Li reports financial support was provided by Open Foundation of Mimistry Key Laboratory for Safety-Critical Software De-velopment and Verification (Nanjing University of Aeronautics and As-tronautics). Juan Li reports financial support was provided by Dual Innovation Doctor Foundation of Jiangsu Province. If there are other authors, they declare that they have no known competing financial in-terests or personal relationships that could have appeared to influence the work reported in this paper.

Acknowledgements

This work was supported by National Natural Science Foundation of China under Grant No. 62202224, Natural Science Foundation of Jiangsu Province under Grant No. BK20220882, China Postdoctoral Sci-ence Foundation under Grant No. 2022TQ0154, Open Foundation of Ministry Key Laboratory for Safety-Critical Software Development and Verification (Nanjing University of Aeronautics and Astronautics) un-der Grant No. NJ2024030, and Dual Innovation Doctor Foundation of Jiangsu Province under Grant No. JSSCBS20220213.

References

Publishing order | Descend order by publishing year | Descend order by cited within

[1]	L. Zhi, N. Hehao, H. Yuanzhi, A. Kang, Z. Xudong, C. Zheng, X. Pei, Self-powered absorptive reconfigurable intelligent surfaces for securing satellite-terrestrial inte-grated networks, China Commun. 21 (9) (2024) 276-291.

[2]	W. Yang, H. Du, Z.Q. Liew, W.Y.B. Lim, Z. Xiong, D. Niyato, X. Chi, X. Shen, C. Miao, Semantic communications for future Internet: fundamentals, applications, and challenges, IEEE Commun. Surv. Tutor. 25 (1) (2023) 213-250.

[3]	P. Zhang, W. Xu, H. Gao, K. Niu, X. Xu, X. Qin, C. Yuan, Z. Qin, H. Zhao, J. Wei, F. Zhang,Toward wisdom-evolutionary and primitive-concise 6G: a new paradigm of semantic communication networks, Engineering 8 (2022) 60-73.

[4]	Q. Lan, D. Wen, Z. Zhang, Q. Zeng, X. Chen, P. Popovski, K. Huang, What is semantic communication? A view on conveying meaning in the era of machine intelligence, J. Commun. Inf. Netw. 6 (4) (2021) 336-371.

[5]	A. Jee, S. Prakriya, Performance of energy and spectrally eﬃcient AF relay-aided incremental CDRT NOMA-based IoT network with imperfect SIC for smart cities, IEEE Internet Things J. 10 (21) (2023) 18766-18781.

[6]	A. Jee, S. Prakriya, Novel channel aware power control for a multi-user downlink NOMA network, IEEE Wirel. Commun. Lett. 13 (2) (2024) 392-396.

[7]	Y. Sun, Z. Lin, K. An, D. Li, C. Li, Y. Zhu, D. Wing Kwan Ng, N. Al-Dhahir, J. Wang, Multi-functional RIS-assisted semantic anti-jamming communication and computing in integrated aerial-ground networks, IEEE J. Sel. Areas Commun. 42 (12) (2024) 3597-3617.

[8]	J. Dai, S. Wang, K. Tan, Z. Si, X. Qin, K. Niu, P. Zhang, Nonlinear transform source-channel coding for semantic communications, IEEE J. Sel. Areas Commun. 40 (8) (2022) 2300-2316.

[9]	H. Xie, Z. Qin, G.Y. Li, B.-H. Juang, Deep learning enabled semantic communication systems, IEEE Trans. Signal Process. 69 (2021) 2663-2675.

[10]	H. Ye, L. Liang, G.Y. Li, B.-H. Juang, Deep learning-based end-to-end wireless com-munication systems with conditional GANs as unknown channels, IEEE Trans. Wirel. Commun. 19 (5) (2020) 3133-3143.

[11]	X. Xu, Y. Xu, H. Dou, M. Chen, L. Wang, Federated KD-assisted image seman-tic communication in IoT edge learning, IEEE Internet Things J. 11 (21) (2024) 34215-34228.

[12]	L.X. Nguyen, H.Q. Le, Y.L. Tun, P.S. Aung, Y.K. Tun, Z. Han, C.S. Hong, An eﬃcient federated learning framework for training semantic communication systems, IEEE Trans. Veh. Technol. 73 (10) (2024) 15872-15877.

[13]	X. Lu, K. Zhu, J. Li, Y. Zhang, Eﬃcient knowledge base synchronization in semantic communication network: a federated distillation approach, in: 2024 IEEE Wireless Communications and Networking Conference (WCNC), IEEE, 2024, pp. 1-6.

[14]	G. Zheng, Q. Ni, K. Navaie, H. Pervaiz, G. Min, A. Kaushik, C. Zarakovitis, Mobility-aware split-federated with transfer learning for vehicular semantic communication networks, IEEE Internet Things J. 11 (10) (2024) 17237-17248.

[15]	A. Ghosh, J. Chung, D. Yin, K. Ramchandran, An eﬃcient framework for clustered federated learning, Adv. Neural Inf. Process. Syst. 33 (2020) 19586-19597.

[16]	S. Wu, Y. Jia, B. Liu, H. Xiang, X. Xu, W. Dou, PFedCS: a personalized federated learn-ing method for enhancing collaboration among similar classifiers,in: Proceedings of the AAAI Conference on Artificial Intelligence, vol. 20, 2025, pp. 21572-21580.

[17]	B. Liu, Y. Ma, Z. Zhou, Y. Shi, S. Li, Y. Tong, CASA: clustered federated learning with asynchronous clients,in: Proceedings of the 30th ACM SIGKDD Conference on Knowledge Discovery and Data Mining, 2024, pp. 1851-1862.

[18]	A. Fallah, A. Mokhtari, A. Ozdaglar,Personalized federated learning with theoretical guarantees: a model-agnostic meta-learning approach, Adv. Neural Inf. Process. Syst. 33 (2020) 3557-3568.

[19]	R. Lee, M. Kim, D. Li, X. Qiu, T. Hospedales, F. Huszár, N. Lane,Fedl2p: federated learning to personalize, Adv. Neural Inf. Process. Syst. 36 (2023) 14818-14836.

[20]	J. Scott, H. Zakerinia,C.H. Lampert, PeFLL: personalized federated learning by learn-ing to learn, arXiv preprint, arXiv:2306.05515.

[21]	X. Ma, J. Zhang, S. Guo, W. Xu,Layer-wised model aggregation for personalized federated learning, in:Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2022, pp. 10092-10101.

[22]	M. Ren, X. Yu, Multibranch multilevel federated learning for a better feature extrac-tion and a plug-and-play dynamic-adjusting double flow personalization approach, Appl. Intell. 53 (2022) 1-16.

[23]	H. Chen, H. Vikalo, et al., The best of both worlds: accurate global and personal-ized models through federated learning with data-free hyper-knowledge distillation, arXiv preprint, arXiv:2301.08968.

[24]	Y. Li, X. Wang, H. Li, P.K. Donta, M. Huang, S. Dustdar, Communication-eﬃcient federated learning for heterogeneous clients, ACM Trans. Internet Technol. 25 (2) (2025) 1-37.

[25]	B. Zhao, H. Xing, L. Xu, Y. Li, L. Feng, J. Peng, Z. Xiao, On forecasting-oriented time series transmission: a federated semantic communication system, IEEE Trans. Mob. Comput. 23 (12) (2024) 13728-13744.

[26]	Y. Wang, W. Ni, W. Yi, X. Xu, P. Zhang, A. Nallanathan, Federated contrastive learn-ing for personalized semantic communication, IEEE Commun. Lett. 28 (8) (2024) 1875-1879.

[27]	J. Peng, H. Xing, L. Xu, S. Luo, P. Dai, L. Feng, J. Song, B. Zhao, Z. Xiao, Adversar-ial reinforcement learning based data poisoning attacks defense for task-oriented multi-user semantic communication, IEEE Trans. Mob. Comput. 23 (12) (2024) 14834-14851.

[28]	B. McMahan, E. Moore, D. Ramage, S. Hampson, B.A. y Arcas, Communication-eﬃcient learning of deep networks from decentralized data, in: Artificial Intelligence and Statistics, PMLR, 2017, pp. 1273-1282.

[29]	P. Tseng, Convergence of a block coordinate descent method for nondiﬀerentiable minimization, J. Optim. Theory Appl. 109 (2001) 475-494.

[30]	Z. Liu, Y. Lin, Y. Cao, H. Hu, Y. Wei, Z. Zhang, S. Lin, B. Guo, Swin transformer: hier-archical vision transformer using shifted windows,in: 2021 IEEE/CVF International Conference on Computer Vision (ICCV), 2021, pp. 9992-10002.

[31]	K. Yang, S. Wang, J. Dai, K. Tan, K. Niu, P. Zhang,WITT: a wireless image transmis-sion transformer for semantic communications, in: ICASSP 2023-2023 IEEE Interna-tional Conference on Acoustics, Speech and Signal Processing (ICASSP), IEEE, 2023, pp. 1-5.

[32]	Y. Lecun, L. Bottou, Y. Bengio, P. Haﬀner, Gradient-based learning applied to docu-ment recognition, Proc. IEEE 86 (11) (1998) 2278-2324.

[33]	A. Krizhevsky, I. Sutskever, G.E. Hinton,Imagenet classification with deep convolu-tional neural networks, Adv. Neural Inf. Process. Syst. (2012) 1097-1105.

[34]	J. Deng, W. Dong, R. Socher, L.-J. Li, K. Li, L. Fei-Fei,ImageNet: a large-scale hier-archical image database, in: 2009 IEEE Conference on Computer Vision and Pattern Recognition, 2009, pp. 248-255.

[35]	fastai, Imagenette dataset, https://github.com/fastai/imagenette, 2019. (Accessed 9 June 2025).

[36]	Z. Wang, E.P. Simoncelli, A.C. Bovik, Multiscale structural similarity for image qual-ity assessment, in: The Thrity-Seventh Asilomar Conference on Signals, Systems & Computers, IEEE, 2003, pp. 1398-1402.

[37]	B. Xie, Y. Wu, Y. Shi, D.W.K. Ng, W. Zhang, Communication-eﬃcient framework for distributed image semantic wireless transmission, IEEE Internet Things J. 10 (24) (2023) 22555-22568.

[38]	Y. Yan, X. Zhang, L. Li, W. Lin, R. Li, W. Cheng, Z. Han,Fssc: federated learning of transformer neural networks for semantic image communication, in: GLOBECOM 2024-2024 IEEE Global Communications Conference, IEEE, 2024, pp. 1659-1664.

[39]	H. Tong, Z. Yang, S. Wang, Y. Hu, O. Semiari, W. Saad, C. Yin, Federated learning for audio semantic communication, Front. Commun. Netw. 2 (2021) 734402.

[40]	H. Sun, H. Tian, W. Ni, J. Zheng,Federated learning-based cooperative model train-ing for task-oriented semantic communication, in: IEEE INFOCOM 2024 -IEEE Conference on Computer Communications Workshops (INFOCOM WKSHPS), 2024, pp. 1-6.

[41]	H. Xing, H. Zhang, X. Wang, L. Xu, Z. Xiao, B. Zhao, S. Luo, L. Feng, Y. Dai, A multi-user deep semantic communication system based on federated learning with dynamic model aggregation, in: 2023 IEEE International Conference on Communi-cations Workshops (ICC Workshops), 2023, pp. 1612-1616.

[42]

L. Li, G. Li, S. Li, R. Xu, J. Li, ZeroTKS: zero-trust knowledge synchronization via federated fine-tuning for secure semantic communications,in:Proceedings of the Twenty-Fifth International Symposium on Theory, Algorithmic Foundations, and Protocol Design for Mobile Networks and Mobile Computing, 2024, pp. 422-427.

[43]	J. Xu, H. Yao, R. Zhang, T. Mai, S. Huang, S. Guo, Federated learning powered se-mantic communication for UAV swarm cooperation, IEEE Wirel. Commun. 31 (4) (2024) 140-146.

[44]	Y. Peng, F. Jiang, L. Dong, K. Wang,K. Yang, Personalized federated learning for generative AI-assisted semantic communications, arXiv preprint, arXiv:2410.02450.

[45]	H. Wei, W. Ni, W. Xu, F. Wang, D. Niyato, P. Zhang, Federated semantic learning driven by information bottleneck for task-oriented communications, IEEE Commun. Lett. 27 (10) (2023) 2652-2656.

[46]	H. Xing, H. Ma, Z. Xiao, B. Zhao, J. Peng, S. Luo, L. Feng, L. Xu, Feddistillsc: a distributed semantic communication system based on federated distillation, in: 2023 IEEE 23rd International Conference on Communication Technology (ICCT), IEEE, 2023, pp. 506-510.

[47]	B. Zhao, H. Xing, X. Wang, Z. Xiao, L. Xu, Classification-oriented distributed semantic communication for multivariate time series, IEEE Signal Process. Lett. 30 (2023) 369-373.

[48]	Y. Li, X. Wang, R. Zeng, P.K. Donta, I. Murturi, M. Huang,S. Dustdar, Federated domain generalization: a survey, arXiv preprint, arXiv:2306.01334.

[49]	Y. Li, X. Wang, R. Zeng, M. Yang, K. Li, M. Huang, S. Dustdar, VARF: an incentive mechanism of cross-silo federated learning in MEC, IEEE Internet Things J. 10 (17) (2023) 15115-15132.

[50]	J. Li, Y. Zhu, J. Wu, W. Wu, T. Zang, L. Lu,All federated or not: optimizing personal model performance in cross-silo federated learning, in: 2024 IEEE 30th International Conference on Parallel and Distributed Systems (ICPADS), 2024, pp. 390-399.