Adaptive reinforced federated graph domain generalization with dynamic aggregation strategy

Abstract

In recent years, research has increasingly transformed data into graph representations, using graph neural networks to extract rich relationships and interaction information. This enhances the model’s ability to understand and process complex data structures. Due to the privacy and sensitivity of certain data, especially in government and enterprise fields, these high-quality data are often strictly controlled, limiting centralized model training. These issues lead to weaker generalization of traditional models for unseen data. To address these challenges, this paper proposes a Reinforced Federated Graph Domain Generalization (RFGDG) method, which improves generalization across domain data scenarios while protecting data privacy through multi-party collaboration. We design a mini-batch processing strategy based on graph sampling, combined with GraphSage, to build an efficient local graph-based node classification model. This sampling strategy reduces computational overhead while preserving graph structure, improving local model performance. To address data heterogeneity and feature inconsistency across clients, we propose a federated graph domain generalization strategy based on random Fourier feature transformation and weighted covariance matrix optimization, which unifies feature representations, reduces redundancy, and enhances adaptability to inconsistent data. We also propose a dynamic parameter aggregation strategy for federated graph neural networks using deep reinforcement learning. With the Deep Deterministic Policy Gradient (DDPG) algorithm, we dynamically adjust aggregation weights based on each client’s contribution, improving global model accuracy and convergence speed. This strategy considers graph structure heterogeneity and client contribution differences, ensuring generalization in multi-client environments. Extensive experiments on three public graph datasets and one dataset from the Weibo platform demonstrate that the proposed RFGDG method significantly improves global model accuracy and shows stronger robustness and adaptability in multi-client environments.

Graphical abstract

Keywords

graph domain generalization / reinforcement learning / federated learning / graphsage

References

[1] Zhu Y, Xu X, Pan B . A method for the dynamic collaboration of the public and experts in large-scale group emergency decision-making: using social media data to evaluate the decision-making quality. Computers & Industrial Engineering, 2023, 176: 108943

[2] Kaufhold M A, Bayer M, Reuter C . Rapid relevance classification of social media posts in disasters and emergencies: a system and evaluation featuring active, incremental and online learning. Information Processing & Management, 2020, 57( 1): 102132

[3] Veličković P . Everything is connected: graph neural networks. Current Opinion in Structural Biology, 2023, 79: 102538

[4] Wei X, Liu Y, Sun J, Jiang Y, Tang Q, Yuan K . Dual subgraph-based graph neural network for friendship prediction in location-based social networks. ACM Transactions on Knowledge Discovery from Data, 2023, 17( 3): 42

[5] Gao C, Zheng Y, Li N, Li Y, Qin Y, Piao J, Quan Y, Chang J, Jin D, He X, Li Y . A survey of graph neural networks for recommender systems: challenges, methods, and directions. ACM Transactions on Recommender Systems, 2023, 1( 1): 3

[6] Yang L, Zhang Q, Shi R, Zhou W, Niu B, Wang C, Cao X, He D, Wang Z, Guo Y. Graph neural networks without propagation. In: Proceedings of the ACM Web Conference 2023. 2023, 469−477

[7] Bongini P, Bianchini M, Scarselli F . Molecular generative graph neural networks for drug discovery. Neurocomputing, 2021, 450: 242–252

[8] Wieder O, Kohlbacher S, Kuenemann M, Garon A, Ducrot P, Seidel T, Langer T . A compact review of molecular property prediction with graph neural networks. Drug Discovery Today: Technologies, 2020, 37: 1–12

[9] Nguyen T K, Liu Z, Fang Y. Link prediction on latent heterogeneous graphs. In: Proceedings of the ACM Web Conference 2023. 2023, 263−273

[10] Cao Q, Shen H, Gao J, Wei B, Cheng X. Popularity prediction on social platforms with coupled graph neural networks. In: Proceedings of the 13th International Conference on Web Search and Data Mining. 2020, 70−78

[11] Huang C, Xu H, Xu Y, Dai P, Xia L, Lu M, Bo L, Xing H, Lai X, Ye Y. Knowledge-aware coupled graph neural network for social recommendation. In: Proceedings of the 35th AAAI Conference on Artificial Intelligence. 2021, 4115−4122

[12] Jin S, Zeng X, Xia F, Huang W, Liu X . Application of deep learning methods in biological networks. Briefings in Bioinformatics, 2021, 22( 2): 1902–1917

[13] Dai G, Wang X, Zou X, Liu C, Cen S . MRGAT: multi-relational graph attention network for knowledge graph completion. Neural Networks, 2022, 154: 234–245

[14] Long Q, Fang Z, Fang C, Chen C, Wang P, Zhou Y. Unveiling delay effects in traffic forecasting: a perspective from spatial-temporal delay differential equations. In: Proceedings of the ACM Web Conference 2024. 2024, 1035−1044

[15] Guan Z, Li Y, Xue Z, Liu Y, Gao H, Shao Y. Federated graph neural network for cross-graph node classification. In: Proceedings of the 7th IEEE International Conference on Cloud Computing and Intelligent Systems (CCIS). 2021, 418−422

[16] Wang H, Kaplan Z, Niu D, Li B. Optimizing federated learning on non-IID data with reinforcement learning. In: Proceedings of IEEE INFOCOM 2020 - IEEE Conference on Computer Communications. 2020, 1698−1707

[17] Banabilah S, Aloqaily M, Alsayed E, Malik N, Jararweh Y . Federated learning review: fundamentals, enabling technologies, and future applications. Information Processing & Management, 2022, 59( 6): 103061

[18] Zhu H, Xu J, Liu S, Jin Y . Federated learning on non-IID data: a survey. Neurocomputing, 2021, 465: 371–390

[19] Briggs C, Fan Z, Andras P. Federated learning with hierarchical clustering of local updates to improve training on non-IID data. In: Proceedings of 2020 International Joint Conference on Neural Networks (IJCNN). 2020, 1−9

[20] Xie H, Ma J, Xiong L, Yang C. Federated graph classification over non-IID graphs. In: Proceedings of the 35th International Conference on Neural Information Processing Systems. 2021, 1440

[21] Zhu X, Li G, Hu W. Heterogeneous federated knowledge graph embedding learning and unlearning. In: Proceedings of the ACM Web Conference 2023. 2023, 2444−2454

[22] Liu Q, Chen C, Qin J, Dou Q, Heng P A. FedDG: federated domain generalization on medical image segmentation via episodic learning in continuous frequency space. In: Proceedings of 2021 IEEE/CVF Conference on Computer Vision and Pattern Recognition. 2021, 1013−1023

[23] Chen J, Jiang M, Dou Q, Chen Q. Federated domain generalization for image recognition via cross-client style transfer. In: Proceedings of 2023 IEEE/CVF Winter Conference on Applications of Computer Vision. 2023, 361−370

[24] Zhou K, Liu Z, Qiao Y, Xiang T, Loy C C . Domain generalization: a survey. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2023, 45( 4): 4396–4415

[25] Liang J, Hu D, Feng J. Domain adaptation with auxiliary target domain-oriented classifier. In: Proceedings of 2021 IEEE/CVF Conference on Computer Vision and Pattern Recognition. 2021, 16627−16637

[26] Wang J, Lan C, Liu C, Ouyang Y, Qin T, Lu W, Chen Y, Zeng W, Yu P S . Generalizing to unseen domains: a survey on domain generalization. IEEE Transactions on Knowledge and Data Engineering, 2023, 35( 8): 8052–8072

[27] Zhang W, Zhao X, Zhao L, Yin D, Yang G H, Beutel A. Deep reinforcement learning for information retrieval: fundamentals and advances. In: Proceedings of the 43rd International ACM SIGIR Conference on Research and Development in Information Retrieval. 2020, 2468−2471

[28] Chen J, Li S E, Tomizuka M . Interpretable end-to-end urban autonomous driving with latent deep reinforcement learning. IEEE Transactions on Intelligent Transportation Systems, 2022, 23( 6): 5068–5078

[29] Zhao W, Queralta J P, Westerlund T. Sim-to-real transfer in deep reinforcement learning for robotics: a survey. In: Proceedings of 2020 IEEE Symposium Series on Computational Intelligence (SSCI). 2020, 737−744

[30] Rahmani S, Baghbani A, Bouguila N, Patterson Z . Graph neural networks for intelligent transportation systems: a survey. IEEE Transactions on Intelligent Transportation Systems, 2023, 24( 8): 8846–8885

[31] Kipf T N, Welling M. Semi-supervised classification with graph convolutional networks. In: Proceedings of the 5th International Conference on Learning Representations. 2017

[32] Veličković P, Cucurull G, Casanova A, Romero A, Liò P, Bengio Y. Graph attention networks. In: Proceedings of the 6th International Conference on Learning Representations. 2018

[33] Hamilton W L, Ying Z, Leskovec J. Inductive representation learning on large graphs. In: Proceedings of the 31st International Conference on Neural Information Processing Systems. 2017, 1025−1035

[34] McMahan B, Moore E, Ramage D, Hampson S, Arcas B A Y. Communication-efficient learning of deep networks from decentralized data. In: Proceedings of the 20th International Conference on Artificial Intelligence and Statistics. 2017, 1273−1282

[35] Li T, Sahu A K, Zaheer M, Sanjabi M, Talwalkar A, Smith V. Federated optimization in heterogeneous networks. In: Proceedings of the 3rd Conference on Machine Learning and Systems. 2020

[36] Li Q, He B, Song D. Model-contrastive federated learning. In: Proceedings of 2021 IEEE/CVF Conference on Computer Vision and Pattern Recognition. 2021, 10708−10717

[37] Li G, Hu Y, Zhang M, Liu J, Yin Q, Peng Y, Dou D. FedHiSyn: a hierarchical synchronous federated learning framework for resource and data heterogeneity. In: Proceedings of the 51st International Conference on Parallel Processing. 2022, 8

[38] Wang B, Li A, Pang M, Li H, Chen Y. GraphFL: a federated learning framework for semi-supervised node classification on graphs. In: Proceedings of 2022 IEEE International Conference on Data Mining (ICDM). 2022, 498−507

[39] Dou Q, Castro D C, Kamnitsas K, Glocker B. Domain generalization via model-agnostic learning of semantic features. In: Proceedings of the 33rd International Conference on Neural Information Processing Systems. 2019, 579

[40] Muandet K, Balduzzi D, Schölkopf B. Domain generalization via invariant feature representation. In: Proceedings of the 30th International Conference on Machine Learning. 2013, 10−18

[41] Huang Z, Wang H, Xing E P, Huang D. Self-challenging improves cross-domain generalization. In: Proceedings of the 16th European Conference on Computer Vision. 2020, 124−140

[42] Cirac G, Farfan J, Avansi G D, Schiozer D J, Rocha A . Cross-domain feature learning and data augmentation for few-shot proxy development in oil industry. Applied Soft Computing, 2023, 149: 110972

[43] Watkins C J C H, Dayan P . Q-learning. Machine Learning, 1992, 8( 3): 279–292

[44] Mnih V, Kavukcuoglu K, Silver D, Graves A, Antonoglou I, Wierstra D, Riedmiller M. Playing Atari with deep reinforcement learning. 2013, arXiv preprint arXiv: 1312.5602

[45] Lillicrap T P, Hunt J J, Pritzel A, Heess N, Erez T, Tassa Y, Silver D, Wierstra D. Continuous control with deep reinforcement learning. In: Proceedings of the 4th International Conference on Learning Representations. 2016

[46] Sen P, Namata G, Bilgic M, Getoor L, Gallagher B, Eliassi-Rad T . Collective classification in network data. AI Magazine, 2008, 29( 3): 93–106

RIGHTS & PERMISSIONS

Higher Education Press