A Quantised Push-Sum Distributed Adaptive Momentum Algorithm for Optimisation Over Directed Networks

Qingguo Lü , Mingyu Guan , Chenglong He , Chen Chen , Xiangguang Dai , Man-Fai Leung , Wei Zhang

CAAI Transactions on Intelligence Technology ›› 2026, Vol. 11 ›› Issue (4) : 1284 -1301.

PDF (2864KB)
CAAI Transactions on Intelligence Technology ›› 2026, Vol. 11 ›› Issue (4) :1284 -1301. DOI: 10.1049/cit2.70122
ORIGINAL RESEARCH
research-article
A Quantised Push-Sum Distributed Adaptive Momentum Algorithm for Optimisation Over Directed Networks
Author information +
History +
PDF (2864KB)

Abstract

In this paper, we investigate a distributed constrained optimisation problem over directed networks. The agents in the networks conduct local computations and communications, endeavouring to collaboratively minimise the aggregation of all locally known convex cost functions subject to a global constraint set. However, since the agents are constantly transmitting information, most existing algorithms for this problem are prone to communication burdens, especially in large-scale networks under a limited communication bandwidth. Problems of this nature emerge in a number of applications, mostly evident in distributed classification tasks, distributed image restoration, distributed compressive sensing etc. To solve these kinds of problems, we propose an effective quantised push-sum distributed adaptive momentum (QPS-DADAM) algorithm. On the one hand, the QPS-DADAM algorithm employs the random quantiser to reduce the communication overhead and avoid the channel blockage. On the other hand, the QPS-DADAM algorithm incorporates the adaptive momentum method into the push-sum protocol to further accelerate the convergence over directed networks. Rigorous theoretical analyses are provided to illustrate that the QPS-DADAM algorithm converges sublinearly to the optimal solution. In addition, numerical simulations further demonstrate the efficacy of the QPS-DADAM algorithm and the correctness of the theoretical discoveries.

Keywords

adaptive momentum method / directed networks / distributed constrained optimisation / quantised communication / sublinear convergence

Cite this article

Download citation ▾
Qingguo Lü, Mingyu Guan, Chenglong He, Chen Chen, Xiangguang Dai, Man-Fai Leung, Wei Zhang. A Quantised Push-Sum Distributed Adaptive Momentum Algorithm for Optimisation Over Directed Networks. CAAI Transactions on Intelligence Technology, 2026, 11 (4) : 1284-1301 DOI:10.1049/cit2.70122

登录浏览全文

4963

注册一个新账户 忘记密码

References

[1]

R. Yan, Y. Lin, N. Yu, and Y. Wu, “ A Low-Carbon Economic Dispatch Model for Electricity Market With Wind Power Based on Improved Ant-Lion Optimisation Algorithm,” CAAI Transactions on Intelligent Technology 8, no. 1 (2023): 29-39, https://doi.org/10.1049/cit2.12138.

[2]

Z. Chen, M. Yu, Z. Zhang, and H. Zeng, “ A Power Allocation Algorithm Based on Q-Learning for Two-Way Relaying System With Dual-Media Parallel Communication,” IET Signal Processing 17, no. 2 (2023): e12174, https://doi.org/10.1049/sil2.12174.

[3]

M. Hong, “ A Distributed, Asynchronous, and Incremental Algorithm for Nonconvex Optimization: An Admm Approach,” IEEE Transactions on Control of Network Systems 5, no. 3 (2018): 935-945, https://doi.org/10.1109/tcns.2017.2657460.

[4]

T. Yang, X. Yi, J. Wu, et al., “ A Survey of Distributed Optimization,” Annual Reviews in Control 47 (2019): 278-305, https://doi.org/10.1016/j.arcontrol.2019.05.006.

[5]

A. Gasparin, S. Lukovic, and C. Alippi, “ Deep Learning for Time Series Forecasting: The Electric Load Case,” CAAI Transactions on Intelligence Technology 7, no. 1 (2022): 1-25, https://doi.org/10.1049/cit2.12060.

[6]

N. Patari, V. Venkataramanan, A. Srivastava, D. K. Molzahn, N. Li, and A. Annaswamy, “ Distributed Optimization in Distribution Systems: Use Cases, Limitations, and Research Needs,” IEEE Transactions on Power Systems 37, no. 5 (2022): 3469-3481, https://doi.org/10.1109/TPWRS.2021.3132348.

[7]

A. Nedić and A. Ozdaglar, “ Distributed Subgradient Methods for Multi-Agent Optimization,” IEEE Transactions on Automatic Control 54, no. 1 (2009): 48-61, https://doi.org/10.1109/TAC.2008.2009515.

[8]

M. Xiong, B. Zhang, D. Yuan, and S. Xu, “ Distributed Quantized Mirror Descent for Strongly Convex Optimization Over Time-Varying Directed Graph,” Science China Information Sciences 65, no. 10 (2022): 202202, https://doi.org/10.1007/s11432-020-3275-3.

[9]

D. Yuan, B. Zhang, D. W. Ho, W. X. Zheng, and S. Xu, “ Distributed Online Bandit Optimization Under Random Quantization,” Automatica 146, no. 1 (2022): 110590, https://doi.org/10.1016/j.automatica.2022.110590.

[10]

X. Cao, T. Başar, S. Diggavi, et al., “ Communication-Efficient Distributed Learning: An Overview,” IEEE Journal on Selected Areas in Communications 41, no. 4 (2023): 851-873, https://doi.org/10.1109/jsac.2023.3242710.

[11]

Y. Pu, M. N. Zeilinger, and C. N. Jones, “ Quantization Design for Unconstrained Distributed Optimization,” in Proceedings of the American Control Conference (2015), 1229-1234, https://doi.org/10.1109/ACC.2015.7170901.

[12]

A. I. Rikos, W. Jiang, T. Charalambous, et al., “ Distributed Optimization With Gradient Descent and Quantized Communication,” in Proceedings of the 22nd World Congress of the International Federation of Automatic Control (IFAC) (2023), 6433-6439, https://doi.org/10.1016/j.ifacol.2023.10.100.

[13]

W. Zhang, Y. Shi, B. Zhang, K. Lu, and D. Yuan, “ Quantized Distributed Online Projection-Free Convex Optimization,” IEEE Control Systems Letters 7, no. 1 (2023): 1837-1842, https://doi.org/10.1109/LCSYS.2023.3282021.

[14]

A. Nedić, A. Olshevsky, A. Ozdaglar, and J. Tsitsiklis, “ On Distributed Averaging Algorithms and Quantization Effects,” IEEE Transactions on Automatic Control 54, no. 11 (2009): 2506-2517, https://doi.org/10.1109/TAC.2009.2031203.

[15]

H. Taheri, A. Mokhtari, H. Hassani, and R. Pedarsani, “ Quantized Push-Sum for Gossip and Decentralized Optimization Over Directed Graphs,” arXiv Preprint arXiv:2002.09964 (2020), https://doi.org/10.48550/arXiv.2002.09964.

[16]

D. Alistarh, D. Grubic, J. Li, et al., “ Qsgd: Communication-Efficient Sgd Via Gradient Quantization and Encoding,” Advances in Neural Information Processing Systems 30 (2017): 1-12, https://doi.org/10.48550/arXiv.1610.02132.

[17]

A. Reisizadeh, H. Taheri, A. Mokhtari, et al., “ Robust and Communication-Efficient Collaborative Learning,” in Proc. of Neural Information Processing Systems (2019), 8386-8397, https://doi.org/10.48550/arXiv.2510.08311.

[18]

A. Reisizadeh, A. Mokhtari, H. Hassani, and R. Pedarsani, “ An Exact Quantized Decentralized Gradient Descent Algorithm,” IEEE Transactions on Signal Processing 67, no. 19 (2019): 4934-4947, https://doi.org/10.1109/TSP.2019.2932876.

[19]

J. Duchi, E. Hazan, and Y. Singer, “ Adaptive Subgradient Methods for Online Learning and Stochastic Optimization,” Journal of Machine Learning Research 12, no. 7 (2011): 1-39, https://dl.acm.org/doi/pdf/10.5555/1953048.2021068.

[20]

D. Kingma, J. B. Adam, “ A Method for Stochastic Optimization,” in Proceedings of the Conference on Learning Representations (2015), 6, https://doi.org/10.48550/arXiv.1412.6980.

[21]

S. J. Reddi, S. Kale, and S. Kumar, “ On the Convergence of Adam and Beyond,” Computer Science, Accessed: Apr. 19, 2019, https://arxiv.org/abs/1904.09237.

[22]

K. Okamoto, N. Hayashi, and S. Takai, “ Distributed Online Adaptive Gradient Descent With Event Triggered Communication,” IEEE Transactions on Control of Network Systems 11, no. 2 (2023): 1-12, https://doi.org/10.1109/TCNS.2023.3294432.

[23]

X. Shen, D. Li, R. Fang, Y. Zhou, and X. Wu, “ Distributed Adaptive Online Learning for Convex Optimization With Weight Decay,” Asian Journal of Control 24, no. 2 (2022): 562-575, https://doi.org/10.1002/asjc.2489.

[24]

P. Nazari, D. A. Tarzanagh, and G. Michailidis, “ Dadam: A Consensus-Based Distributed Adaptive Gradient Method for Online Optimization,” IEEE Transactions on Signal Processing 70, no. 1 (2022): 6065-6079, https://doi.org/10.1109/TSP.2022.3223214.

[25]

Y. Zhang, Q. Ai, and Z. Li, “ Admm-Based Distributed Response Quantity Estimation: A Probabilistic Perspective,” IET Generation, Transmission & Distribution 14, no. 26 (2020): 6594-6602, https://doi.org/10.1049/iet-gtd.2020.1380.

[26]

K. Muhammad, Mustaqeem, A. Ullah, et al., “ Human Action Recognition Using Attention Based Lstm Network With Dilated Cnn Features,” Future Generation Computer Systems 125 (2021): 820-830, https://doi.org/10.1016/j.future.2021.06.045.

[27]

S. Travin, O. Gromov, and G. Duca, “ Statistical Computational Model of Fission Products Composition of Irradiated Nuclear Fuel and Their Contribution to Gas-Aerosol Emissions of Nuclear Power Plants,” Romanian Journal of Information Science and Technology 27, no. 3-4 (2024): 310-322, https://doi.org/10.59277/romjist.2024.3-4.05.

[28]

M. S. Kiran and H. Hakli, “ A Tree-Seed Algorithm Based on Intelligent Search Mechanisms for Continuous Optimization,” Applied Soft Computing 98 (2021): 106938, https://doi.org/10.1016/j.asoc.2020.106938.

[29]

E. K. Zavadskas, H. Dnçer, S. Yüksel, and S. Et, “ Intelligent Expert Systems Using Molecular Fuzzy Genetic Algorithms and Multi-Objective Particle Swarm Optimization for Circular-Oriented Project Investments,” Romanian Journal of Information Science and Technology 28, no. 3 (2025): 260-273, https://doi.org/10.59277/romjist.2025.3.02.

[30]

K. I. Tsianos, S. Lawlor, and M. G. Rabbat, “ Push-Sum Distributed Dual Averaging for Convex Optimization,” in Proceedings of the IEEE Conference on Decision and Control, (2012), 5453-5458, https://doi.org/10.1109/TSMC.2022.3200680.

[31]

Y. Li, C. Chen, K. Zhang, et al., “ Q-dadam: A Quantized Distributed Online Optimization Algorithm With Adaptive Momentum,” IEEE Transactions on Control of Network Systems 12, no. 2 (2025): 1540-1551, https://doi.org/10.1109/TCNS.2025.3526555.

[32]

Q. Lü, X. Liao, T. Xiang, H. Li, and T. Huang, “ Privacy Masking Stochastic Subgradient-Push Algorithm for Distributed Online Optimization,” IEEE Transactions on Cybernetics 51, no. 6 (2021): 3224-3237, https://doi.org/10.1109/TCYB.2020.2973221.

[33]

K. Zhang, X. Liao, and Q. Lü, “ Privacy-Protected Decentralized Dual Averaging Push With Edge-Based Correlated Perturbations Over Time-Varying Directed Networks,” IEEE Transactions on Network Science and Engineering 9, no. 6 (2022): 4145-4158, https://doi.org/10.1109/TNSE.2022.3195953.

[34]

L. Bottou, F. E. Curtis, and J. Nocedal, “ Optimization Methods for Large-Scale Machine Learning,” SIAM Review 60, no. 2 (2018): 223-311, https://doi.org/10.1137/16m1080173.

[35]

H. J. Kushner and G. G. Yin, Stochastic Approximation and Recursive Algorithms and Applications (Springer Science & Business Media, 2003), https://doi.org/10.1007/b97441.

[36]

S. Bubeck, “ Convex Optimization: Algorithms and Complexity,” Foundations and Trends in Machine Learning 8, no. 3-4 (2015): 231-357, https://doi.org/10.1561/2200000050.

[37]

P. J. Huber, “ Robust Estimation of a Location Parameter,” Annals of Mathematical Statistics 35, no. 1 (1964): 73-101, https://doi.org/10.1007/978-1-4612-4380-9_35.

[38]

J. A. Nelder and R. W. M. Wedderburn, “ Generalized Linear Models,” Journal of the Royal Statistical Society: Series A 135, no. 3 (1972): 370-384, https://doi.org/10.2307/2344614.

[39]

G. Lan, S. Lee, and Y. Zhou, “ Communication-Efficient Algorithms for Decentralized and Stochastic Optimization,” Mathematical Programming 180, no. 1-2 (2020): 237-284, https://doi.org/10.1007/s10107-018-1355-4.

[40]

Q. Lü, X. Liao, H. Li, and T. Huang, “A Computation-Efficient Decentralized Algorithm for Composite Constrained Optimization,” IEEE Transactions on Signal and Information Processing over Networks 6, no. 6 (2020): 774-789, https://doi.org/10.1109/TSIPN.2020.3037837.

[41]

A. Nedić and A. Olshevsky, Distributed Optimization Over Time-Varying Directed Graphs, (2013), https://doi.org/10.48550/arXiv.1303.2289.

[42]

Y. Wang, L. Lin, and J. Chen, “ Communication-Compressed Adaptive Gradient Method for Distributed Nonconvex Optimization,” Proceedings of the 25th International Conference on Artificial Intelligence and Statistics (AISTATS) 28-30 (2022): 6292-6320, https://doi.org/10.48550/arXiv.2111.00705.

[43]

Y. Niu, H. Li, Z. Wang, Q. Lu, D. Xia, and L. Ji, “ A Distributed Stochastic Proximal-Gradient Algorithm for Composite Optimization,” IEEE Transactions on Control of Network Systems 8, no. 3 (2021): 1383-1393, https://doi.org/10.1109/TCNS.2021.3065653.

[44]

Z. Yang, W. He, and Y. Yuan, “ Differentially Private Distributed Optimization Over Diagraphs With Application to Image Deblurring,” IEEE Transactions on Control of Network Systems 12, no. 1 (2024): 634-647, https://doi.org/10.1109/TCNS.2024.3395851.

[45]

D. Ren, W. Zuo, D. Zhang, L. Zhang, and M. H. Yang, “ Simultaneous Fidelity and Regularization Learning for Image Restoration,” IEEE Transactions on Pattern Analysis and Machine Intelligence 43, no. 1 (2019): 284-299, https://doi.org/10.1109/TPAMI.2019.2926357.

PDF (2864KB)

0

Accesses

0

Citation

Detail

Sections
Recommended

/

〈 〉