Coevolutionary Neural Dynamics With Learnable Parameters for Nonconvex Optimisation

Abstract

Nonconvex optimisation plays a crucial role in science and industry. However, existing methods often encounter local optima or provide inferior solutions when solving nonconvex optimisation problems, lacking robustness in noise scenarios. To address these limitations, we aim to develop a robust, efficient and globally convergent solver for nonconvex optimisation. This is achieved by combining the efficient local exploitation ability of a parameter-learnt neural dynamics (PLND) model with the global search capability of the coevolutionary mechanism. We combine their characteristics to design a coevolutionary neural dynamics with learnable parameters (CNDLP) model. The gradient information is used to find the optimal solution more effectively, and neural dynamics models have robustness, which ensures that the infiuence of noise can be effectively sup-pressed in the calculation process. Theoretical analyses show the global convergence and robustness of the designed CNDLP model. Numerical experiments on 9 benchmark functions and a practical engineering design example are conducted with five existing meta-heuristic algorithms. Benchmarks cover diverse problems, from classical landscapes like benchmark Shubert to high-dimensional cases such as 30-dimensional Rosenbrock. Results confirm CNDLP's excellent performance in both solution quality and convergence speed under noise.

Keywords

coevolutionary neural dynamics with learnable parameters (CNDLP) / nonconvex optimization / robustness

Conflicts of Interest

The authors declare no confiicts of interest.

Data Availability Statement

Data will be made available on request.

References

[1] M. Wang, Y. Xie, and S. Qin, “An Adaptive Memristor-Programming Neurodynamic Approach to Nonsmooth Nonconvex Optimization Problems,” IEEE Transactions on Systems, Man, and Cybernetics: Sys-tems 53, no. 11 (2023): 6874-6885, https://doi.org/10.1109/tsmc.2023.3287237.

[2] Z. Xia, Y. Liu, W. Yu, and J. Wang, “A Collaborative Neurodynamic Optimization Approach to Distributed Nash-Equilibrium Seeking in Multicluster Games With Nonconvex Functions,” IEEE Transactions on Cybernetics 54, no. 5 (2024): 3105-3119, https://doi.org/10.1109/tcyb.2023.3289712.

[3] Y. Luo, X. Deng, J. Wu, Y. Liu, and Z. Zhang, “A New Finite-Time Circadian Rhythms Learning Network for Solving Nonlinear and Non-convex Optimization Problems With Periodic Noises,” IEEE Trans-actions on Cybernetics 52, no. 11 (2022): 12514-12524, https://doi.org/10.1109/tcyb.2021.3071764.

[4] J. Zhang, M. Liu, and L. Jin, “Logistic Adaptive Controller With Overhead Reduction for Multirobot Systems,” IEEE Transactions on Industrial Electronics 72, no. 1 (2025): 660-669, https://doi.org/10.1109/tie.2024.3404155.

[5] J. Tan, M. Shang, and L. Jin, “Metaheuristic-Based RNN for Manipulability Optimization of Redundant Manipulators,” IEEE Transactions on Industrial Informatics 20, no. 4 (2024): 6489-6498, https://doi.org/10.1109/tii.2023.3348830.

[6] X. Cao and S. Li, “A Novel Dynamic Neural System for Nonconvex Portfolio Optimization With Cardinality Restrictions,” IEEE Trans-actions on Systems, Man, and Cybernetics: Systems 53, no. 11 (2023): 6943-6952, https://doi.org/10.1109/tsmc.2023.3288224.

[7] W. Zhou, H.-T. Zhang and J. Wang, “Sparse Bayesian Learning Based on Collaborative Neurodynamic Optimization,” IEEE Trans-actions on Cybernetics 52, no. 12 (2022): 13669-13683, https://doi.org/10.1109/tcyb.2021.3090204.

[8] S. T. Thomdapu, H. Vardhan, and K. Rajawat, “Stochastic Compo-sitional Gradient Descent Under Compositional Constraints,” IEEE Transactions on Signal Processing 71 (2023): 1115-1127, https://doi.org/10.1109/tsp.2023.3244326.

[9] H. Ye, L. Luo and Z. Zhang, “Accelerated Proximal Subsampled Newton Method,” IEEE Transactions on Neural Networks and Learning Systems 32, no. 10 (2021): 4374-4388, https://doi.org/10.1109/tnnls.2020.3017555.

[10] J. Zhang, H. Liu, A. M.-C. So and Q. Ling, “Variance-Reduced Stochastic Quasi-Newton Methods for Decentralized Learning,” IEEE Transactions on Signal Processing 71 (2023): 311-326, https://doi.org/10.1109/tsp.2023.3240652.

[11] Y. Cui, Y. Li, and C. Ye, “Sample-Based and Feature-Based Feder-ated Learning for Unconstrained and Constrained Nonconvex Optimi-zation via Mini-Batch SSCA,” IEEE Transactions on Signal Processing 70 (2022): 3832-3847, https://doi.org/10.1109/tsp.2022.3185895.

[12] W. Ji, J. Qiu, and H.-K. Lam, “A New Sampled-Data Output-Feed-back Controller Design of Nonlinear Systems via Fuzzy Affine Models,” IEEE Transactions on Cybernetics 52, no. 3 (2022): 1681-1690, https://doi.org/10.1109/tcyb.2020.2984331.

[13] C. Xiong, C. Zhang, M. Lu, et al., “Convex Dual Theory Analysis of Two-Layer Convolutional Neural Networks With Soft-Thresholding,” IEEE Transactions on Neural Networks and Learning Systems 36, no. 2 (2025): 3423-3435, https://doi.org/10.1109/tnnls.2024.3353795.

[14] H. Jiang, K. Chou, Y. Tian, X. Zhang, and Y. Jin, “Efficient Surro-gate Modeling Method for Evolutionary Algorithm to Solve Bilevel Optimization Problems,” IEEE Transactions on Cybernetics 54, no. 7 (2024): 4335-4347, https://doi.org/10.1109/tcyb.2023.3309598.

[15] A. Rosales-Pérez, S. García, and F. Herrera, “Handling Imbalanced Classification Problems With Support Vector Machines via Evolutionary Bilevel Optimization,” IEEE Transactions on Cybernetics 53, no. 8 (2023): 4735-4747, https://doi.org/10.1109/tcyb.2022.3163974.

[16] Y. Xiao, H. Cui, A. G. Hussien, and F. A. Hashim, “MSAO: A multi- strategy Boosted Snow Ablation Optimizer for Global Optimization and Real-World Engineering Applications,” Advanced Engineering Infor-matics 61 (2024): 102464, https://doi.org/10.1016/j.aei.2024.102464.

[17] Y. Xiao, H. Cui, R. A. Khurma, and P. A. Castillo, “Artificial Lemming Algorithm: A Novel Bionic Meta-Heuristic Technique for Solving Real-World Engineering Optimization Problems,” Artificial In-telligence Review 58, no. 3 (2025): 1-36, https://doi.org/10.1007/s10462-024-11023-7.

[18] C. L. Camacho-Villalón, M. Dorigo, and T. Stützle, “PSO-X: A Component-Based Framework for the Automatic Design of Particle Swarm Optimization Algorithms,” IEEE Transactions on Evolutionary Computation 26, no. 3 (2022): 402-416, https://doi.org/10.1109/tevc.2021.3102863.

[19] N. Onizawa, K. Katsuki, D. Shin, W. J. Gross, and T. Hanyu, “Fast- Converging Simulated Annealing for Ising Models Based on Integral Stochastic Computing,” IEEE Transactions on Neural Networks and Learning Systems 34, no. 12 (2023): 10999-11005, https://doi.org/10.1109/tnnls.2022.3159713.

[20] Y. Jiang, Z.-H. Zhan, K. C. Tan, and J. Zhang, “Optimizing Niche Center for Multimodal Optimization Problems,” IEEE Transactions on Cybernetics 53, no. 4 (2023): 2544-2557, https://doi.org/10.1109/tcyb.2021.3125362.

[21] J. Xue and B. Shen, “A Novel Swarm Intelligence Optimization Approach: Sparrow Search Algorithm,” Syst. Sci. Control. Eng. 8, no. 1 (2020): 22-34, https://doi.org/10.1080/21642583.2019.1708830.

[22] S. Mirjalili and A. Lewis, “The Whale Optimization Algorithm ” Advances in Engineering Software 95 (2016): 51-67, https://doi.org/10.1016/j.advengsoft.2016.01.008.

[23] F. Salajegheh and E. S. PSOG, “Enhanced Particle Swarm Optimi-zation by a Unit Vector of First and Second Order Gradient Directions,” Swarm and Evolutionary Computation 46 (2019): 25-81, https://doi.org/10.1016/j.swevo.2019.01.010

[24] C. Ibtissem and L. Nouredine, “A Hybrid Method Based on Conjugate Gradient Trained Neural Network and Differential Evolution for Non Linear Systems Identification,” in Proceedings of the International Con-ference on Electrical Engineering and Software Applications, (2013), 1-5.

[25] A. K. Parwani, P. Talukdar, and P. Subbarao, “A Hybrid Approach Using CGM and DE Algorithm for Estimation of Boundary Heat Flux in a Parallel Plate Channel,” Numerical Heat Transfer Part A: Appications 65, no. 5 (2013): 461-481, https://doi.org/10.1080/10407782.2013.832063.

[26] L. Wei, L. Jin, and X. Luo, “A Robust Coevolutionary Neural-Based Optimization Algorithm for Constrained Nonconvex Optimization,” IEEE Transactions on Neural Networks and Learning Systems 35, no. 6 (2024): 7778-7791, https://doi.org/10.1109/tnnls.2022.3220806.

[27] J. Jin, J. Zhu, L. Zhao, L. Chen, L. Chen, and J. Gong, “A Robust Predefined-Time Convergence Zeroing Neural Network for Dynamic Matrix Inversion,” IEEE Transactions on Cybernetics 53, no. 6 (2023): 3887-3900, https://doi.org/10.1109/tcyb.2022.3179312.

[28] Z. Zeng, J. Wang, and X. Luo, “Global Exponential Stability of a General Class of Recurrent Neural Networks With Time-Varying De-lays,” IEEE Transactions on Circuits and Systems I: Fundamental Theory and Applications 50, no. 10 (2003): 1353-1358, https://doi.org/10.1109/TCSI.2003.817760.

[29] Y. Qi, L. Jin, X. Luo, and M. Zhou, “Recurrent Neural Dynamics Models for Perturbed Nonstationary Quadratic Programs: A Control- Theoretical Perspective,” IEEE Transactions on Neural Networks and Learning Systems 33, no. 3 (2022): 1216-1227, https://doi.org/10.1109/tnnls.2020.3041364.

[30] B. Qiu, J. Guo, X. Li, Z. Zhang, and Y. Zhang, “Discrete-Time Advanced Zeroing Neurodynamic Algorithm Applied to Future Equality-Constrained Nonlinear Optimization With Various Noises,” IEEE Transactions on Cybernetics 52, no. 5 (2022): 3539-3552, https://doi.org/10.1109/tcyb.2020.3009110.

[31] L. Jin, Z. Su, D. Fu, and X. Xiao, “Coevolutionary Neural Solution for Nonconvex Optimization With Noise Tolerance,” IEEE Transactions on Neural Networks and Learning Systems 35, no. 12 (2024): 17571-17581, https://doi.org/10.1109/tnnls.2023.3306374.

[32] Z. Zeng, J. Wang, and X. Luo, “Stability Analysis of Delayed Cellular Neural Networks Described Using Cloning Templates,” IEEE Trans-actions on Circuits and Systems I: Regular Papers 51, no. 11 (2004): 2313-2324, https://doi.org/10.1109/TCSI.2003.817760.

[33] T. Q. Chen, Y. Rubanova,J. Bettencourt, and D. K. Duvenaud, “Neural Ordinary Differential Equations,” in Advances in Neural In-formation Processing Systems, (2018), 6571-6583.

[34] L. Jin, Y. Zhang, S. Li, and Y. Zhang, “Noise-Tolerant ZNN Models for Solving Time-Varying Zero-Finding Problems: A Control-Theoretic Approach,” IEEE Transactions on Automatic Control 62, no. 2 (2017): 992-997, https://doi.org/10.1109/tac.2016.2566880.

[35] X. Chen, Z. Su, L. Jin, and S. Li, “A Correntropy-Based Echo State Network With Application to Time Series Prediction,” IEEE/CAA J. Autom. Sinica 12, no. 2 (2025): 425-435, https://doi.org/10.1109/jas.2024.124932.

[36] X. Yao, Y. Liu and G. Lin, “Evolutionary Programming Made Faster ” IEEE Transactions on Evolutionary Computation 3, no. 2 (1999): 82-102, https://doi.org/10.1109/4235.771163.

[37] Z. Yan, J. Fan, and J. Wang, “A Collective Neurodynamic Approach to Constrained Global Optimization,” IEEE Transactions on Neural Networks and Learning Systems 28, no. 5 (2017): 1206-1215, https://doi.org/10.1109/tnnls.2016.2524619.

[38] V. Kozyakin, “On Accuracy of Approximation of the Spectral Radius by the Gelfand Formula,” Linear Algebra Appl 431, no. 11 (2009): 2134-2141, https://doi.org/10.1016/j.laa.2009.07.008.

[39] R. Tanabe and A. S. Fukunaga, “Improving the Search Performance of SHADE Using Linear Population Size Reduction,” in Proceedings of the IEEE Congress on Evolutionary Computation, (2014), 1658-1665.

[40] K. Cymerys and M. Oszust, “Attraction-Repulsion Optimization Algorithm for Global Optimization Problems,” Swarm and Evolutionary Computation 84 (2024): 101459, https://doi.org/10.1016/j.swevo.2023.101459.

[41] R. Sowmya, M. Premkumar, and P. Jangir, “Newton-Raphson-Based Optimizer: A New Population-Based Metaheuristic Algorithm for Continuous Optimization Problems,” Engineering Applications of Artificial Intelligence 128 (2024): 107532, https://doi.org/10.1016/j.engappai.2023.107532.

[42] Y. Gao, “PID-based Search Algorithm: A Novel Metaheuristic Al-gorithm Based on PID Algorithm,” Expert Systems with Applications 232 (December 2023): 120886, https://doi.org/10.1016/j.eswa.2023.120886.

[43] M. Jamil and X. -S. Yang, “A Literature Survey of Benchmark Functions for Global Optimisation Problems,” International Journal of Mathematical Modelling and Numerical Optimisation 4, no. 2 (2013): 150-2194, https://doi.org/10.1504/ijmmno.2013.055204.

[44] L. Abualigah, D. Yousri, M. Abd Elaziz, A. A. Ewees, M. A. Al- qaness, and A. H. Gandomi, “Aquila Optimizer: A Novel Meta- Heuristic Optimization Algorithm,” Computers & Industrial Engineer-ing 157 (2021): 107250, https://doi.org/10.1016/j.cie.2021.107250.

Funding

Leading Talent of the Qinghai Province Kunlun Talents Programme ・ High-Level Innovative and Entrepreneurial Talents(QHKLYC-GDCXCY-2024-359)