An improved conjugate gradient algorithm by adapting a new line search technique

i

This article has been updated. A Correction to this article was published on .

i

The Original Article was published on

i

The Original Article was published on

i

This article has been updated

i

This article was retracted on

i

This article has been updated

Abstract

The conjugate gradient (CG) method is an optimization technique known for its rapid convergence; it has blossomed into significant developments and applications. Numerous variations of CG methods have emerged to enhance computational efficiency and address real-world challenges. This work presents a new conjugate gradient method for solving nonlinear unconstrained optimization problems by introducing a new conjugate gradient parameter. To improve the convergence properties, we have proposed a new inexact line search technique that fits in with the suggested approach and can also be useful for other gradient descent methods. The existence of a steplength that meets the new line search conditions is established. The generated descent direction and the convergence properties of the suggested approach are studied under the new line search conditions, where the global convergence is proven under mild assumptions. The proposed approach is evaluated on various test functions, and a comparison with recent similar algorithms is carried out. Furthermore, the proposed algorithm is applied for restoring images with different noise levels.

Keywords

optimization algorithms / conjugate gradient methods / inexact line search technique / global convergence / image processing

CHANGE HISTORY

A Correction to this paper has been published:
https://doi.org

The Retraction Note was updated to reflect reception of statements of disagreement from all authors.

This article has been retracted.Please see the Retraction Notice for more detail:
https://doi.org

References

[1] R. Ziadi, A. Bencherif-Madani, A perturbed quasi-Newton algorithm for bound-constrained global optimization, J. Comput. Math., 43 (2025), 143-173. https://doi.org/10.4208/jcm.2307-m2023-0016

[2] R. Ziadi, A. Bencherif-Madani, A mixed algorithm for smooth global optimization, J. Math. Model., 11 (2023), 207-228. https://doi.org/10.22124/JMM.2022.23133.2061

[3] O. O. O. Yousif, R. Ziadi, M. A. Saleh, A. Z. Almaymuni, Another updated parameter for the Hestenes-Stiefel conjugate gradient method, Int. J. Anal. Appl., 23 (2025), 10. https://doi.org/10.28924/2291-8639-23-2025-10

[4] C. Souli, R. Ziadi, A. Bencherif-Madani, H. M. Khudhur, A hybrid CG algorithm for nonlinear unconstrained optimization with application in image restoration, J. Math. Model., 12 (2024), 301-317. https://doi.org/10.22124/JMM.2024.26151.2317

[5] Y. Chen, H. Huang, K. Huang, M. Roohi, C. Tang, A selective chaos-driven encryption technique for protecting medical images, Phys. Scr., 100 (2025), 0152a3. https://doi.org/10.1088/1402-4896/ad9fad

[6] K. Xiong, S. Wang, The online random Fourier features conjugate gradient algorithm, IEEE Signal Process. Lett., 26 (2019), 740-744. https://doi.org/10.1109/LSP.2019.2907480

[7] M. A. Saleh, Enhancing deep learning optimizers for detecting malware using line search method under strong Wolfe conditions, in 2023 3rd International Conference on Computing and Information Technology (ICCIT), (2023), 222-226. https://doi.org/10.1109/ICCIT58132.2023.10273908

[8] M. Zhang, X. Wang, X. Chen, A. Zhang, The kernel conjugate gradient algorithms, IEEE Trans. Signal Process., 66 (2018), 4377-4387. https://doi.org/10.1109/TSP.2018.2853109

[9] K. Xiong, H. H. Lu, S. Wang, Kernel correntropy conjugate gradient algorithms based on half-quadratic optimization, IEEE Trans. Cyber., 55 (2020), 5497-5510. https://doi.org/10.1109/TCYB.2019.2959834

[10] R. Ziadi, R. Ellaia, A. Bencherif-Madani, Global optimization through a stochastic perturbation of the PolakC Ribière conjugate gradient method, J. Comput. Appl. Math., 317 (2017), 672-684. https://doi.org/10.1016/j.cam.2016.12.021

[11] A. Mehamdia, Y. Chaib, T. Bechouat, Two modified conjugate gradient methods for unconstrained optimization and applications, RAIRO Oper. Res., 57 (2023), 333-353. https://doi.org/10.1051/ro/2023010

[12] M. R. Hestenes, E. Stiefel, Methods of conjugate gradients for solving linear systems, J. Res. Natl. Bur. Stand., 49 (1952), 409-436.

[13] R. Fletcher, Practical Methods of Optimization , Wiley, New York, 1997.

[14] B. T. Polyak, The conjugate gradient method in extreme problems, USSR Comput. Math. Math. Phys., 9 (1969), 94-112. https://doi.org/10.1016/0041-5553(69)90035-4

[15] E. Polak, G. Ribiere, Note sur la convergence de méthodes de directions conjuguées, Rev. Française Inf. Rech. Opr., 16 (1969), 35-43.

[16] Y. Liu, C. Storey, Efficient generalized conjugate gradient algorithms. Part 1: Theory, J. Optim. Theory Appl., 69 (1992), 129-137. https://doi.org/10.1007/BF00940464

[17] Y. H. Dai, Y. Yuan, An efficient hybrid conjugate gradient method for unconstrained optimization, Ann. Oper. Res., 103 (2001), 33-47. https://doi.org/10.1023/A:1012930416777

[18] D. Ahmed, C. Storey, Efficient hybrid conjugate gradient techniques, J. Optim. Theory Appl., 64 (1990), 379-397. https://doi.org/10.1007/BF00939455

[19] J. K. Liu, S. J. Li, New hybrid conjugate gradient method for unconstrained optimization, Appl. Math. Comput., 245 (2014), 36-43. https://doi.org/10.1016/j.amc.2014.07.096

[20] N. Andrei, Another nonlinear conjugate gradient algorithm for unconstrained optimization, Optim. Meth. Soft., 24 (2008), 89-104. https://doi.org/10.1080/10556780802393326

[21] X. Y. Zheng, X. L. Dong, J. R. Shi, W. Yang, Further comment on another hybrid conjugate gradient algorithm for unconstrained optimization by Andrei, Numer. Algorithms, 84 (2019), 603-608. https://doi.org/10.1007/s11075-019-00771-1

[22] S. S. Djordjevic New hybrid conjugate gradient method as a convex combination of LS and CD methods, Filomat, 31 (2017), 1813-1825. https://doi.org/10.1007/s10473-019-0117-6

[23] C. Souli, R. Ziadi, I. E. Lakhdari, A. Leulmi, An efficient hybrid conjugate gradient method for unconstrained optimization and image restoration problems, Iran. J. Numer. Anal. Optim., 15 (2024), 99-123. https://doi.org/10.1109/LSP.2019.290748010.22067/ijnao.2024.88087.1449

[24] M. Rivaie, M. Mamat, L. W. June, I. Mohd, A new class of nonlinear conjugate gradient coefficients with global convergence properties, Appl. Math. Comput., 218 (2012), 11323-11332. https://doi.org/10.1016/j.amc.2012.05.030

[25] Z. X. Wei, S. W. Yao, L. Y. Liu, The convergence properties of some new conjugate gradient methods, Appl. Math. Comput., 183 (2006), 1341-1350. https://doi.org/10.1016/j.amc.2006.05.150

[26] L. Zhang, An improved Wei-Yao-Liu nonlinear conjugate gradient method for optimization computation, J. Appl. Math. Comput., 215 (2009), 2269-2274. https://doi.org/10.1016/j.amc.2009.08.016

[27] I. M. Sulaiman, N. A. Bakar, M. Mamat, B. A. Hassan, M. Malik, M. A. Ahmed, A new hybrid conjugate gradient algorithm for optimization models and its application to regression analysis, J. Electr. Eng. Comput. Sci., 23 (2021), 1100-1109. https://doi.org/10.11591/ijeecs.v23.i2.pp1100-1109

[28] S. Yao, Z. Wei, H. Huang, A note about WYLs conjugate gradient method and its applications, Appl. Math. Comput., 191 (2007), 381-388. https://doi.org/10.1016/j.amc.2007.02.094

[29] I. Bongartz, A. R. Conn, N. Gould, P. L. Toint, CUTEr: A constrained and unconstrained testing environment, ACM Trans. Math. Softw. 29 (2003), 373-394. https://doi.org/10.1145/962437.96243

[30] N. Andrei, An unconstrained optimization test functions, Adv. Mode. Optim., 10 (2008), 147-161.

[31] M. Hamoda, M. Mamat, M. Rivaie, Z. Salleh, A conjugate gradient method with strong Wolfe-Powell line search for unconstrained optimization, Appl. Math. Sci., 10 (2016), 13-16. http://dx.doi.org/10.12988/ams.2016.56449

[32] I. E. Livieris, P. Pintelas, Globally convergent modified Perrys conjugate gradient method, Appl. Math. Comput., 218 (2012), 9197-9207. https://doi.org/10.1016/j.amc.2012.02.076

[33] W. W. Hager, H. Zhang, A new conjugate gradient method with guaranteed descent and an efficient line search, SIAM J. Optim., 16 (2005), 170-192. https://doi.org/10.1137/030601880

[34] E. D. Dolan, J. J. Mor, Benchmarking optimization software with performance profiles, Math. Program., 91 (2002), 201-213. https://doi.org/10.1007/s101070100263