A Modified Conjugate Gradient Method with Elastic Properties for Unconstrained Problem

Hayder Ali Mohsin  Al-Baidhani

Hayder Ali Mohsin Al-Baidhani University of Kashan Faculty of Mathematical Sciences Department of Applied Mathematics Thesis

Keywords: Modified Conjugate Gradient Method, Elastic Properties, Unconstrained Optimization, Sufficient Descent Condition

Abstract

In this paper, we suggest a new version of the conjugate gradient (CG) method that adds elastic features to improve its ability to solve problems where there are no limits on the solutions. Traditional Conjugate Gradient (CG) methods are effective but can become unstable and slow to find solutions when dealing with difficult problems that aren't well-structured or that don't fit a simple curve. Our suggested method includes a flexible adjustment system that changes how we search and the size of our steps. This helps keep things stable and makes the process faster. The method meets the basic requirements for going downhill and has been shown to work well overall based on common expectations. We tested a new method on a group of standard optimization problems without limits, comparing it to traditional CG methods. The results show that our method works better than others because it requires fewer steps and less time to compute. This proves that it is strong and effective for many different tests.

References

Y. Dai and Y. Yuan, Nonlinear Conjugate Gradient Methods. Shanghai, China: Shanghai Scientific and Technical Publishers, 2000.

R. Fletcher and C. Reeves, “Function minimization by conjugate gradients,” Comput. J., vol. 7, pp. 149–154, 1964.

M. Hestenes and E. Stiefel, “Methods of conjugate gradient for solving linear systems,” J. Res. Natl. Bur. Stand., vol. 49, pp. 409–436, 1952.

B. Polak and G. Ribière, “Note on the convergence of conjugate directions methods,” Rev. Fr. Inform. Rech. Oper., vol. 3, pp. 35–43, 1969.

B. Polyak, “The conjugate gradient method in extremal problems,” USSR Comput. Math. Math. Phys., vol. 9, pp. 94–112, 1969.

Y. Liu and C. Sorey, “Efficient generalized conjugate gradient algorithms. Part 1: Theory,” J. Optim. Theory Appl., vol. 69, pp. 177–182, 1991.

W. W. Hager and H. Zhang, “A survey of nonlinear conjugate gradient methods,” Pac. J. Optim., vol. 2, pp. 35–58, 2006.

J. Gibert and J. Nocedal, “Global convergence properties of conjugate gradient methods for optimization,” SIAM J. Optim., vol. 2, pp. 21–42, 1992.

Y. Yuan, “Analysis of the conjugate gradient method,” Optim. Method Softw., vol. 2, pp. 19–29, 1993.

M. Powell, “Nonconvex minimization calculations and the conjugate gradient method,” Numer. Anal. Lect. Notes Math., vol. 1066, pp. 122–141, 1984.

W. W. Hager and H. Zhang, “A new conjugate gradient method with guaranteed descent and an efficient line search,” SIAM J. Optim., vol. 16, pp. 170–192, 2005.

W. Cheng, “A two-term PRP-based descent method,” Numer. Funct. Anal. Optim., vol. 28, pp. 1217–1230, 2007.

G. Yu, L. Guan, and G. Li, “Global convergence of modified Polak-Ribiére-Polyak conjugate gradient methods with sufficient descent property,” J. Ind. Manag. Optim., vol. 4, pp. 565–579, 2008.

G. Yuan, “Modified nonlinear conjugate gradient methods with sufficient descent property for large-scale optimization problems,” Optim. Lett., vol. 3, pp. 11–21, 2009.

I. Livieris and P. Pintelas, “A new class of spectral conjugate gradient methods based on a modified secant equation for unconstrained optimization,” J. Comput. Appl. Math., vol. 239, pp. 396–405, 2013.

Z. Wei, S. Yao, and L. Liu, “The convergence properties of some new conjugate gradient methods,” Appl. Math. Comput., vol. 183, pp. 1341–1350, 2006.

L. Zhang, “An improved Wei-Yao-Liu nonlinear conjugate gradient method for optimization computation,” Appl. Math. Comput., vol. 215, pp. 2269–2274, 2009.

L. Zhang, W. Zhou, and D. Li, “A descent modified Polak-Ribière-Polyak conjugate gradient method and its global convergence,” IMA J. Numer. Anal., vol. 26, pp. 629–640, 2006.

X. Dong, H. Liu, Y. He, S. Babaie-Kafaki, and R. Ghanbari, “A new three-term conjugate gradient method with descent direction for unconstrained optimization,” Math. Model. Anal., vol. 21, pp. 399–411, 2016.

G. Zoutendijk, “Nonlinear programming, computational methods,” in Integer and Nonlinear Programming, J. Abadie, Ed. Amsterdam, The Netherlands: North-Holland, 1970, pp. 37–86.