×

A conjugate gradient method with descent direction for unconstrained optimization. (English) Zbl 1179.65075

The authors give a modified conjugate gradient method with the Wolfe-Powell rule for unconstrained optimization problems.
This method has the following properties: (i) The sufficient descent property is satisfied without any line search; (ii) The search direction will be in a trust region automatically; (iii) The Zoutendijk condition holds for the Wolfe-Powell line search technique; (iv) This method inherits an important property of the well-known Polak-Ribiere-Polyak method: the tendency to turn towards the steepest descent direction if a small step is generated away from the solution, preventing a sequence of tiny steps from happening.
The global convergence is established and the linearly convergent rate of the given method is established for convex functions. Numerical results show that this method is interesting.

MSC:

65K05 Numerical mathematical programming methods
90C30 Nonlinear programming

Software:

minpack
PDFBibTeX XMLCite
Full Text: DOI

References:

[1] Dai, Y., A nonmonotone conjugate gradient algorithm for unconstrained optimization, J. Syst. Sci. Complex., 15, 139-145 (2002) · Zbl 1019.90039
[2] Wei, Z.; Li, G.; Qi, L., New nonlinear conjugate gradient formulas for large-scale unconstrained optimization problems, Appl. Math. Comput., 179, 407-430 (2006) · Zbl 1106.65055
[3] Wei, Z.; Yao, S.; Liu, L., The convergence properties of some new conjugate gradient methods, Appl. Math. Comput., 183, 1341-1350 (2006) · Zbl 1116.65073
[4] Yuan, G. L.; Lu, X. W., A new line search method with trust region for unconstrained optimization, Commun. Appl. Nonlinear Anal., 1, 15, 35-49 (2008) · Zbl 1146.49031
[5] Yuan, G. L.; Lu, X. W.; Wei, Z. X., New two-point stepsize gradient methods for solving unconstrained optimization problems, Natur. Sci. J. Xiangton Univ., 1, 29, 13-15 (2007) · Zbl 1164.65429
[6] Yuan, G. L.; Wei, Z. X., New line search methods for unconstrained optimization, J. Korean Statist. Soc., 38, 29-39 (2009) · Zbl 1293.65098
[7] Yuan, G. L.; Wei, Z. X., The superlinear convergence analysis of a nonmonotone BFGS algorithm on convex objective functions, Acta Math. Sin. (Engl. Ser.), 1, 24, 35-42 (2008) · Zbl 1152.65072
[8] G.L. Yuan, Z.X. Wei, Convergence analysis of a modified BFGS method on convex minimizations, Comput. Optim. Appl., doi:10.1007/s10589-008-9219-0; G.L. Yuan, Z.X. Wei, Convergence analysis of a modified BFGS method on convex minimizations, Comput. Optim. Appl., doi:10.1007/s10589-008-9219-0 · Zbl 1228.90077
[9] Yuan, G. L.; Wei, Z. X., A rank-one fitting method for unconstrained optimization problems, Math. Appl., 1, 22, 118-122 (2009) · Zbl 1183.90408
[10] Dai, Y.; Yuan, Y., A nonlinear conjugate gradient with a strong global convergence properties, SIAM J. Optim., 10, 177-182 (2000)
[11] Fletcher, R., Practical Method of Optimization, Vol I: Unconstrained Optimization (1997), Wiley: Wiley New York
[12] Fletcher, R.; Reeves, C., Function minimization bu conjugate gradients, Compute. J., 7, 149-154 (1964) · Zbl 0132.11701
[13] Hestenes, M. R.; Stiefel, E., Method of conjugate gradient for solving linear equations, J, Res. Nat. Bur. Stand., 49, 409-436 (1952) · Zbl 0048.09901
[14] Liu, Y.; Storey, C., Effcient generalized conjugate gradient algorithms, part 1: Theory, J. Optim. Theory Appl., 69, 17-41 (1992)
[15] Polak, E.; Ribiere, G., Note sur la xonvergence de directions conjugees, Rev. Francaise informat Recherche Operatinelle, 3e Annee, 16, 35-43 (1969) · Zbl 0174.48001
[16] Dai, Y.; Yuan, Y., Nonlinear Conjugate Gradient Methods (2000), Science Press of Shanghai: Science Press of Shanghai Shanghai · Zbl 1030.90141
[17] Yuan, Y.; Sun, W., Theory and Methods of Optimization (1999), Science Press of China: Science Press of China Beijing
[18] Powell, M. J.D., Nonconvex minimization calculations and the conjugate gradient method, (Lecture Notes in Mathematics, vol. 1066 (1984), Spinger-Verlag: Spinger-Verlag Berlin), 122-141 · Zbl 0531.65035
[19] Gibert, J. C.; Nocedal, J., Global convergence properties of conugate gradient methods for optimization, SIAM J. Optimiz., 2, 21-42 (1992)
[20] Dai, Y.; Yuan, Y., An efficient hybrid conjugate gradient method for unconstrained optimization, Ann. Oper. Res., 103, 33-47 (2001) · Zbl 1007.90065
[21] Ahmed, T.; Storey, D., Efficient hybrid conjugate gradient techniques, J. Optim. Theory Appl., 64, 379-394 (1990) · Zbl 0666.90063
[22] Al-Baali, A., Descent property and global convergence of the Flecher-Reeves method with inexact line search, IMA J. Numer. Anal., 5, 121-124 (1985) · Zbl 0578.65063
[23] Hu, Y. F.; Storey, C., Global convergence result for conjugate method, J. Optim. Theory Appl., 71, 399-405 (1991) · Zbl 0794.90063
[24] Grippo, L.; Lucidi, S., A globally convergent version of the Polak-Ribière gradient method, Math. Program., 78, 375-391 (1997) · Zbl 0887.90157
[25] Hager, W. W.; Zhang, H. C., A new conjugate gradient method with guaranteed descent and an efficient line search, SIAM J. Optim., 1, 16, 170-192 (2005) · Zbl 1093.90085
[26] G.H. Yu, Nonlinear self-scaling conjugate gradient methods for large-scale optimization problems, Thesis of Doctor’s Degree, Sun Yat-Sen University, 2007; G.H. Yu, Nonlinear self-scaling conjugate gradient methods for large-scale optimization problems, Thesis of Doctor’s Degree, Sun Yat-Sen University, 2007
[27] Yuan, G. L., Modified nonlinear conjugate gradient methods with sufficient descent property for large-scale optimization problems, Optim. Lett., 3, 11-21 (2009) · Zbl 1154.90623
[28] Yuan, G. L.; Lu, X. W., A modified PRP conjugate gradient method, Ann. Oper. Res., 166, 73-90 (2009) · Zbl 1163.90798
[29] Zhang, L.; Zhou, W.; Li, D., A descent modified Polak-Ribière-Polyak conjugate method and its global convergence, IMA J. Numer. Anal., 26, 629-649 (2006) · Zbl 1106.65056
[30] Zoutendijk, G., Nonlinear programming computational methods, (Abadie, J., Integer and Nonlinear Programming (1970), NorthHolllad: NorthHolllad Amsterdam), 37-86 · Zbl 0336.90057
[31] Ortega, J. M.; Rheinboldt, W. C., Iterative Solution of Nonlinear Equation in Seveal Variables (1970), Academic Press · Zbl 0241.65046
[32] Rockafellar, R. T., Convex Analysis (1970), Princeton University Press · Zbl 0229.90020
[33] Moré, J. J.; Garbow, B. S.; Hillstrome, K. E., Testing unconstrained optimization software, ACM Trans. Math. Software., 7, 17-41 (1981) · Zbl 0454.65049
[34] Dolan, E. D.; Moré, J. J., Benchmarking optimization software with performance profiles, Math. Program., 91, 201-213 (2002) · Zbl 1049.90004
[35] Shi, Z. J., Convergence of line search methods for unconstrained optimization, Appl. Math. Comput., 157, 393-405 (2004) · Zbl 1072.65087
[36] Zhang, H. C.; Hager, W. W., A nonmonotone line search technique and its application to unconstrained optimization, SIAM J. Optim, 4, 14, 1043-1056 (2004) · Zbl 1073.90024
This reference list is based on information provided by the publisher or from digital mathematics libraries. Its items are heuristically matched to zbMATH identifiers and may contain data conversion errors. In some cases that data have been complemented/enhanced by data from zbMATH Open. This attempts to reflect the references listed in the original paper as accurately as possible without claiming completeness or a perfect matching.