MODIFIED LIMITED MEMORY BFGS METHOD WITH NONMONOTONE LINE SEARCH FOR UNCONSTRAINED OPTIMIZATION
In this paper, we propose two limited memory BFGS algorithms with a nonmonotone line search technique for unconstrained optimization problems. The global convergence of the given methods will be established under suitable conditions. Numerical results show that the presented algorithms are more competitive than the normal BFGS method.
- C. G. Broyden, J. E. Dennis Jr, and J. J. More, On the local and superlinear convergence of quasi-Newton methods, J. Inst. Math. Appl. 12 (1973), 223-245.
- R. H. Byrd and J. Nocedal, A tool for the analysis of quasi-Newton methods with application to unconstrained minimization, SIAM J. Numer. Anal. 26 (1989), no. 3, 727-739.
- R. H. Byrd, J. Nocedal, and R. B. Schnabel, Representations of quasi-Newton matrices and their use in limited memory methods, Math. Programming 63 (1994), no. 2, Ser. A, 129-156.
- R. Byrd, J. Nocedal, and Y. Yuan, Global convergence of a class of quasi-Newton methods on convex problems, SIAM J. Numer. Anal. 24 (1987), no. 5, 1171-1190.
- Y. Dai, Convergence properties of the BFGS algorithm, SIAM J. Optim. 13 (2002), no. 3, 693-701.
- Y. Dai and Q. Ni, Testing different conjugate gradient methods for large-scale unconstrained optimization, J. Comput. Math. 21 (2003), no. 3, 311-320.
- W. C. Davidon, Variable metric methods for minimization, A. E. C. Research and Development Report ANL-599, 1959.
- R. Dembo and T. Steihaug, Truncated Newton algorithms for large-scale unconstrained optimization, Math. Programming 26 (1983), no. 2, 190-212.
- J. E. Dennis Jr. and J. J. More, Quasi-Newton methods, motivation and theory, SIAM Rev. 19 (1977), no. 1, 46-89.
- J. E. Dennis Jr. and J. J. More, A characterization of superlinear convergence and its application to quasi-Newton methods, Math. Comp. 28 (1974), 549-560.
- J. E. Dennis Jr. and R. B. Schnabel, Numerical Methods for Unconstrained Optimization and Nonlinear Equations, Prentice Hall Series in Computational Mathematics. Prentice Hall, Inc., Englewood Cliffs, NJ, 1983.
- L. C. W. Dixon, Variable metric algorithms: necessary and sufficient conditions for identical behavior of nonquadratic functions, J. Optimization Theory Appl. 10 (1972), 34-40.
- E. D. Dolan and J. J. More, Benchmarking optimization software with performance profiles, Math. Program. 91 (2002), no. 2, Ser. A, 201-213.
- R. Fletcher, Practical Methods of Optimization, Second edition. A Wiley-Interscience Publication. John Wiley & Sons, Ltd., Chichester, 1987.
- N. I. M. Gould, D. Orban, and Ph. L. Toint, CUTEr (and SifDec), a constrained and unconstrained testing environment, revisite, ACM Transactions on Mathematical Software 29 (2003), 373-394.
- A. Griewank, The global convergence of partitioned BFGS on problems with convex decompositions and Lipschitzian gradients, Math. Programming 50 (1991), no. 2, (Ser. A), 141-175.
- A. Griewank and Ph. L. Toint, Local convergence analysis for partitioned quasi-Newton updates, Numer. Math. 39 (1982), no. 3, 429-448.
- L. Grippo, F. Lamparillo, and S. Lucidi, A nonmonotone line search technique for Newton's method, SIAM J. Numer. Anal. 23 (1986), no. 4, 707-716.
- L. Grippo, F. Lamparillo, and S. Lucidi, A truncated Newton method with nonmonotone line search for unconstrained optimization, J. Optim. Theory Appl. 60 (1989), no. 3, 401-419.
- L. Grippo, F. Lamparillo, and S. Lucidi, A class of nonmonotone stabilization methods in unconstrained optimization, Numer. Math. 59 (1991), no. 8, 779-805.
- J. Y. Han and G. H. Liu, Global convergence analysis of a new nonmonotone BFGS algorithm on convex objective functions, Comput. Optim. Appl. 7 (1997), no. 3, 277-289.
- D. Li and M. Fukushima, A modified BFGS method and its global convergence in non-convex minimization, J. Comput. Appl. Math. 129 (2001), no. 1-2, 15-35.
- L. Grippo, F. Lamparillo, and S. Lucidi, On the global convergence of the BFGS method for nonconvex unconstrained optimization problems, SIAM J. Optim. 11 (2001), no. 4, 1054-1064.
- G. Li, C. Tang, and Z.Wei, New conjugacy condition and related new conjugate gradient methods for unconstrained optimization, J. Comput. Appl. Math. 202 (2007), no. 2, 523-539.
- G. H. Liu and J. Y. Han, Notes on the general form of stepsize selection, OR and Decision Making I (1992), 619-624.
- G. H. Liu and J. Y. Han, Global convergence Analysis of the variable metric algorithm with a generalized Wolf linesearch, Technical Report, Institute of Applied Mathematics, Academia Sinica, Beijing, China, no. 029, 1993.
- G. H. Liu, J. Y. Han, and D. F. Sun, Global convergence of the BFGS algorithm with nonmonotone linesearch, Optimization 34 (1995), no. 2, 147-159.
- G. H. Liu and J. M. Peng, The convergence properties of a nonmonotonic algorithm, J. Comput. Math. 1 (1992), 65-71.
- W. F. Mascarenhas, The BFGS method with exact line searches fails for non-convex objective functions, Math. Program. 99 (2004), no. 1, Ser. A, 49-61.
- J. J. More, B. S. Garbow, and K. E. Hillstrome, Testing unconstrained optimization software, ACM Trans. Math. Software 7 (1981), no. 1, 17-41.
- S. G. Nash, A survey of truncated-Newton methods, Numerical analysis 2000, Vol. IV, Optimization and nonlinear equations. J. Comput. Appl. Math. 124 (2000), no. 1-2, 45-59.
- M. J. D. Powell, On the convergence of the variable metric algorithm, J. Inst. Math. Appl. 7 (1971), 21-36.
- M. J. D. Powell, Some global convergence properties of a variable metric algorithm for minimization without exact line searches, Nonlinear programming (Proc. Sympos., New York, 1975), pp. 53？72. SIAM-AMS Proc., Vol. IX, Amer. Math. Soc., Providence, R. I., 1976.
- M. J. D. Powell, A new algorithm for unconstrained optimization, 1970 Nonlinear Programming (Proc. Sympos., Univ. of Wisconsin, Madison, Wis., 1970) pp. 31-65 Academic Press, New York.
- M. Raydan, The Barzilai and Borwein gradient method for the large scale unconstrained minimization problem, SIAM J. Optim. 7 (1997), no. 1, 26-33.
- J. Schropp, A note on minimization problems and multistep methods, Numer. Math. 78 (1997), no. 1, 87-101.
- J. Schropp, One-step and multistep procedures for constrained minimization problems, IMA J. Numer. Anal. 20 (2000), no. 1, 135-152.
- Ph. L. Toint, Global convergence of the partitioned BFGS algorithm for convex partially separable optimization, Math. Programming 36 (1986), no. 3, 290-306.
- D. J. van Wyk, Differential optimization techniques, Appl. Math. Modelling 8 (1984), no. 6, 419-424.
- M. N. Vrahatis, G. S. Androulakis, J. N. Lambrinos, and G. D. Magolas, A class of gradient unconstrained minimization algorithms with adaptive stepsize, J. Comput. Appl. Math. 114 (2000), no. 2, 367-386.
- Z. Wei, G. Li, and L. Qi, New quasi-Newton methods for unconstrained optimization problems, Appl. Math. Comput. 175 (2006), no. 2, 1156-1188.
- Z. Wei, G. Yu, G. Yuan, and Z. Lian, The superlinear convergence of a modified BFGS-type method for unconstrained optimization, Comput. Optim. Appl. 29 (2004), no. 3, 315-332.
- G. L. Yuan, Modified nonlinear conjugate gradient methods with sufficient descent property for large-scale optimization problems, Optim. Lett. 3 (2009), no. 1, 11-21.
- G. L. Yuan and X. W. Lu, A new line search method with trust region for unconstrained optimization, Comm. Appl. Nonlinear Anal. 15 (2008), no. 1, 35-49.
- G. L. Yuan and X. W. Lu, A modified PRP conjugate gradient method, Ann. Oper. Res. 166 (2009), 73-90.
- G. L. Yuan, X. Lu, and Z. Wei, A conjugate gradient method with descent direction for unconstrained optimization, J. Comput. Appl. Math. 233 (2009), no. 2, 519-530.
- Y. Yuan and W. Sun, Theory and Methods of Optimization, Science Press of China, 1999.
- G. L. Yuan and Z. X. Wei, New line search methods for unconstrained optimization, J. Korean Statist. Soc. 38 (2009), no. 1, 29-39.
- G. L. Yuan and Z. X. Wei, The superlinear convergence analysis of a nonmonotone BFGS algorithm on convex objective functions, Acta Math. Sin. (Engl. Ser.) 24 (2008), no. 1, 35-42.
- G. L. Yuan and Z. X. Wei, Convergence analysis of a modified BFGS method on convex minimizations, Comput. Optim. Appl. doi: 10.1007/s10589-008-9219-0.
- J. Z. Zhang, N. Y. Deng, and L. H. Chen, New quasi-Newton equation and related methods for unconstrained optimization, J. Optim. Theory Appl. 102 (1999), no. 1, 147-167.
유료 다운로드의 경우 해당 사이트의 정책에 따라 신규 회원가입, 로그인, 유료 구매 등이 필요할 수 있습니다. 해당 사이트에서 발생하는 귀하의 모든 정보활동은 NDSL의 서비스 정책과 무관합니다.
원문복사신청을 하시면, 일부 해외 인쇄학술지의 경우 외국학술지지원센터(FRIC)에서
무료 원문복사 서비스를 제공합니다.
NDSL에서는 해당 원문을 복사서비스하고 있습니다. 위의 원문복사신청 또는 장바구니 담기를 통하여 원문복사서비스 이용이 가능합니다.
- 이 논문과 함께 출판된 논문 + 더보기