The limited memory conjugate gradient method

Document Type

Article

Publication Date

12-1-2013

Abstract

In theory, the successive gradients generated by the conjugate gradient method applied to a quadratic should be orthogonal. However, for some ill-conditioned problems, orthogonality is quickly lost due to rounding errors, and convergence is much slower than expected. A limited memory version of the nonlinear conjugate gradient method is developed. The memory is used to both detect the loss of orthogonality and to restore orthogonality. An implementation of the algorithm is presented based on the CG-DESCENT nonlinear conjugate gradient method. Limited memory CG-DESCENT (L-CG-DESCENT) possesses a global convergence property similar to that of the memoryless algorithm but has much better practical performance. Numerical comparisons to the limited memory BFGS method (L-BFGS) are given using the CUTEr test problems. © 2013 Society for Industrial and Applied Mathematics.

Publication Source (Journal or Book title)

SIAM Journal on Optimization

First Page

2150

Last Page

2168

This document is currently not available here.

Share

COinS