BFGS with Update Skipping and Varying Memory

Abstract

We give conditions under which limited-memory quasi-Newton methods with exact line searches will terminate in n steps when minimizing n-dimensional quadratic functions. We show that although all Broyden family methods terminate in n steps in their full-memory versions, only BFGS does so with limited-memory. Additionally, we show that full-memory Broyden family methods with exact line searches terminate in at most n + p steps when p matrix updates are skipped. We introduce new limited-memory BFGS variants and test them on nonquadratic minimization problems.

Publication
SIAM Journal on Optimization
Date
Citation
T. G. Kolda, D. P. O’Leary, L. Nazareth. BFGS with Update Skipping and Varying Memory. SIAM Journal on Optimization, Vol. 8, No. 4, pp. 1060-1083, 1998. https://doi.org/10.1137/S1052623496306450

Keywords

minimization, quasi-Newton, BFGS, limited-memory, update skipping, Broyden family

Comments

The companion web page (with source code) is http://www.cs.umd.edu/~oleary/LBFGS/.

BibTeX

@article{KoOlNa98,  
author = {Tamara G. Kolda and Dianne P. O'Leary and Larry Nazareth}, 
title = {{BFGS} with Update Skipping and Varying Memory}, 
journal = {SIAM Journal on Optimization}, 
volume = {8}, 
number = {4}, 
pages = {1060--1083}, 
month = {November}, 
year = {1998},
doi = {10.1137/S1052623496306450},
}