Variational quasi-Newton methods for unconstrained optimization

Research output: Contribution to journalArticle

13 Citations (Scopus)

Abstract

In this paper, we propose new members of the Broyden family of quasi-Newton methods. We develop, on the basis of well-known least-change results for the BFGS and DFP updates, a measure for the Broyden family which seeks to take into account the change in both the Hessian approximation and its inverse. The proposal is then to choose the formula which gives the least value of this measure in terms of the two parameters available, and hence to produce an update which is optimal in the sense of the given measure. Several approaches to the problem of minimizing the measure are considered, from which new updates are obtained. In particular, one approach yields a new variational result for the Davidon optimally conditioned method and another yields a reasonable modification to this method. The paper is also concerned with the possibility of estimating, in a certain sense, the size of the eigenvalues of the Hessian approximation on the basis of two available scalars. This allows one to derive further modifications to the above-mentioned methods. Comparisons with the BFGS and Davidson methods are made on a set of standard test problems that show promising results for certain new methods.

Original languageEnglish
Pages (from-to)127-143
Number of pages17
JournalJournal of Optimization Theory and Applications
Volume77
Issue number1
DOIs
Publication statusPublished - Apr 1993

Fingerprint

Quasi-Newton Method
Unconstrained Optimization
Newton-Raphson method
Variational Methods
Update
Approximation
Test Problems
Two Parameters
Choose
Scalar
Eigenvalue

Keywords

  • Broyden family of updates
  • Davidon optimally conditioned update
  • least-change BFGS and DFP updates
  • quasi-Newton methods
  • symmetric rank-one update
  • Unconstrained optimization

ASJC Scopus subject areas

  • Control and Optimization
  • Applied Mathematics
  • Management Science and Operations Research

Cite this

Variational quasi-Newton methods for unconstrained optimization. / Al-Baali, M.

In: Journal of Optimization Theory and Applications, Vol. 77, No. 1, 04.1993, p. 127-143.

Research output: Contribution to journalArticle

@article{c8c413176ef340de9682cf92ab1fb745,
title = "Variational quasi-Newton methods for unconstrained optimization",
abstract = "In this paper, we propose new members of the Broyden family of quasi-Newton methods. We develop, on the basis of well-known least-change results for the BFGS and DFP updates, a measure for the Broyden family which seeks to take into account the change in both the Hessian approximation and its inverse. The proposal is then to choose the formula which gives the least value of this measure in terms of the two parameters available, and hence to produce an update which is optimal in the sense of the given measure. Several approaches to the problem of minimizing the measure are considered, from which new updates are obtained. In particular, one approach yields a new variational result for the Davidon optimally conditioned method and another yields a reasonable modification to this method. The paper is also concerned with the possibility of estimating, in a certain sense, the size of the eigenvalues of the Hessian approximation on the basis of two available scalars. This allows one to derive further modifications to the above-mentioned methods. Comparisons with the BFGS and Davidson methods are made on a set of standard test problems that show promising results for certain new methods.",
keywords = "Broyden family of updates, Davidon optimally conditioned update, least-change BFGS and DFP updates, quasi-Newton methods, symmetric rank-one update, Unconstrained optimization",
author = "M. Al-Baali",
year = "1993",
month = "4",
doi = "10.1007/BF00940782",
language = "English",
volume = "77",
pages = "127--143",
journal = "Journal of Optimization Theory and Applications",
issn = "0022-3239",
publisher = "Springer New York",
number = "1",

}

TY - JOUR

T1 - Variational quasi-Newton methods for unconstrained optimization

AU - Al-Baali, M.

PY - 1993/4

Y1 - 1993/4

N2 - In this paper, we propose new members of the Broyden family of quasi-Newton methods. We develop, on the basis of well-known least-change results for the BFGS and DFP updates, a measure for the Broyden family which seeks to take into account the change in both the Hessian approximation and its inverse. The proposal is then to choose the formula which gives the least value of this measure in terms of the two parameters available, and hence to produce an update which is optimal in the sense of the given measure. Several approaches to the problem of minimizing the measure are considered, from which new updates are obtained. In particular, one approach yields a new variational result for the Davidon optimally conditioned method and another yields a reasonable modification to this method. The paper is also concerned with the possibility of estimating, in a certain sense, the size of the eigenvalues of the Hessian approximation on the basis of two available scalars. This allows one to derive further modifications to the above-mentioned methods. Comparisons with the BFGS and Davidson methods are made on a set of standard test problems that show promising results for certain new methods.

AB - In this paper, we propose new members of the Broyden family of quasi-Newton methods. We develop, on the basis of well-known least-change results for the BFGS and DFP updates, a measure for the Broyden family which seeks to take into account the change in both the Hessian approximation and its inverse. The proposal is then to choose the formula which gives the least value of this measure in terms of the two parameters available, and hence to produce an update which is optimal in the sense of the given measure. Several approaches to the problem of minimizing the measure are considered, from which new updates are obtained. In particular, one approach yields a new variational result for the Davidon optimally conditioned method and another yields a reasonable modification to this method. The paper is also concerned with the possibility of estimating, in a certain sense, the size of the eigenvalues of the Hessian approximation on the basis of two available scalars. This allows one to derive further modifications to the above-mentioned methods. Comparisons with the BFGS and Davidson methods are made on a set of standard test problems that show promising results for certain new methods.

KW - Broyden family of updates

KW - Davidon optimally conditioned update

KW - least-change BFGS and DFP updates

KW - quasi-Newton methods

KW - symmetric rank-one update

KW - Unconstrained optimization

UR - http://www.scopus.com/inward/record.url?scp=0027576637&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=0027576637&partnerID=8YFLogxK

U2 - 10.1007/BF00940782

DO - 10.1007/BF00940782

M3 - Article

AN - SCOPUS:0027576637

VL - 77

SP - 127

EP - 143

JO - Journal of Optimization Theory and Applications

JF - Journal of Optimization Theory and Applications

SN - 0022-3239

IS - 1

ER -