Quasi-newton based preconditioning and damped quasi-newton schemes for nonlinear conjugate gradient methods

Mehiddin Al-Baali, Andrea Caliciotti, Giovanni Fasano, Massimo Roma

Research output: Chapter in Book/Report/Conference proceedingConference contribution

1 Citation (Scopus)

Abstract

In this paper, we deal with matrix-free preconditioners for nonlinear conjugate gradient (NCG) methods. In particular, we review proposals based on quasi-Newton updates, and either satisfying the secant equation or a secant-like equation at some of the previous iterates. Conditions are given proving that, in some sense, the proposed preconditioners also approximate the inverse of the Hessian matrix. In particular, the structure of the preconditioners depends both on low-rank updates along with some specific parameters. The low-rank updates are obtained as by-product of NCG iterations. Moreover, we consider the possibility to embed damped techniques within a class of preconditioners based on quasi-Newton updates. Damped methods have proved to be effective to enhance the performance of quasi-Newton updates, in those cases where the Wolfe linesearch conditions are hardly fulfilled. The purpose is to extend the idea behind damped methods also to improve NCG schemes, following a novel line of research in the literature. The results, which summarize an extended numerical experience using large-scale CUTEst problems, is reported, showing that these approaches can considerably improve the performance of NCG methods.

Original languageEnglish
Title of host publicationNumerical Analysis and Optimization - NAO-IV, 2017
EditorsLucio Grandinetti, Mehiddin Al-Baali, Anton Purnama
PublisherSpringer New York LLC
Pages1-21
Number of pages21
Volume235
ISBN (Print)9783319900254
DOIs
Publication statusPublished - Jan 1 2018
Event4th International Conference on Numerical Analysis and Optimization, NAO-IV 2017 - Muscat, Oman
Duration: Jan 2 2017Jan 5 2017

Other

Other4th International Conference on Numerical Analysis and Optimization, NAO-IV 2017
CountryOman
CityMuscat
Period1/2/171/5/17

Fingerprint

Quasi-Newton
Conjugate Gradient Method
Preconditioning
Damped
Preconditioner
Update
Conjugate Gradient
Chord or secant line
Hessian matrix
Line Search
Large-scale Problems
Iterate
Iteration
Line

Keywords

  • Conjugate gradient method
  • Damped techniques
  • Large scale unconstrained optimization
  • Nonlinear conjugate gradient methods
  • Preconditioning
  • Quasi-Newton methods

ASJC Scopus subject areas

  • Mathematics(all)

Cite this

Al-Baali, M., Caliciotti, A., Fasano, G., & Roma, M. (2018). Quasi-newton based preconditioning and damped quasi-newton schemes for nonlinear conjugate gradient methods. In L. Grandinetti, M. Al-Baali, & A. Purnama (Eds.), Numerical Analysis and Optimization - NAO-IV, 2017 (Vol. 235, pp. 1-21). Springer New York LLC. https://doi.org/10.1007/978-3-319-90026-1_1

Quasi-newton based preconditioning and damped quasi-newton schemes for nonlinear conjugate gradient methods. / Al-Baali, Mehiddin; Caliciotti, Andrea; Fasano, Giovanni; Roma, Massimo.

Numerical Analysis and Optimization - NAO-IV, 2017. ed. / Lucio Grandinetti; Mehiddin Al-Baali; Anton Purnama. Vol. 235 Springer New York LLC, 2018. p. 1-21.

Research output: Chapter in Book/Report/Conference proceedingConference contribution

Al-Baali, M, Caliciotti, A, Fasano, G & Roma, M 2018, Quasi-newton based preconditioning and damped quasi-newton schemes for nonlinear conjugate gradient methods. in L Grandinetti, M Al-Baali & A Purnama (eds), Numerical Analysis and Optimization - NAO-IV, 2017. vol. 235, Springer New York LLC, pp. 1-21, 4th International Conference on Numerical Analysis and Optimization, NAO-IV 2017, Muscat, Oman, 1/2/17. https://doi.org/10.1007/978-3-319-90026-1_1
Al-Baali M, Caliciotti A, Fasano G, Roma M. Quasi-newton based preconditioning and damped quasi-newton schemes for nonlinear conjugate gradient methods. In Grandinetti L, Al-Baali M, Purnama A, editors, Numerical Analysis and Optimization - NAO-IV, 2017. Vol. 235. Springer New York LLC. 2018. p. 1-21 https://doi.org/10.1007/978-3-319-90026-1_1
Al-Baali, Mehiddin ; Caliciotti, Andrea ; Fasano, Giovanni ; Roma, Massimo. / Quasi-newton based preconditioning and damped quasi-newton schemes for nonlinear conjugate gradient methods. Numerical Analysis and Optimization - NAO-IV, 2017. editor / Lucio Grandinetti ; Mehiddin Al-Baali ; Anton Purnama. Vol. 235 Springer New York LLC, 2018. pp. 1-21
@inproceedings{b696791ffd26468da972a9dd14acaec0,
title = "Quasi-newton based preconditioning and damped quasi-newton schemes for nonlinear conjugate gradient methods",
abstract = "In this paper, we deal with matrix-free preconditioners for nonlinear conjugate gradient (NCG) methods. In particular, we review proposals based on quasi-Newton updates, and either satisfying the secant equation or a secant-like equation at some of the previous iterates. Conditions are given proving that, in some sense, the proposed preconditioners also approximate the inverse of the Hessian matrix. In particular, the structure of the preconditioners depends both on low-rank updates along with some specific parameters. The low-rank updates are obtained as by-product of NCG iterations. Moreover, we consider the possibility to embed damped techniques within a class of preconditioners based on quasi-Newton updates. Damped methods have proved to be effective to enhance the performance of quasi-Newton updates, in those cases where the Wolfe linesearch conditions are hardly fulfilled. The purpose is to extend the idea behind damped methods also to improve NCG schemes, following a novel line of research in the literature. The results, which summarize an extended numerical experience using large-scale CUTEst problems, is reported, showing that these approaches can considerably improve the performance of NCG methods.",
keywords = "Conjugate gradient method, Damped techniques, Large scale unconstrained optimization, Nonlinear conjugate gradient methods, Preconditioning, Quasi-Newton methods",
author = "Mehiddin Al-Baali and Andrea Caliciotti and Giovanni Fasano and Massimo Roma",
year = "2018",
month = "1",
day = "1",
doi = "10.1007/978-3-319-90026-1_1",
language = "English",
isbn = "9783319900254",
volume = "235",
pages = "1--21",
editor = "Lucio Grandinetti and Mehiddin Al-Baali and Anton Purnama",
booktitle = "Numerical Analysis and Optimization - NAO-IV, 2017",
publisher = "Springer New York LLC",

}

TY - GEN

T1 - Quasi-newton based preconditioning and damped quasi-newton schemes for nonlinear conjugate gradient methods

AU - Al-Baali, Mehiddin

AU - Caliciotti, Andrea

AU - Fasano, Giovanni

AU - Roma, Massimo

PY - 2018/1/1

Y1 - 2018/1/1

N2 - In this paper, we deal with matrix-free preconditioners for nonlinear conjugate gradient (NCG) methods. In particular, we review proposals based on quasi-Newton updates, and either satisfying the secant equation or a secant-like equation at some of the previous iterates. Conditions are given proving that, in some sense, the proposed preconditioners also approximate the inverse of the Hessian matrix. In particular, the structure of the preconditioners depends both on low-rank updates along with some specific parameters. The low-rank updates are obtained as by-product of NCG iterations. Moreover, we consider the possibility to embed damped techniques within a class of preconditioners based on quasi-Newton updates. Damped methods have proved to be effective to enhance the performance of quasi-Newton updates, in those cases where the Wolfe linesearch conditions are hardly fulfilled. The purpose is to extend the idea behind damped methods also to improve NCG schemes, following a novel line of research in the literature. The results, which summarize an extended numerical experience using large-scale CUTEst problems, is reported, showing that these approaches can considerably improve the performance of NCG methods.

AB - In this paper, we deal with matrix-free preconditioners for nonlinear conjugate gradient (NCG) methods. In particular, we review proposals based on quasi-Newton updates, and either satisfying the secant equation or a secant-like equation at some of the previous iterates. Conditions are given proving that, in some sense, the proposed preconditioners also approximate the inverse of the Hessian matrix. In particular, the structure of the preconditioners depends both on low-rank updates along with some specific parameters. The low-rank updates are obtained as by-product of NCG iterations. Moreover, we consider the possibility to embed damped techniques within a class of preconditioners based on quasi-Newton updates. Damped methods have proved to be effective to enhance the performance of quasi-Newton updates, in those cases where the Wolfe linesearch conditions are hardly fulfilled. The purpose is to extend the idea behind damped methods also to improve NCG schemes, following a novel line of research in the literature. The results, which summarize an extended numerical experience using large-scale CUTEst problems, is reported, showing that these approaches can considerably improve the performance of NCG methods.

KW - Conjugate gradient method

KW - Damped techniques

KW - Large scale unconstrained optimization

KW - Nonlinear conjugate gradient methods

KW - Preconditioning

KW - Quasi-Newton methods

UR - http://www.scopus.com/inward/record.url?scp=85048243750&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=85048243750&partnerID=8YFLogxK

U2 - 10.1007/978-3-319-90026-1_1

DO - 10.1007/978-3-319-90026-1_1

M3 - Conference contribution

SN - 9783319900254

VL - 235

SP - 1

EP - 21

BT - Numerical Analysis and Optimization - NAO-IV, 2017

A2 - Grandinetti, Lucio

A2 - Al-Baali, Mehiddin

A2 - Purnama, Anton

PB - Springer New York LLC

ER -