Abstract

This paper discusses theoretically the evolution of a conjugate direction algorithm for minimizing an arbitrary nonlinear, non quadratic function using Broyden-Fletcher-Goldfarb-Shano (BFGS) update in quasi-Newton Method. The updating rule is initialized by a Moore Penrose’s generalized inverse. Specifically, an approximation to the inverse Hessian is constructed and the updating rule for this approximation is imbedded in the BFGS update. Numerical experiments show that, using the proposed line search algorithm and the modified quasi-Newton algorithm for unconstrained problems are very competitive. This paper produces a new analysis that demonstrates that the BFGS method with a line search is  step q-superlinear convergent with assumption of linearly independent iterates. The analysis assumes that the inverse Hessian approximations are positive definite and bounded asymptotically, which from computational experience, are of reasonable assumptions.

Key words: Quasi-Newton method, Moore-Penrose generalized inverse, Broyden-Fletcher-Goldfarb-Shano (BFGS) update, superlinear convergence, conjugate directions, orthogonalization of matrices.