The University of Jordan

Nonlinear Optimization (0301772)

Nonlinear Optimization

Course Description:
Theory and algorithms for unconstrained nonlinear optimization problems, including line search, trust region, conjugate gradient, Newton and quasi-Newton methods.​


Numerical Optimization, by J. Nocedal and S. J. Wright. Second Edition, Springer, 2006

Topic Outline and Schedule:

The following is a rough plan. As the course progresses, I may include new topics and/or delete some of the ones listed here.

Linear and nonlinear optimization models1
The graphical method for 2-variable problems1
Mathematical background2

Fundamentals of constrained and unconstrained optimization:

- First-order necessary conditions.

- Second-order necessary conditions.

- Second-order sufficient conditions.

2, 3

Line search methods for one-dimensional nonlinear optimization:

- Golden section search method.

- Fibonacci method.

- Newton's method and secant method.

4, 5

Gradient methods for higher-dimensional nonlinear optimization:

- Steepest Descent/Ascent method.

6, 7

Newton's method for higher-dimensional problems:

- Newton's method for nonlinear optimization.

- Newton's method for nonlinear systems.

8, 9

Conjugate direction methods for higher -dimensional nonlinear optimization:

- Basic conjugate direction method.

- Conjugate gradient method.


Quasi-Newton methods for higher-dimensional nonlinear optimization:

- The single-rank symmetric algorithm.

- The Davidon-Fletcher-Powell algorithm.

- The Broyden- Fletcher-Goldfarb-Shanno algorithm.


Nonlinear Optimization (0301772)