Nonlinear Optimization
Course Description:
Theory and algorithms for unconstrained nonlinear optimization problems, including line search, trust region, conjugate gradient, Newton and quasi-Newton methods.
Textbook:
Numerical Optimization, by J. Nocedal and S. J. Wright. Second Edition, Springer, 2006
Topic Outline and Schedule:
The following is a rough plan. As the course progresses, I may include new topics and/or delete some of the ones listed here.
| Topics | Week(s) |
| Linear and nonlinear optimization models | 1 |
| The graphical method for 2-variable problems | 1 |
| Mathematical background | 2 |
Fundamentals of constrained and unconstrained optimization: - First-order necessary conditions. - Second-order necessary conditions. - Second-order sufficient conditions. | 2, 3 |
Line search methods for one-dimensional nonlinear optimization: - Golden section search method. - Fibonacci method. - Newton's method and secant method. | 4, 5 |
Gradient methods for higher-dimensional nonlinear optimization: - Steepest Descent/Ascent method. | 6, 7 |
Newton's method for higher-dimensional problems: - Newton's method for nonlinear optimization. - Newton's method for nonlinear systems. | 8, 9 |
Conjugate direction methods for higher -dimensional nonlinear optimization: - Basic conjugate direction method. - Conjugate gradient method. | 10-12 |
Quasi-Newton methods for higher-dimensional nonlinear optimization: - The single-rank symmetric algorithm. - The Davidon-Fletcher-Powell algorithm. - The Broyden- Fletcher-Goldfarb-Shanno algorithm. | 13-16
|
