| Description |
This course will explore theory and algorithms for nonlinear optimization. We will focus on problems that arise in machine learning and computational statistics, paying close attention to concerns about complexity, scaling, and implementation in these domains. Whenever possible, methods will be linked to particular application examples in data analysis. Topics will include
Descent algorithms and line search methods
Acceleration, momentum, and conjugate gradients
Oracle complexity of optimization
Newton and Quasi-Newton methods
Coordinate descent
Stochastic and incremental gradient methods
Derivative-free optimization
Subgradient calculus and algorithms
The proximal point method
Projected gradient descent
Lagrangian decomposition
The Alternating Direction Method of Multipliers |
