Unit 4: Optimization for Machine Learning
March 17: Lecture 10 — Machine Learning Applications
- Intro to ML problems
- Formal Optimization problems in ML
Resources:
- Slides
- Léon Bottou, Frank E. Curtis, Jorge Nocedal. Optimization Methods for Large-Scale Machine Learning (2018)
March 20: Lecture 11 — Machine Learning Applications
- Stochastic vs Batch Optimization Methods
- Stochastic Gradient Analysis
- Iterate Averaging Methods
Resources:
- Slides
- Léon Bottou, Frank E. Curtis, Jorge Nocedal. Optimization Methods for Large-Scale Machine Learning (2018)
- An overview of gradient descent optimization algorithms
March 24: Lecture 12 — Machine Learning Applications
Dynamic Sample Size Methods
- Iterated Averaging Methods
- Gradient Aggregation
Resources:
- Slides
- Léon Bottou, Frank E. Curtis, Jorge Nocedal. Optimization Methods for Large-Scale Machine Learning (2018) [Sec. 5]