Unit 1: Introduction, Univariate Problems

Feb 3: Lecture 1 — Introduction

Contents:

  • Course Organization
  • Optimization problems
  • Classification
  • Constraints
  • Critical Points
  • Conditions for Local Minima
  • Contour Plots
  • Asymptotic Notation
  • Taylor Expansion
  • Convexity
  • Norms
  • Matrix Calculus
  • Positive Definiteness
  • Gaussian Distribution

Resources:

Feb 5: Derivatives and Gradients

Contents:

  • Derivatives
  • Derivatives in Multiple Dimensions
  • Numerical Differentiation
  • Automatic Differentiation

Resources:

  • Slides

  • Chapter 2 and Appendix of [KW]

Feb 10: Definitions, Derivatives and Gradients

We continued with the material from February 5.

Feb 11: Exercises 1