Optimization (ACM 40990 and ACM 41030)
Description: For Optimization in Machine Learning (ACM 40990, Spring 2025), refer to the Brightspace page. For Optimization Algorithms (ACM 41030, Spring 2025), refer to this page.
Lecture Notes:
- Complete set of typed notes, v1: January 2025
- Week 2, Lecture 1: Lecture notes and video
Course Documents:
Examinable results - Part 1:
- Week 1: Theorems / Results in Section 1.3 (Convex Sets)
- Week 1: Results in Section 1.4 (Convex Functions)
- Week 2: Theorem 2.8 (Convex functions and their minimizer)
- Week 2: Quadratic Model Problem, Section 2.3
- Week 2: BFGS formulae, pages 29-31
Exercises #1: Line-search methods.
Code repository:
Exercises #2: Newton iteration and the Strong Wolfe Conditions.
Code repository:
Exercises #3: BFGS revisited and the Trust-Region Method.
Exercises #4: Global Optimization and Simulated Annealing
Code repository:
Exercises #5: Constraints (ACM 41030 only)
Exercises #6: More Constraints (ACM 41030 only)