Optimization (ACM 40990 and ACM 41030)
Current modules (Spring 2026)
Description: For Optimization in Machine Learning (ACM 40990, Spring 2026), refer to this page for the first seven weeks. For Optimization Algorithms (ACM 41030, Spring 2026), refer to this page for all weeks.
Course Documents:
- Complete set of typed notes, v1: January 2026
- Side note Section 1.3 (Convexity of Polyhedra)
- Introduction to ACM 40990 (January 2026)
- Introduction to ACM 41030 (January 2026)
Examinable results - Part 1:
- Week 1: Theorems / Results in Section 1.3 (Convex Sets)
- Week 1: Results in Section 1.4 (Convex Functions)
- Week 2: Theorem 2.8 (Convex functions and their minimizer)
- Week 2: Quadratic Model Problem, Section 2.3
- Week 2: BFGS formulae, pages 29-30
- Week 3: Theorem 4.2
- Week 3: Barzilai-Borwein formula, Section 4.3
Lecture Notes:
- Week 1, all lectures
- Week 2, Lecture 1: Lecture notes and video
- Week 2, Lectures 2-3
- Week 3, Lecture 1: Lecture notes and video
Exercises #1: Line-search methods.
Code repository:
Exercises #2: Newton iteration and the Strong Wolfe Conditions.
Code repository:
Exercises #3: BFGS revisited and the Trust-Region Method.
Exercises #4: Global Optimization and Simulated Annealing
Code repository:
Exercises #5: Constraints (ACM 41030 only)
Exercises #6: More Constraints (ACM 41030 only)
Foundations of Data Science: Materials for Short Course on 8/12/2023:
Special Lectures on Optimization KIUT, 29th April 2025