Optimization (ACM 40990 and ACM 41030)
Current modules (Spring 2026)
Description: For Optimization in Machine Learning (ACM 40990, Spring 2026), refer to this page for the first seven weeks. For Optimization Algorithms (ACM 41030, Spring 2026), refer to this page for all weeks.
Course Documents:
- Complete set of typed notes, v2: March 2026
- Side note Section 1.3 (Convexity of Polyhedra)
- Introduction to ACM 40990 (January 2026)
- Introduction to ACM 41030 (January 2026)
- Handwritten Notes, Weeks 1-7
Lecture Notes (Weeks 8-12, ACM 41030 only):
Examinable results (Weeks 8-12, ACM 41030 only):
- Week 8: The projection operator, Section 12.4
- Week 8: First-order optimality conditions in case of a single inequality constraint, Section 13.2
- Week 9: Nothing!
- Week 10: Showing that if the LICQs are satisfied, then the Lagrange Multipliers are unique - Section 15.1, Theorem 15.1
- Week 10: Showing that the tangant cone is inside the set of LFDDs; showing that the tangant cone and the LFDDs are the same when the LICQs are satisfied - Section 15.2, Theorem 15.3
Exercises #1: Line-search methods.
Code repository:
Exercises #2: Newton iteration and the Strong Wolfe Conditions.
Code repository:
Exercises #3: BFGS revisited and the Trust-Region Method.
Exercises #4: Global Optimization and Simulated Annealing
Code repository:
Exercises #5: Constraints (ACM 41030 only)
Exercises #6: More Constraints (ACM 41030 only)
Foundations of Data Science: Materials for Short Course on 8/12/2023:
Special Lectures on Optimization KIUT, 29th April 2025