Optimization (ACM 40990 and ACM 41030)
Description: For Optimization in Machine Learning (ACM 40990, Spring 2025), refer to the Brightspace page. For Optimization Algorithms (ACM 41030, Spring 2025), refer to this page.
Lecture Notes:
- Complete set of typed notes, v1: January 2025
- Complete set of typed notes, v2: February 2025 (Updated Sections 8.1-8.3)
- Week 2, Lecture 1: Lecture notes and video
- Week 3, Lecture 1: Lecture notes and video
- Week 4, Lecture 1: Lecture notes and video
- Week 5, Lecture 1: Lecture notes and video
- Week 6, Lecture 1: Lecture notes and video
- Week 7, Lecture 1: Lecture notes and video
Course Documents:
- Introduction to ACM 41030 (January 2025)
- Side note Section 1.3 (Convexity of Polyhedra)
- Week 6, Bonus Assignment
- Exam 1 Guidelines
Examinable results - Part 1:
- Week 1: Theorems / Results in Section 1.3 (Convex Sets)
- Week 1: Results in Section 1.4 (Convex Functions)
- Week 2: Theorem 2.8 (Convex functions and their minimizer)
- Week 2: Quadratic Model Problem, Section 2.3
- Week 2: BFGS formulae, pages 29-31, but not the Sherman-Morrison-Woodbury formula
- Week 3: Theorem 4.2
- Week 3: Barzilai-Borwein formula, Section 4.3
- Week 4: Convergence criterion for Quasi-Newton Methods, Theorem 6.2, Section 6.2
- Week 5: Nothing
- Week 6: Cauchy-point calculation, Section 7.6
- Week 7: Convergence Proof, Simulated Annealing, Section 18.4
- End of list!
Exercises #1: Line-search methods.
Code repository:
Exercises #2: Newton iteration and the Strong Wolfe Conditions.
- Questions
- Model answers (Correction to Model Answers, Question 2 on 20/02)
- Data file for the 10-dimensional OP
Code repository:
Exercises #3: BFGS revisited and the Trust-Region Method.
Exercises #4: Global Optimization and Simulated Annealing
Code repository:
Exercises #5: Constraints (ACM 41030 only)
Exercises #6: More Constraints (ACM 41030 only)