AA 203: Optimal and Learning-Based Control
Spring 2023
Course Description
Optimal control solution techniques for systems with known and unknown dynamics. Dynamic programming, Hamilton-Jacobi reachability, and direct and indirect methods for trajectory optimization. Introduction to model predictive control. Adaptive control, model-based and model-free reinforcement learning, and connections between modern reinforcement learning and fundamental optimal control ideas.
Instructors
Spencer M. Richards
|
Daniele Gammelli
|
|---|
Course Assistants
Thomas Lew
|
|---|
Meeting Times
Lectures meet on Mondays and Wednesdays from 9:45am to 11:15am in Skilling Auditorium; lecture recordings will be available at the class canvas site.
Office hours will be scheduled during the first week of classes and then listed here.
Syllabus
The class syllabus can be found here.
Schedule
Subject to change.
| Week | Topic | Lecture Slides |
|---|---|---|
| 1 |
Introduction; control, stability, and metrics
Introduction to learning; system identification and adaptive control Recitation: Automatic differentiation (autodiff) with JAX Monday: HW0 (ungraded) out |
Lecture 1;
Code
Lecture 2; Code Recitation 1; Code |
| 2 |
Nonlinear optimization theory
Calculus of variations Recitation: Convex optimization with CVXPY Monday: HW1 out |
Lecture 3;
Code
Lecture 4 Recitation 2 |
| 3 |
Indirect methods for optimal control
Pontryagin's maximum principle; introduction to dynamic programming Recitation: Regression models Friday: project proposal due |
Lecture 5;
Code
Lecture 6 Recitation 3 |
| 4 |
Discrete LQR, stochastic DP, value iteration, policy iteration
Introduction to reinforcement learning, dual control, LQG Recitation: Training neural networks with JAX Monday: HW1 due, HW2 out |
Lecture 7;
Code
Lecture 8 Recitation 4 |
| 5 |
Nonlinearity; tracking LQR
Iterative LQR, differential dynamic programming (DDP) |
Lecture 9;
Code
Lecture 10; Code |
| 6 |
Direct methods for optimal control; sequential convex programming
HJB, HJI, reachability analysis Monday: HW2 due |
Lecture 11;
Code
Lecture 12; Code |
| 7 |
Intro to model predictive control
Feasibility and stability of MPC Monday: project midterm report due; Tuesday: HW3 out |
Lecture 13
Lecture 14 |
| 8 |
MPC implementation considerations, robust MPC
Adaptive and learning MPC |
Lecture 15;
Code
Lecture 16 |
| 9 |
Model-based RL
Model-free RL: policy gradient and actor critic Tuesday: HW3 due; Wednesday: HW4 out |
Lecture 17
Lecture 18; Code |
| 10 |
No lecture (Memorial Day)
Model-based policy learning, wrap up, recent work, and future directions Wednesday: HW4 due, project video presentation and final report due |
Lecture 19 |
Follow this link to access the course website for the previous edition of Optimal and Learning-Based Control.