AA 203: Optimal and Learning-Based Control
Spring 2021
Course Description
Optimal control solution techniques for systems with known and unknown dynamics. Dynamic programming, Hamilton-Jacobi reachability, and direct and indirect methods for trajectory optimization. Introduction to model predictive control. Adaptive control, model-based and model-free reinforcement learning, and connections between modern reinforcement learning and fundamental optimal control ideas.
Instructors
Ed Schmerling
|
James Harrison
|
|---|
Course Assistants
Matt Tsao
|
Spencer M. Richards
|
|---|
Meeting Times
Lectures will be online; details of lecture recordings and office hours are available in the syllabus.
Syllabus
The class syllabus can be found here.
Schedule
Subject to change. Lecture notes are available here. We will try to have the lecture notes updated before the class.
| Week | Topic | Lecture Slides |
|---|---|---|
| 1 |
Introduction; Control, stability, and metrics
Introduction to learning; System identification and adaptive control |
Lecture 1;
Code
Lecture 2 |
| 2 |
Unconstrained optimization
Constrained optimization Recitation: Convex optimization and optimization tools Homework: Assignment 1 released |
Lecture 3;
Code
Lecture 4; Code Recitation 1 |
| 3 |
Dynamic programming, discrete LQR
Stochastic DP, value iteration, policy iteration Recitation: Automatic differentiation and Jax Project: Proposal due |
Lecture 5
Lecture 6; Code Recitation 2; Code |
| 4 |
LQG, dual control, introduction to reinforcement learning
Nonlinearity; Tracking LQR, iterative LQR, DDP Recitation: Regression models Homework: Assignment 1 due Homework: Assignment 2 released |
Lecture 7
Lecture 8 Recitation 3 |
| 5 |
Direct methods for optimal control; sequential convex programming
HJB, HJI, reachability analysis Recitation: Training neural networks and PyTorch |
Lecture 9;
Code
Lecture 10; Code Recitation 4 |
| 6 |
Intro to model predictive control
Feasibility and stability of MPC Homework: Assignment 2 due Project: Midterm report due |
Lecture 11
Lecture 12 |
| 7 |
Adaptive and learning MPC
Model-based RL Homework: Assignment 3 released |
Lecture 13
Lecture 14 |
| 8 |
Model-free RL: policy gradient and actor critic
Model-based policy learning |
Lecture 15
Lecture 16 |
| 9 |
Model-based policy learning (cont.)
Calculus of variations Homework: Assignment 3 due Homework: Assignment 4 released |
Lecture 17
Lecture 18 |
| 10 |
Indirect methods for optimal control
Pontryagin's maximum principle, wrap up, recent work, and future directions Homework: Assignment 4 due Project: Video presentation and final report due |
Lecture 19;
Code
Lecture 20 |