AA 203: Optimal and Learning-Based Control

Spring 2023

Course Description

Optimal control solution techniques for systems with known and unknown dynamics. Dynamic programming, Hamilton-Jacobi reachability, and direct and indirect methods for trajectory optimization. Introduction to model predictive control. Adaptive control, model-based and model-free reinforcement learning, and connections between modern reinforcement learning and fundamental optimal control ideas.

Course Assistants

Thomas Lew

Meeting Times

Lectures meet on Mondays and Wednesdays from 9:45am to 11:15am in Skilling Auditorium; lecture recordings will be available at the class canvas site.

Office hours will be scheduled during the first week of classes and then listed here.

Syllabus

The class syllabus can be found here.

Schedule

Subject to change.

Week Topic Lecture Slides
1 Introduction; control, stability, and metrics
Introduction to learning; system identification and adaptive control
Recitation: Automatic differentiation (autodiff) with JAX
Monday: HW0 (ungraded) out
Lecture 1; Code
Lecture 2; Code
Recitation 1; Code
2 Nonlinear optimization theory
Calculus of variations
Recitation: Convex optimization with CVXPY
Monday: HW1 out
Lecture 3; Code
Lecture 4
Recitation 2
3 Indirect methods for optimal control
Pontryagin's maximum principle; introduction to dynamic programming
Recitation: Regression models
Friday: project proposal due
Lecture 5; Code
Lecture 6
Recitation 3
4 Discrete LQR, stochastic DP, value iteration, policy iteration
Introduction to reinforcement learning, dual control, LQG
Recitation: Training neural networks with JAX
Monday: HW1 due, HW2 out
Lecture 7; Code
Lecture 8
Recitation 4
5 Nonlinearity; tracking LQR
Iterative LQR, differential dynamic programming (DDP)
Lecture 9; Code
Lecture 10; Code
6 Direct methods for optimal control; sequential convex programming
HJB, HJI, reachability analysis
Monday: HW2 due
Lecture 11; Code
Lecture 12; Code
7 Intro to model predictive control
Feasibility and stability of MPC
Monday: project midterm report due; Tuesday: HW3 out
Lecture 13
Lecture 14
8 MPC implementation considerations, robust MPC
Adaptive and learning MPC
Lecture 15; Code
Lecture 16
9 Model-based RL
Model-free RL: policy gradient and actor critic
Tuesday: HW3 due; Wednesday: HW4 out
Lecture 17
Lecture 18; Code
10 No lecture (Memorial Day)
Model-based policy learning, wrap up, recent work, and future directions
Wednesday: HW4 due, project video presentation and final report due

Lecture 19

Follow this link to access the course website for the previous edition of Optimal and Learning-Based Control.