AA 203: Optimal and Learning-Based Control

Spring 2021

Course Description

Optimal control solution techniques for systems with known and unknown dynamics. Dynamic programming, Hamilton-Jacobi reachability, and direct and indirect methods for trajectory optimization. Introduction to model predictive control. Adaptive control, model-based and model-free reinforcement learning, and connections between modern reinforcement learning and fundamental optimal control ideas.

Course Assistants

Matt Tsao Spencer M. Richards

Meeting Times

Lectures will be online; details of lecture recordings and office hours are available in the syllabus.


The class syllabus can be found here.


Subject to change. Lecture notes are available here. We will try to have the lecture notes updated before the class.

Week Topic Lecture Slides
1 Introduction; Control, stability, and metrics
Introduction to learning; System identification and adaptive control
Lecture 1; Code
Lecture 2
2 Unconstrained optimization
Constrained optimization
Recitation: Convex optimization and optimization tools
Homework: Assignment 1 released
Lecture 3; Code
Lecture 4; Code
Recitation 1
3 Dynamic programming, discrete LQR
Stochastic DP, value iteration, policy iteration
Recitation: Automatic differentiation and Jax
Project: Proposal due
Lecture 5
Lecture 6; Code
Recitation 2; Code
4 LQG, dual control, introduction to reinforcement learning
Nonlinearity; Tracking LQR, iterative LQR, DDP
Recitation: Regression models
Homework: Assignment 1 due
Homework: Assignment 2 released
Lecture 7
Lecture 8
Recitation 3
5 Direct methods for optimal control; sequential convex programming
HJB, HJI, reachability analysis
Recitation: Training neural networks and PyTorch
Lecture 9; Code
Lecture 10; Code
Recitation 4
6 Intro to model predictive control
Feasibility and stability of MPC
Homework: Assignment 2 due
Project: Midterm report due
Lecture 11
Lecture 12
7 Adaptive and learning MPC
Model-based RL
Homework: Assignment 3 released
Lecture 13
Lecture 14
8 Model-free RL: policy gradient and actor critic
Model-based policy learning
Lecture 15
Lecture 16
9 Model-based policy learning (cont.)
Calculus of variations
Homework: Assignment 3 due
Homework: Assignment 4 released
Lecture 17
Lecture 18
10 Indirect methods for optimal control
Pontryagin's maximum principle, wrap up, recent work, and future directions
Homework: Assignment 4 due
Project: Video presentation and final report due
Lecture 19; Code
Lecture 20