AA 203: Optimal and Learning-based Control

Spring 2019

Course Description

Optimal control solution techniques for systems with known and unknown dynamics. Dynamic programming, Hamilton-Jacobi reachability, and direct and indirect methods for trajectory optimization. Introduction to model predictive control. Model-based reinforcement learning, and connections between modern reinforcement learning in continuous spaces and fundamental optimal control ideas.

Meeting Times

Lectures meet on Mondays and Wednesdays from 1:30 PM to 2:50 PM in Herrin T175.

Professor Pavone's office hours are Mondays 3-5pm in Durand 261.

CA office hours are Tuesdays 1:30pm-3pm (Durand 353) and Fridays 12-1:30pm (Durand 270)

Syllabus

The class syllabus can be found here.

Schedule

Subject to change. Lecture notes are available here. We will try to have the lecture notes updated before the class.

Week Topic Lecture Slides
1 Course overview, unconstrained nonlinear optimization
Constrained nonlinear optimization
Recitation: Linear dynamical systems
HW1 out
Lecture 1
Lecture 2
2 Dynamic programming, discrete LQR
Iterative LQR, DDP, and LQG
HW2 out, HW1 due
Lecture 3
Lecture 4
3 Hamilton-Jacobi-Bellman and Hamilton-Jacobi-Isaacs equations
Reachability analysis
HW3 out, HW2 due
Lecture 5
Lecture 6
4 Calculus of variations
Indirect methods for optimal control
HW4 out, HW3 due
Lecture 7
Lecture 8
5 Pontryagin's maximum principle
Numerical aspects of indirect optimal control
Midterm (May 2, evening)
HW5 out, HW4 due
Lecture 9
Lecture 10
6 Direct methods for optimal control
Direct collocation and sequential quadratic programming
Recitation: Linear, quadratic, convex, and mixed-integer linear programming
HW6 out, HW5 due
Lecture 11
Lecture 12
Recitation
7 Introduction to model predictive control
Feasibility and stability of MPC
HW7 out, HW6 due
Lecture 13
Lecture 14
8 Adaptive optimal control, dual control, adaptive LQR
Model-based reinforcement learning: linear methods
HW8 out, HW7 due
Lecture 15
Lecture 16
9 Nonlinear regression fundamentals
Lecture 17
10 Model-based reinforcement learning: nonlinear methods
Intro to model-free RL, connections to model-based RL
HW8 due
Lecture 18
Lecture 19