Matt Foutter

Contacts:

Email: mfoutter at stanford dot edu

Matt Foutter

Matt is a PhD student in the Department of Mechanical Engineering. Matt’s research interests lie at the intersection of machine learning and robotics with the goal of enabling an autonomous robot to safely navigate in an unfamiliar environment. Specifically, he is interested in developing methodologies that monitor a robot’s operation at deployment - preemptively catching failure modes and enacting safety preserving actions.

Prior to joining Stanford, Matt graduated summa cum laude from the University of Michigan, Ann Arbor, with a B.S.E in Aerospace Engineering and minor in Computer Science. There, he conducted research in classical control techniques with highly flexible wings under Prof. Carlos Cesnik in the Active Aeroelasticity and Structures Research Laboratory. Also, as an undergraduate, he raced a solar car 1,800 miles across Australia and interned at SpaceX and MIT Lincoln Lab.

Outside of the lab, Matt enjoys his free time by reading and playing multiple sports including basketball and table tennis.

ASL Publications

R. Dyro, M. Foutter, R. Li, L. Di Lillo, E. Schmerling, X. Zhou, and M. Pavone, “Realistic Extreme Behavior Generation for Improved AV Testing,” in Proc. IEEE Conf. on Robotics and Automation, 2025. (Submitted)
[BibTeX] [Abstract]

Abstract: This work introduces a framework to diagnose the strengths and shortcomings of Autonomous Vehicle (AV) collision avoidance technology with synthetic yet realistic potential collision scenarios adapted from real-world, collision-free data. Our framework generates counterfactual collisions with diverse crash properties, e.g., crash angle and velocity, between an adversary and a target vehicle by adding perturbations to the adversary’s predicted trajectory from a learned AV behavior model. Our main contribution is to ground these adversarial perturbations in realistic behavior as defined through the lens of data-alignment in the behavior model’s parameter space. Then, we cluster these synthetic counterfactuals to identify plausible and representative collision scenarios to form the basis of a test suite for downstream AV system evaluation. We demonstrate our framework using two state-of-the-art behavior prediction models as sources of realistic adversarial perturbations, and show that our scenario clustering evokes interpretable failure modes from a baseline AV policy under evaluation.
```
@inproceedings{DyroFoutterEtAl2024,
  author = {Dyro, R. and Foutter, M. and Li, R. and Di Lillo, L. and Schmerling, E. and Zhou, X. and Pavone, M.},
  title = {Realistic Extreme Behavior Generation for Improved {AV} Testing},
  booktitle = {{Proc. IEEE Conf. on Robotics and Automation}},
  year = {2025},
  note = {Submitted},
  keywords = {sub},
  owner = {jthluke},
  timestamp = {2024-10-30},
  url = {/wp-content/papercite-data/pdf/Dyro.Foutter.Li.ea.ICRA2025.pdf}
}
```
R. Sinha, A. Elhafsi, C. Agia, M. Foutter, E. Schmerling, and M. Pavone, “Real-Time Anomaly Detection and Planning with Large Language Models,” in Robotics: Science and Systems, Delft, Netherlands, 2024.
[BibTeX] [Abstract]

Abstract: Foundation models, e.g., large language models, trained on internet-scale data possess zero-shot generalization capabilities that make them a promising technology for anomaly detection for robotic systems. Fully realizing this promise, however, poses two challenges: (i) mitigating the considerable computational expense of these models such that they may be applied online, and (ii) incorporating their judgement regarding potential anomalies into a safe control framework. In this work we present a two-stage reasoning framework: a fast binary anomaly classifier based on analyzing observations in an LLM embedding space, which may trigger a slower fallback selection stage that utilizes the reasoning capabilities of generative LLMs. These stages correspond to branch points in a model predictive control strategy that maintains the joint feasibility of continuing along various fallback plans as soon as an anomaly is detected (while the selector decides), thus ensuring safety. We demonstrate that, even when instantiated with relatively small language models, our fast anomaly classifier outperforms autoregressive reasoning with state-of-the-art GPT models. This enables our runtime monitor to improve the trustworthiness of dynamic robotic systems under resource and time constraints.
```
@inproceedings{SinhaElhafsiEtAl2024,
  author = {Sinha, R. and Elhafsi, A. and Agia, C. and Foutter, M. and Schmerling, E. and Pavone, M.},
  title = {Real-Time Anomaly Detection and Planning with Large Language Models},
  booktitle = {{Robotics: Science and Systems}},
  address = {Delft, Netherlands},
  month = jul,
  year = {2024},
  owner = {amine},
  url = {https://arxiv.org/abs/2407.08735},
  timestamp = {2024-09-19}
}
```

M. Foutter, P. Bhoj, R. Sinha, A. Elhafsi, S. Banerjee, C. Agia, J. Kruger, T. Guffanti, D. Gammelli, S. D’Amico, and M. Pavone, “Adapting a Foundation Model for Space-based Tasks,” in Robotics: Science and Systems - Workshop on Semantics for Robotics: From Environment Understanding and Reasoning to Safe Interaction, 2024.

[BibTeX] [Abstract]

Abstract:

@inproceedings{FoutterBohjEtAl2024,
  author = {Foutter, M. and Bhoj, P. and Sinha, R. and Elhafsi, A. and Banerjee, S. and Agia, C. and Kruger, J. and Guffanti, T. and Gammelli, D. and D'Amico, S. and Pavone, M.},
  title = {Adapting a Foundation Model for Space-based Tasks},
  booktitle = {{Robotics: Science and Systems - Workshop on Semantics for Robotics: From Environment Understanding and Reasoning to Safe Interaction}},
  year = {2024},
  asl_abstract = {Foundation models, e.g., large language models, possess attributes of intelligence which offer promise to endow a robot with the contextual understanding necessary to navigate complex, unstructured tasks in the wild. In the future of space robotics, we see three core challenges which motivate the use of a foundation model adapted to space-based applications: 1) Scalability of ground-in-the-loop operations; 2) Generalizing prior knowledge to novel environments; and 3) Multi-modality in tasks and sensor data. Therefore, as a first-step towards building a foundation model for space-based applications, we automatically label the AI4Mars dataset to curate a language annotated dataset of visual-question-answer tuples. We fine-tune a pretrained LLaVA checkpoint on this dataset to endow a vision-language model with the ability to perform spatial reasoning and navigation on Mars' surface. In this work, we demonstrate that 1) existing vision-language models are deficient visual reasoners in space-based applications, and 2) fine-tuning a vision-language model on extraterrestrial data significantly improves the quality of responses even with a limited training dataset of only a few thousand samples.},
  asl_address = {Delft, Netherlands},
  asl_url = {https://arxiv.org/abs/2408.05924},
  url = {https://arxiv.org/abs/2408.05924},
  owner = {foutter},
  timestamp = {2024-08-12}
}

M. Foutter, R. Sinha, S. Banerjee, and M. Pavone, “Self-Supervised Model Generalization using Out-of-Distribution Detection,” in Conf. on Robot Learning - Workshop on Out-of-Distribution Generalization in Robotics, Atlanta, Georgia, 2023.
[BibTeX] [Abstract]

Abstract: Autonomous agents increasingly rely on learned components to streamline safe and reliable decision making. However, data dissimilar to that seen in training, deemed to be Out-of-Distribution (OOD), creates undefined behavior in the output of our learned-components, which can have detrimental consequences in a safety critical setting such as autonomous satellite rendezvous. In the wild, we typically are exposed to a mix of in-and-out of distribution data where OOD inputs correspond to uncommon and unfamiliar data when a nominally competent system encounters a new situation. In this paper, we propose an architecture that detects the presence of OOD inputs in an online stream of data. The architecture then uses these OOD inputs to recognize domain invariant features between the original training and OOD domain to improve model inference. We demonstrate that our algorithm more than doubles model accuracy on the OOD domain with sparse, unlabeled OOD examples compared to a naive model without such data on shifted MNIST domains. Importantly, we also demonstrate our algorithm maintains strong accuracy on the original training domain, generalizing the model to a mix of in-and-out of distribution examples seen at deployment. Code for our experiment is available at: https://github.com/StanfordASL/CoRL_OODWorkshop_DANN-DL
```
@inproceedings{FoutterSinhaEtAl2023,
  author = {Foutter, M. and Sinha, R. and Banerjee, S. and Pavone, M.},
  title = {Self-Supervised Model Generalization using Out-of-Distribution Detection},
  booktitle = {{Conf. on Robot Learning - Workshop on Out-of-Distribution Generalization in Robotics}},
  year = {2023},
  address = {Atlanta, Georgia},
  month = nov,
  owner = {jthluke},
  timestamp = {2024-09-20},
  url = {https://openreview.net/forum?id=z5XS3BY13J}
}
```