Jonas Frey
Jonas is a Postdoctoral Researcher at the Department of Aeronautics and Astronautics at Stanford University and Berkeley Artificial Intelligence Research (BAIR) Lab at UC Berkeley. His research focuses on learning-based perception and navigation, with the goal of advancing robotics and autonomous systems. He develops algorithms that enable robots to efficiently learn, understand, and interact with the real world-leveraging reinforcement learning, foundation models, and investigaes scalable representation learning in sim and real. His broader mission is to enable robots to aid in search and rescue, firefighting, and disaster response.
Before joining Stanford, Jonas earned his Ph.D. in Robotics at the Legged Robotics Lab, ETH Zurich, and the Max Planck Institute for Intelligent Systems. During his time at ETH Zurich, he was main lead at ETH Zurich for the Natural Intelligence (NI) European research project, established research collaboration with the University of Oxford, and NASA’s Jet Propulsion Laboratory (JPL), which he additionaly joing for an internship focusing on off-road autonomy. He also secured a research grant for an Open Data Initiative and co-led the development of ETH Zurich’s GrandTour dataset project.
Outside the lab, Jonas is passionate about rowing, running, road biking and robotics.
Awards:
- RSS - Best System Paper Finalist (2025)
- RSS - Reliable Robotics Workshop - Best Paper (2025)
- ICRA - Best Paper Finalist - Cognitive Robotics (2024)
- CoRL - Best Paper Finalist (2023)
- Max Planck ETH Center for Learning Systems Fellowship (2022-2025)
- NeurIPS - 4th Robot Learning Workshop Self-Supervised and Lifelong Learning - Best Paper Runner-Up (2021)
- M.Sc. ETH Zurich - summa cum laude (2021)
- Scholarship of the German people (2019-2021)
- Karolina Ruedi Foundation (2019)
- Continuous Learning in International Collaborative Studie (2018)
ASL Publications
-
M. Ganai, K. Luo, J. Frey, C. Barrett, and M. Pavone, “Self-Supervised Bootstrapping of Action-Predictive Embodied Reasoning,” in Robotics: Science and Systems, Sydney, Australia, 2026.
Abstract: Embodied Chain-of-Thought (CoT) reasoning has significantly enhanced Vision-Language-Action (VLA) models, yet current methods rely on rigid templates to specify reasoning primitives (e.g., objects in the scene, high-level plans, structural affordances). These templates can force policies to process irrelevant information that distracts from critical action-prediction signals. This creates a bottleneck: without successful policies, we cannot verify reasoning quality; without quality reasoning, we cannot build robust policies. We introduce R&B-EnCoRe, which enables models to bootstrap embodied reasoning from internet-scale knowledge through self-supervised refinement. By treating reasoning as a latent variable within importance-weighted variational inference, models can generate and distill a refined reasoning training dataset of embodiment-specific strategies without external rewards, verifiers, or human annotation. We validate R&B-EnCoRe across manipulation (Franka Panda in simulation, WidowX in hardware), legged navigation (bipedal, wheeled, bicycle, quadruped), and autonomous driving embodiments using various VLA architectures with 1B, 4B, 7B, and 30B parameters. Our approach achieves 28% gains in manipulation success, 101% improvement in navigation scores, and 21% reduction in collision-rate metric over models that indiscriminately reason about all available primitives. R&B-EnCoRe enables models to distill reasoning that is predictive of successful control, bypassing manual annotation engineering while grounding internet-scale knowledge in physical execution.
@inproceedings{GanaiLuoEtAl2026, author = {Ganai, M. and Luo, K. and Frey, J. and Barrett, C. and Pavone, M.}, title = {Self-Supervised Bootstrapping of Action-Predictive Embodied Reasoning}, booktitle = {{Robotics: Science and Systems}}, year = {2026}, address = {Sydney, Australia}, owner = {mganai}, url = {https://arxiv.org/abs/2602.08167}, timestamp = {2026-04-26} } -
S. He, L. Huang, A. Lilja, F. Hubel, J. Frey, M. Pavone, S. S. Sastry, J. Malik, and C. Tomlin, “FARM: Find Anything using Relational Spatial Memory,” arXiv preprint arXiv:2606.15476, 2026. (Submitted)
Abstract: Robots operating in homes, warehouses, and other object-rich environments need memory systems that can find specific object instances on demand. Object-level memory alone is often insufficient: scenes contain many plausibly matching objects, and users refer to the target through relations to landmarks and surrounding objects (e.g. “the tall lamp below the dartboard and to the left of the poster”), demanding a relational spatial memory that supports retrieval through semantic, appearance, and spatial predicates over objects. To achieve this, we present FARM (Find Anything using Relational Spatial Memory), which builds, in real time at 5-10 Hz, a compact, open-vocabulary, object-level memory with geometry, visual-language descriptors, and viewpoint evidence. At query time, FARM uses VLMs to parse the query and score visual evidence, while grounding spatial constraints explicitly through object symbols and relational predicates. This structured use of VLMs enables more accurate and robust retrieval than end-to-end reasoning over frame histories or scene-graph context. In experiments on 44k language queries spanning 67 indoor and outdoor scenes, ranging from 15 to 15,000 m^2, FARM improves Recall@5 and Recall@10 over prior methods by 164% and 224%, and a final VLM reranking stage improves Accuracy@1 by 35%, while running in real time. We further demonstrate closed-loop deployment on a quadrupedal robot using onboard sensors and compute.
@article{HeEtAl2026, title = {FARM: Find Anything using Relational Spatial Memory}, author = {He, Siming and Huang, Leo and Lilja, Adam and Hubel, Fabio and Frey, Jonas and Pavone, Marco and Sastry, S Shankar and Malik, Jitendra and Tomlin, Claire}, journal = {arXiv preprint arXiv:2606.15476}, keywords = {sub}, owner = {frey}, timestamp = {2026-06-26}, url = {https://arxiv.org/abs/2606.15476}, year = {2026} }