Robot safety is a nuanced concept. We commonly equate safety with collision-avoidance, but in complex, real-world environments (i.e., the “open world’’) it can be much more: for example, a mobile manipulator should understand when it is not confident about a requested task, that areas roped off by caution tape should never be breached, and that objects should be gently pulled from clutter to prevent falling. However, designing robots that have such a nuanced safety understanding---and can reliably generate appropriate actions---is an outstanding challenge. In this talk, I will describe my group’s work on systematically uniting modern machine learning models (such as large vision-language models and latent world models) with classical formulations of safety in the control literature to generalize safe robot decision-making to increasingly open world interactions. Throughout the talk, I will present experimental instantiations of these ideas in domains like vision-based navigation and robotic manipulation.

Our work is broadly motivated by the emergence of learning-based methods in control theory and robotics, with a specific focus on scenarios that have humans in-the-loop with control systems. For instance, learning algorithms are being deployed in semi-autonomous vehicles, robot assistants, brain-machine interfaces, and exoskeletons, where they interact dynamically with a human partner to complete tasks. When learning algorithms are employed in this way, a dynamic game is created that is played between two intelligent agents (the human and machine learners), requiring new techniques to guarantee safety and performance.

General-purpose robot policies hold immense promise, yet they often struggle to generalize to novel scenarios, particularly struggling with grounding language in the physical world. In this talk, I will first propose a systematic taxonomy of robot generalization, providing a framework for understanding and evaluating current state-of-the-art generalist policies. This taxonomy highlights key limitations and areas for improvement. I will then discuss a simple idea for improving the steerability of these policies by improving language grounding in robotic manipulation and navigation. Finally, I will present our recent effort in applying these principles to scaling up generalist policy learning for dexterous manipulation.

Many of us are collecting large scale multitask teleop demonstration data for manipulation, with the belief that it can enable rapidly deploying robots in novel applications and delivering robustness in the 'open world'. But rigorous evaluation of these models is a bottleneck. In this talk, I'll describe our recent efforts at TRI to quantify some of the key 'multitask hypotheses', and some of the tools that we've built in order to make key decisions about data, architecture, and hyperparameters more quickly and with more confidence. And, of course, I’ll bring some cool robot videos.

Aerial robotics have become ubiquitous, but (like most robots) they still struggle to operate at high speed in unstructured, cramped environments. By considering a vehicle's mechanical design simultaneously with the design of controls and automation algorithms, we have more degrees of freedom to find creative solutions to problems. In this talk I will present some of my group's work on enhancing aerial robots, including purely algorithmic approaches ('how can I do more with the hardware I already have?') and with hardware co-design ('how can I change the vehicle so that the hard problem is actually easy?'). I will discuss two exemplary challenges for aerial robots: first: flight through narrow, unstructured environments, and second: long duration and range flight within the constraints of battery-electric power. Lastly, I will discuss some work on adaptive and learning control, specifically for robustness to parametric uncertainty. For flight through narrow environments, I will present an algorithmic approach for high speed path planning that incorporates perception uncertainty, and can be used on a standard drone. We will then present two alternative approaches that modify the system design: one a vehicle that can change its shape to fit through narrower spaces, and a second that is highly collision resilient, and for whom collisions are therefore neither mission- nor safety-critical. For overcoming energetic challenges, we will present a strategy for real-time optimization of flight characteristics for a vehicle, specifically using extremum seeking control to modfiy the system airspeed and yaw angle; an algorithm that can be applied to any aerial robot. We then again show two design modifications to work around the problem -- first, a morphing system that can reduce its drag area at speed, and secondly a system capable of mid-air battery replacement for indefinite flight.

TBD

TBD

TBD

TBD