The speaker presents research on integrating rigorous classical engineering methods with modern machine learning to develop scalable, safe autonomous systems that strictly enforce safety through techniques like control barrier functions and temporal logic-based task decomposition. Complementing this design approach, they emphasize a probabilistic, adaptive testing framework to systematically identify and analyze failure modes, enhancing robustness and trustworthiness in real-world applications such as autonomous driving and drone delivery.
The speaker presents research from their lab focused on building safe and intelligent autonomous systems by bridging rigorous classical engineering methods with modern machine learning approaches. They emphasize that while classical methods like control theory and formal methods provide strong safety and stability guarantees through mathematical models, these models can be limited in scalability and fidelity. Conversely, data-driven machine learning approaches offer scalability and model-free learning but often lack strict guarantees, especially critical in safety-sensitive applications such as human-robot interaction. The lab’s goal is to combine the strengths of both paradigms to create scalable, safe autonomy without compromising on rigorous safety assurances.
The first major approach discussed is ensuring safety and mission satisfaction by design. This involves embedding hard safety constraints directly into the learning process rather than treating them as soft penalties. The speaker illustrates this with a warehouse delivery example involving aerial and ground robots that must coordinate under strict safety constraints like collision avoidance and no-fly zones. They introduce the concept of control barrier functions (CBFs), mathematical constructs that guarantee safety by defining safe and unsafe states and guiding control inputs accordingly. Extending this to multi-agent systems, the lab developed graph control barrier functions (GCBFs) that enable decentralized policies ensuring global safety from local observations, allowing scalable coordination among many heterogeneous robots.
To handle complex mission objectives, the lab uses temporal logic to rigorously specify tasks with temporal and logical dependencies, such as recurring deliveries and safety requirements. They developed a value decomposition method that breaks down complex temporal logic specifications into atomic sub-tasks with known value functions, enabling parallel learning and execution with formal guarantees. This approach has been demonstrated in simulations involving large numbers of agents performing coordinated tasks and in hardware experiments including drone delivery and autonomous driving scenarios where traffic rules are encoded as temporal logic constraints. The result is a scalable, mathematically grounded framework for learning policies that satisfy complex missions while strictly enforcing safety.
Recognizing that no system is fail-proof, the second complementary approach focuses on testing and validation to identify when and how failures occur. The speaker frames testing as a probabilistic inference problem, sampling from a posterior distribution over environmental and design parameters to discover diverse failure modes rather than relying on brittle optimization methods prone to local minima. They propose an active inference loop where test cases are adaptively generated based on risk prediction models and user-defined utility functions, incorporating subjective assessments from humans or large language models. This framework enables systematic exploration of failure distributions, contextual definitions of failure, and retrospective analysis to diagnose root causes, improving robustness and trustworthiness of autonomous systems.
The talk concludes with examples of applying these testing methods to real-world problems such as autonomous racing, self-driving cars, and even large language model evaluation in disaster response scenarios. The speaker highlights the importance of combining rigorous design with evidence-based testing to build autonomy that is safe by design, scalable in architecture, and reliable through provable guarantees and empirical validation. They acknowledge challenges such as conflicting safety constraints and adversarial scenarios, emphasizing that their framework allows prioritization and relaxation of constraints when necessary, while continuously seeking to understand and mitigate failure risks through ongoing testing and analysis.
Useful Links
