Phil Hetzel from Braintrust outlined the four maturity phases of running AI agent evaluations, progressing from manual human annotation to automated LLM-based judging and complex interaction trace analysis for agents interfacing with external systems. He emphasized the importance of evolving eval strategies to ensure agent quality, reliability, and safety in real-world scenarios, advocating for a balanced approach combining deterministic checks and LLM evaluations while continuously refining methods through production data and advanced analytics.
Phil Hetzel, lead of solutions engineering at Braintrust, presented on the maturity phases of running evaluations (evals) for AI agents. He began by introducing himself and his background in consulting and systems implementation, highlighting his transition to Braintrust due to his interest in bridging the gap between generative AI proofs of concept and production deployment. Braintrust focuses on agent quality through evals and observability, which are crucial for ensuring agents perform reliably and safely in real-world scenarios, mitigating risks related to reputation, cost, compliance, and legal issues.
The core purpose of evals is to maintain agent quality by testing how well agents handle real usage and failure modes. Unlike exhaustive unit tests, evals focus on high-level failure modes identified by subject matter experts, aiming for directional rather than perfect results. Evals consist of three components: the task (agent or prompt under test), the dataset of example inputs, and scoring functions to judge output quality. Phil emphasized that evals evolve as agents become more complex, requiring more sophisticated evaluation strategies.
Phil outlined four maturity stages in eval practices. The first stage, “just getting started,” involves manual human annotation where experts review agent outputs, provide thumbs up or down, and justify their decisions to capture domain knowledge. This foundational step helps build a knowledge base for scaling evaluations. The second stage introduces automation by using large language models (LLMs) to judge agent outputs, supplemented by deterministic code checks for objective failure modes. This stage also involves incorporating production or user acceptance testing (UAT) data to better simulate real-world conditions.
The third stage addresses the complexity of agents interacting with external systems through tool calls, including data gathering and CRUD operations. Evaluating these agents requires capturing and analyzing entire interaction traces, which can be large and complex. Challenges include representing external system states during offline evals and safely interacting with live systems without affecting production data. Emerging solutions involve embedding system state within traces and using time-based queries on vector databases to approximate production environments during evaluation.
In closing, Phil touched on advanced techniques such as topic modeling to automatically identify failure modes in production and the use of cloud code and CLI tools for automated eval workflows. He acknowledged the ongoing debate between deterministic and LLM-based evaluation methods, advocating for a balanced approach where LLM judges are themselves rigorously evaluated against human judgments. He invited further discussion at the Braintrust booth, emphasizing the evolving nature of evals as a critical component of agent quality management.
