In the video, Ornella Bahidika and Joel Allou from Microsoft introduce Ace, a live AI voice tutor designed to run complete lessons reliably from start to finish. They emphasize that the key to Ace’s success is that the large language model (LLM) is not in control of the lesson flow. Instead of relying on the LLM to manage multi-step processes, which often leads to skipped steps, premature endings, or loops, they treat the model as a talented actor directed by a harness that manages the lesson’s state and progression.
Ornella explains that the lesson is structured as a small state machine with defined steps such as introduction, teaching, checking, grading, advancing, and wrapping up. Each step sends a specific, focused task to the model, which returns the output. The harness then validates this output, updates the lesson state, and determines the next step. This design ensures the model never decides the lesson’s flow, improving reliability and control.
Joel elaborates on the concept of harness engineering, contrasting it with approaches that rely heavily on large, complex models to handle all aspects of processing and decision-making. By building a harness that controls the lesson’s steps and provides only the necessary input to the model, they can use smaller, more efficient models like Haiku 4.5 instead of larger ones like Opus 4.7. This approach reduces costs, latency, and complexity while maintaining performance.
The video also showcases logs from a lesson, illustrating how the harness manages various components such as speaking prompts, whiteboard drawings, queue clearing, and lesson termination. Joel highlights that the harness handles critical questions like when the lesson is done, whether the student has learned the material, and what the next steps should be. This external control ensures the model focuses solely on generating outputs for specific tasks without managing the overall flow.
In conclusion, Ornella and Joel stress that this harness-based approach is broadly applicable beyond voice tutors, including coding agents, operational runbooks, and onboarding flows. They advise developers to assess the reliability of their agents and, if outcomes are inconsistent, to remove control flow decisions from the model and build abstractions around it. Their key message is to let the model “talk” but not “drive,” ensuring dependable and efficient AI-driven applications.
