Aditya Bhargava argues that the framework and tools surrounding AI models—the harness—can significantly impact performance, often more than the model itself, especially for open-source or weaker models. He introduces Agency, a programming language designed to build safe, efficient, and autonomous agents through incremental enhancements like tool integration, safety mechanisms, sub-agents, and self-optimization, promoting democratized AI development beyond proprietary systems.
In this talk, Aditya Bhargava, a staff engineer at Etsy and author of “Grokking Algorithms,” challenges the prevailing notion that the model is the most critical component in AI systems. Instead, he argues that the harness—the framework and tools surrounding the model—can matter more, especially for weaker or open-source models that can be run locally. He highlights research showing that different harnesses applied to the same model can yield performance differences of over 20 percentage points, emphasizing the potential of investing in harness development to reduce reliance on proprietary models.
Aditya introduces his work on Agency, a new programming language designed specifically for building agents, which he equates with harnesses in this context. Agency aims to simplify the creation of agents by providing language-level support for essential features like tool integration, safety mechanisms, and interrupt handling. He demonstrates how Agency’s syntax, inspired by TypeScript and Python, allows developers to define tools easily and incorporate them into agents, enabling capabilities such as reading and writing files while maintaining safety through human-in-the-loop approval processes.
The talk progresses through a series of examples illustrating how to improve an agent’s harness incrementally. Starting with a basic model that cannot interact with files, Aditya adds tools, then safety handlers requiring user approval, and finally partial function application to constrain tool usage safely without human intervention. He further enhances the agent by implementing a feedback loop based on the REACT pattern (Reason and Act), allowing the agent to iteratively test and fix code until it passes all tests autonomously, showcasing the power of a well-designed harness.
Aditya then explores advanced harness concepts such as sub-agents, which are modular agents used as tools by a main agent to extend capabilities without overwhelming context or confusing the system. He also discusses self-optimization, where Agency’s built-in optimizers systematically improve agent performance by tuning prompts and strategies based on measurable objectives. This approach moves beyond trial and error, enabling continuous, data-driven enhancement of the harness.
In conclusion, Aditya emphasizes that building a better harness involves a progression: equipping agents with tools, ensuring safety, enabling autonomy, incorporating reasoning, leveraging sub-agents, and applying self-optimization. He invites developers to experiment with Agency to create powerful, safe, and efficient agents that can rival cutting-edge models while running locally. His work suggests a promising future where harness innovation democratizes AI capabilities beyond the control of a few large companies.