Stop Giving Your Agent Tools. Give It a Sandbox - Mikyo King, Arize AI

Mikyo King from Arize AI presents Code Mode, a novel approach that equips intelligent agents with programmable sandboxes to dynamically discover and execute only the necessary tools, enhancing efficiency, token usage, and enabling complex workflows beyond traditional static tool catalogs. By integrating multi-language sandboxes and rigorous evaluation frameworks, Arize AI demonstrates significant improvements in agent performance, cost reduction, and emergent behaviors, advocating for this paradigm shift in software development and agent design.

Mikyo King from Arize AI discusses the evolution of software development and introduces the concept of Code Mode as a transformative approach for building intelligent agents. Traditionally, software has been divided between tools for engineers and non-engineers, with engineers writing scripts to automate tasks while non-technical users rely on manual processes or pre-built software. Code Mode dissolves this boundary by enabling coding agents to dynamically build the software they need, leveraging foundation models trained to perform such tasks. This shift allows agents to operate more flexibly and efficiently, moving beyond static tool catalogs to dynamic tool discovery and execution.

Conventional agents typically rely on a large catalog of tools, calling them sequentially and accumulating context that can lead to inefficiencies such as context bloat, increased latency, and longer user wait times. Code Mode offers an alternative architecture where the agent can search for and execute only the necessary tools within a programmable sandbox environment. This environment supports complex workflows including sequential, parallel, and conditional execution of tools, allowing multiple tools to be called in a single turn and only essential information to be returned to the model’s context window. This approach reduces overhead, improves token efficiency, and shortens the interaction turns needed to achieve results.

The success of Code Mode stems from several key advantages: progressive disclosure of tools, composability of workflows, token efficiency, and emergent behavior. Instead of overwhelming the agent with all tools upfront, it selectively picks and uses tools as needed. The ability to compose tools in various sequences or parallel flows enables more sophisticated and efficient operations. Token efficiency is improved by limiting context window usage to relevant outputs only. Most notably, emergent behavior allows the agent to create its own tools dynamically, addressing complex tasks like data anonymization without exposing sensitive information in the context window.

Arize AI has embraced Code Mode by integrating it into their open-source observability platform, Arize Phoenix. They have implemented sandboxes for Bash scripting, Python programming, and SQL querying, enabling the agent to perform a wide range of operations from simple command-line tasks to complex data queries and UI interactions. This multi-language sandbox approach unlocks powerful capabilities such as aggregating data efficiently and interacting with the frontend in real-time. However, Code Mode also introduces challenges including resource management, security concerns around secrets and permissions, operational complexity, debugging difficulties, and the need for robust evaluation frameworks.

To address these challenges, Arize AI uses their own platform to rigorously evaluate Code Mode agents through the Harbor framework, which runs agents in controlled sandbox environments with strict network restrictions and repeated trials to reduce flakiness. Their evaluations show significant improvements post-Code Mode adoption, including a 43% reduction in token usage and a 31% decrease in cost, while maintaining latency. This data-driven approach allows them to refine agent behavior continuously. Ultimately, Mikyo emphasizes that Code Mode enables fewer interaction turns, supports long-horizon tasks, and fosters emergent behaviors, making it a compelling paradigm for future agent development. He encourages developers to give their agents sandboxes and leverage evaluation tools to unlock these benefits.

Useful Links