Nouha Dziri from Cohere presents a multi-agent team approach that achieves frontier-level coding results at up to 70% lower cost by strategically allocating more agents to the planning stage and combining a strong lead model with cheaper worker models. While this hierarchical structure improves task success and efficiency, challenges remain in coordination, communication, and scaling, highlighting the need for better team organization and adaptive strategies in complex coding tasks.
Nouha Dziri from Cohere presents a novel approach to optimizing multi-agent coding systems to achieve frontier-level coding results at significantly reduced costs. Traditionally, enhancing an agent’s performance involves either extending its reasoning time or running multiple independent attempts in parallel, both of which can be costly and inefficient for complex tasks. Dziri proposes splitting tasks among a team of agents, where a lead agent delegates subtasks to workers who operate with fresh contexts and report back compact results. This hierarchical team structure allows for faster completion of decomposable tasks but raises challenges in coordination, cost, and communication.
The research explores various design choices for organizing such agent teams, including team structure, communication methods, team size, model capabilities, and allocation of agents across different software engineering stages—planning, implementation, and verification. Experiments on challenging benchmarks like Program Bench, which involves reverse engineering binaries, show that while teams outperform solo agents in task success, they initially incur four to seven times higher costs. Two lead coordination styles were tested: blocking leads that wait for all workers to report before proceeding, and asynchronous leads that work concurrently but risk ignoring late results. Blocking leads performed better but were more expensive and slower.
A key insight from the study is that allocating more agents to the planning stage yields the greatest efficiency gains, especially in tasks where discovery and understanding of the program behavior are bottlenecks. Parallelizing planning allows multiple workers to explore different aspects of the task independently, providing the lead with richer information to guide implementation and verification. Conversely, adding agents to implementation or verification stages did not improve performance significantly and only increased costs. This finding was task-dependent, as other tasks like Commit Zero benefited more from scaling implementation agents.
Cost reduction was further achieved by combining a strong lead model with cheaper worker models. Using less expensive models for workers while retaining a powerful lead maintained near frontier performance but at substantially lower costs—up to 70% cheaper than homogeneous strong-model teams. Communication patterns revealed that most messages flowed through the lead rather than directly between workers, and increased peer-to-peer communication did not improve outcomes in the current setup. The study suggests that better communication protocols and team self-organization could enhance scalability and efficiency in future multi-agent systems.
In conclusion, Dziri emphasizes that multi-agent teams represent a promising new axis for scaling inference-time compute in coding tasks, but cost and coordination remain challenges. Effective team design requires identifying and scaling bottleneck stages, leveraging heterogeneous model capabilities, and improving communication strategies. Open questions include how to organize and scale teams of thousands of agents, enable adaptive team formation without extensive prompt engineering, and ensure safety and security in autonomous multi-agent systems. The work lays foundational insights for building cost-effective, high-performing agent teams for complex coding and software engineering tasks.