Joseph Wang and his co-founder Sid introduce Emulated, a data lab focused on enhancing the reliability and autonomy of AI agents, particularly in software engineering contexts. They highlight a common theme in AI development: the progression toward agents capable of performing complex, long-term tasks with minimal supervision. Their discussion centers on the challenges this evolution poses for data and model layers, emphasizing the need for sophisticated simulation environments that go beyond simple code generation to encompass the full scope of software engineering, including customer interactions, infrastructure management, and operational complexities.
They point out a significant gap in current AI models, which excel at application-layer tasks but struggle with infrastructure-level reasoning, such as handling database concurrency issues that can cause system failures. This gap is attributed primarily to limitations in the quality and scope of training data. Existing benchmarks focus narrowly on code generation within isolated codebases, neglecting the broader, real-world context in which software operates. Emulated addresses this by simulating entire software engineering companies within containerized environments, incorporating organizational contexts like projects, incidents, and customer feedback, as well as infrastructure challenges like network failures and distributed system complexities.
The team illustrates their approach with an example of an SCD consensus cluster, showing how traditional environments focus narrowly on source code, while Emulated’s simulations include the surrounding ecosystem—tickets, postmortems, deployment systems, and live traffic management. This holistic simulation requires agents to handle real-time operational issues, such as failing nodes and deployment rollbacks, mirroring the long-horizon, multifaceted nature of real-world software engineering. They emphasize that single-node sandboxes, while useful, are insufficient for capturing the complexity of modern infrastructure, which demands multi-node sandboxes with access to real cloud resources.
Joseph and Sid discuss the limitations of current post-training pipelines, which are typically homogeneous and confined to single-node environments, contrasting this with the need for multi-node, real-infrastructure simulations to better prepare AI agents for real-world challenges. They envision a future where AI agents can provision and manage real infrastructure, including resource provisioning, security, API management, deployment, monitoring, and billing. This vision entails significant technical challenges, such as managing the cost and complexity of simulating large-scale cloud services and bridging the sim-to-real gap to ensure agents perform reliably in live environments.
Finally, they explain the motivation behind Emulated’s focus on infrastructure: it aligns with their expertise and offers a clear, well-defined problem space that facilitates high-quality data generation. While infrastructure is their starting point, they acknowledge the potential to extend their approach to other domains and workflows. They invite engineers and researchers interested in distributed systems, model training, and AI autonomy to engage with them, emphasizing the exciting and complex challenges ahead in building fully autonomous software engineering agents capable of managing entire companies through high-fidelity emulation of real-world environments.
