The VS Code team shares their experience shifting from monthly to weekly releases to better integrate rapidly evolving AI features, emphasizing the need for significant changes in development processes, engineering systems, and team collaboration to maintain quality and manage increased complexity. They highlight AI-powered tools for automated testing, issue triaging, and feature prototyping, alongside a dynamic, data-driven approach to planning and continuous improvement that balances innovation with stability.
In this talk, Pierce Boggan and Josh Spicer from the VS Code team discuss their experience transitioning VS Code’s release cycle from monthly to weekly, driven largely by the rapid advancements and integration of AI capabilities. They emphasize that adopting AI is not just about adding new features but requires significant evolution in product development, engineering processes, and team collaboration. The team observed a dramatic increase in AI-generated code commits and overall productivity, but also a corresponding rise in issues and pull requests, which necessitated more frequent, smaller releases to manage risk and maintain quality.
Josh highlights several AI-powered tools and workflows that have transformed the developer inner loop at VS Code. One key innovation is the component browser, which allows UI components to be tested and validated automatically within pull requests using AI-generated screenshots and iterative feedback loops. This tool accelerates development and improves confidence in changes without requiring full product builds. Additionally, AI agents assist in prototyping new features rapidly, enabling better communication and alignment across the team, while specialized “skills” help monitor performance regressions and other critical metrics continuously.
The team also invested heavily in evolving their engineering system to handle the increased volume and complexity of issues generated both internally and by the community. They built AI-driven automation for triaging issues, assigning owners, detecting duplicates, and even automatically fixing certain bugs based on telemetry data. This self-healing system reduces manual overhead and helps maintain product stability despite the faster release cadence and growing user base. The engineering system integrates tightly with GitHub and internal dashboards, providing transparency and control over the automated workflows.
Pierce explains the extensive effort involved in integrating new AI models into VS Code, including prompt engineering, offline and online evaluation through their custom VSC-Bench infrastructure, and continuous A/B testing to optimize performance and cost trade-offs. This rigorous process ensures that new AI capabilities deliver real value to users while balancing token usage and resolution rates. The team’s approach reflects a deep commitment to data-driven decision-making and continuous improvement in a rapidly evolving AI landscape.
Finally, the speakers describe how the team’s collaboration and planning model had to be reimagined to keep pace with AI-driven development. Traditional long-term planning gave way to dynamic, temporary workstreams focused on specific priorities, with daily or frequent check-ins to maintain alignment among the 40-person team. This agile approach helps manage the compressed delivery cycles and ensures that judgment and prioritization remain central despite the temptation to add features rapidly. The talk concludes with an invitation for feedback and emphasizes that successful AI adoption requires evolving not just technology but also team processes and culture.
