Stephen Toub, a Microsoft Distinguished Engineer, undertook the monumental task of rewriting a 400,000-line codebase from TypeScript and Node.js to Rust, specifically for the GitHub Copilot runtime. The motivation behind this port was not merely to follow trends but to address real technical challenges: improving embeddability, efficiency, and performance in a rapidly evolving AI landscape. The original runtime, built on Node.js and V8, limited where and how it could be used, and with AI models and techniques advancing weekly, having a single, extensible, and high-performance harness was crucial for GitHub and Microsoft’s AI projects.
Toub experimented with multiple languages and runtimes, including TypeScript, Python, Go, C#, and Rust, evaluating them on performance, embedability, and interoperability. Rust and C# emerged as the top contenders, both offering strong performance and embedding capabilities. Ultimately, Rust was chosen due to factors like the existing Rust expertise within GitHub and the ecosystem’s momentum. The migration strategy was incremental rather than a big-bang rewrite, allowing continuous shipping and testing, which was essential given the fast pace of AI development and the need to avoid long pauses in feature delivery.
To manage correctness and regressions during the port, extensive end-to-end tests were critical. While unit tests existed, many had to be rewritten alongside the code due to the language change. The incremental approach allowed for continuous validation in production-like environments, catching most regressions early. Despite this, some bugs slipped through, often due to concurrency issues or agents inadvertently modifying tests or APIs. Toub leveraged AI agents extensively for code review and merging, focusing his human review on architectural and high-level decisions, trusting agents for detailed line-by-line checks.
The project’s cost was approximately $120,000 in AI token usage and about three weeks of Toub’s developer time spread over 14-15 weeks. This investment was significantly lower than traditional rewrites, which could take years and large teams. The benefits included drastically improved performance, reduced resource usage, and a shared, maintainable runtime that multiple teams could leverage. Toub emphasized that such migrations should be justified by clear business value and return on investment, and that modern AI tools have made previously prohibitive projects feasible and cost-effective.
Reflecting on his career, Toub highlighted the value of diverse roles beyond pure software development, including consulting, technical writing, product management, and advising executives, which enriched his perspective and skills. He recommended young engineers focus on mastering core principles and foundational knowledge, such as understanding low-level computing concepts and algorithms, to remain effective in an AI-augmented future. While uncertain about how AI will reshape software engineering, he stressed the importance of adaptability and continuous learning in the rapidly evolving tech landscape.
Useful Links
