Samuel Denton from Applied Compute presents a framework for integrating continual learning into enterprises through a “distillation spectrum” that ranges from offline to online learning, combined with offline and online hints to guide model improvements. He demonstrates practical techniques and empirical results showing how these approaches can enhance model performance both immediately and continuously, emphasizing Applied Compute’s commitment to advancing scalable, real-world continual learning solutions.
In this presentation, Samuel Denton from Applied Compute discusses the integration of continual learning into enterprise systems, focusing on the concept of distillation and its application across a spectrum from offline to online learning. He introduces the “distillation spectrum,” which ranges from offline distillation—where learning occurs from a single batch of production traces—to fully online distillation, where models continuously update in real-time as they serve production traffic. Denton emphasizes that enterprises exist at various points along this spectrum, and Applied Compute aims to provide solutions that deliver value across the entire range.
Denton further elaborates on a two-dimensional framework combining the distillation spectrum with the source of “hints” used to guide learning. Hints can be offline, derived from static data or known behavioral priors, or online, dynamically generated from the model’s current rollout in production. This creates four quadrants of distillation approaches: offline hints with offline traces, offline hints with online traces, online hints with offline traces, and online hints with online traces. Applied Compute primarily focuses on the first and fourth quadrants, which correspond to immediate improvements using offline data and scalable, continuous improvements through online learning.
The presentation includes empirical results demonstrating the effectiveness of these approaches. In one offline example, Applied Compute improved a model’s behavior on a coding benchmark by encouraging it to complete tasks more efficiently without degrading overall performance. This was achieved by injecting offline hints into production traces, nudging the model toward desired behaviors without explicit changes to the output tokens. In an online example, they improved a coding agent’s ability to format hyperlinks correctly—a behavior that was previously out of distribution—by using online hints tailored to each rollout, resulting in a significant increase in correct formatting without sacrificing other capabilities.
Denton shares practical insights and techniques that enhance distillation effectiveness. One key tip is the importance of per-step hinting, where hints are injected at specific points in the rollout rather than at the start, allowing more targeted and effective learning. Another technique is relevance mask self-distillation, which uses a judge model to select which tokens from the teacher model are most relevant for the student to learn from, helping to avoid learning irrelevant or detrimental patterns and reducing catastrophic degradation. These methods help refine the distillation process and improve learning outcomes.
In conclusion, Denton highlights distillation as a powerful tool for continual learning in enterprises, capable of delivering immediate value through offline methods and enabling ongoing improvements via online learning. Applied Compute’s research spans the full distillation and hinting spectrum, with a focus on practical applications that meet enterprises where they are today while pushing toward more advanced continual learning systems. He closes by acknowledging his team’s contributions and inviting interested individuals to join their efforts in advancing continual learning research.
