The episode features Francois Shaard explaining diffusion models, a powerful machine learning technique that excels at generating and reconstructing complex data by learning to reverse the process of adding noise. He highlights their broad applicability across fields like image generation, protein folding, and robotics, advising founders to prioritize diffusion models for future AI innovations due to their simplicity, versatility, and rapidly advancing capabilities.
In this episode of Decoded, the host interviews Francois Shaard, a YC visiting partner and Stanford PhD student, about diffusion—a foundational machine learning technique that has become central to modern AI. Francois explains that diffusion models are designed to learn data distributions, particularly excelling at mapping from high-dimensional input to high-dimensional output, even with relatively small datasets. Unlike traditional autoregressive models like large language models (LLMs), diffusion models use a process of adding noise to data and then training a model to reverse this process, effectively denoising and reconstructing the original data.
The conversation traces the evolution of diffusion models, starting from the seminal 2015 paper by Yoshua Bengio, which established the core components of modern diffusion techniques. Over time, researchers have experimented with different ways of adding noise, loss functions, and model architectures, leading to significant improvements in performance and efficiency. Notably, the process of predicting the error or velocity between noisy and original data has proven easier for models to learn than predicting the data itself, resulting in simpler and more effective training procedures.
Francois demonstrates the practical implementation of diffusion models, highlighting the importance of the noise schedule—the way noise is incrementally added to data during training. He explains that getting the noise schedule right is crucial for stable and effective learning. The discussion also covers recent innovations like flow matching, which simplifies the process by focusing on the global velocity between noise and data, allowing for extremely concise and generalizable code that can be applied to a wide range of data types, from images to proteins to weather data.
The episode emphasizes the broad applicability of diffusion models. While they are widely known for powering image and video generation tools like Stable Diffusion, Sora, and Midjourney, diffusion techniques are also revolutionizing fields such as protein folding (e.g., AlphaFold), robotics (diffusion policies for control), weather forecasting (GenCast), and even code generation. Francois notes that diffusion has become the state-of-the-art in nearly every area of AI except for LLMs and certain game-playing applications, where other methods still dominate.
For founders and researchers, Francois advises that diffusion should be a primary consideration for any machine learning application, whether building new models or leveraging existing ones. The technology is rapidly improving, becoming simpler and more powerful, and is poised to redefine entire industries. He encourages founders to “skate to where the puck is going,” predicting that diffusion-based approaches will soon enable breakthroughs in robotics, life sciences, and beyond, fundamentally reshaping the AI landscape.