Andreas Blattmann and Dustin Podell from Black Forest Labs discuss their journey from developing foundational diffusion models like Stable Diffusion to creating advanced multimodal models such as Flux 3, which integrate images, video, audio, and robotics to mimic human intuitive intelligence. They highlight technical innovations, the importance of data quality, and practical applications including image editing with Flux Context and robotics control with Flux Action, while emphasizing responsible AI development and community engagement.
The episode features Andreas Blattmann (Andy) and Dustin Podell from Black Forest Labs (BFL), a research lab born out of the core team behind Stable Diffusion and Stability AI. Andy, a co-inventor of latent diffusion, and Dustin, who rose from a freelance engineer to research lead, discuss their journey from developing foundational diffusion models for image generation to expanding into multimodal models that handle images, video, audio, and robotics. They emphasize the philosophical beauty of diffusion modeling and how their German roots influenced the lab’s name and location. The launch of Stable Diffusion was a pivotal moment for many, including the hosts and guests, inspiring career shifts and igniting a vibrant open-source community.
BFL’s evolution from an image generation lab to a pioneer in visual intelligence is a central theme. They explain how their models have progressed from Stable Diffusion XL (SDXL), which introduced multi-resolution training and post-training refinements, to Flux 3, a multimodal model trained on images, videos, audio, and actions. This progression reflects their goal to mimic human intuitive intelligence (system one) by learning from natural signals and videos, enabling applications in video generation, image editing, and robotics. They highlight the importance of data quality and quantity, noting that better data often outweighs algorithmic tweaks in improving model performance.
The discussion delves into technical advances such as flow matching, a mathematically simpler and more scalable alternative to classic diffusion, and the transition from convolutional to transformer architectures. They also cover the challenges and breakthroughs in video modeling, including temporal consistency and the ability to fine-tune video-trained models for 3D understanding with minimal additional data. The guests contrast diffusion and autoregressive models, noting that while autoregressive models are more data-efficient during training, diffusion models offer faster sampling and scalability advantages. They also touch on the emerging research in continuous diffusion language models and reinforcement learning applied to diffusion.
A significant portion of the conversation focuses on Flux Context, BFL’s image editing model that enables character-consistent, high-speed edits using multimodal prompts (text and images). Dustin shares how Flux Context started as a side project and evolved through targeted data collection and distillation, addressing a major pain point in generative media: controllability and iterative editing. The team also discusses the importance of human evaluation, benchmarking, and community engagement in refining their models, emphasizing that real-world use cases and playful experimentation drive meaningful improvements beyond standard metrics.
Finally, the episode explores BFL’s robotics efforts with Flux Action, a model that integrates video, audio, and action prediction to control robot arms in real-world and simulated environments. They describe how their general multimodal pre-training enables efficient fine-tuning for specific robotic tasks with relatively little data, aiming for robots that can learn new actions from a single demonstration. Looking ahead, they predict significant commercial robotics applications emerging by 2027, especially in manufacturing, and express optimism about combining visual intelligence with language models for higher-level planning. Throughout, the guests stress responsible AI development, practical use cases, and the ongoing challenge of communicating the benefits of generative AI to broader audiences.
Useful Links
- Black Forest Labs blog on FLUX 3 — Directly explains the main claim about FLUX 3’s multimodal training and capabilities.