The video discusses Flint, a new mixture of experts (MoE) coding model developed by Recursive, which aims to offer a faster alternative to dense models like Quen 3.827B. While dense models provide strong performance, they tend to be slow, especially on consumer hardware like MacBook Pros. Flint, with its 40 billion parameters but only 2.5 billion active per token, achieves speeds of around 130 tokens per second on an M4 MacBook, roughly ten times faster than Quen 3.8. This speed advantage makes Flint particularly appealing for applications involving multiple calls or agent loops, despite some trade-offs in coding benchmark performance.
Flint’s architecture is notable for its use of a sliding window attention mechanism in most layers, allowing it to handle very long contexts efficiently—up to 262k tokens with 65k output tokens. The model uses 256 experts, selecting six per token plus one shared expert, which helps reduce memory bandwidth demands and improve speed. Recursive has released Flint under an open Apache 2 license with multiple quantized versions available, including a highly optimized NVFP4 variant that balances speed and accuracy through quantization-aware training and distillation techniques.
The company behind Flint, Recursive, is led by Richard Socher, a well-known figure in NLP with a history of innovative work and leadership roles at Stanford, Salesforce, and various startups. The team includes experts from Google, OpenAI, DeepMind, and Meta, and has raised significant funding based on their vision of recursive self-improvement in AI models. Their Eureka machine system automates research by proposing, testing, and refining ideas in parallel, effectively creating a feedback loop that improves model architecture, quantization, and kernel optimizations without extensive human intervention.
Demonstrations of Flint show it performing well on coding tasks and agent workflows, with fast token generation speeds around 200 tokens per second on GPUs like the RTX Pro 6000. The model exhibits detailed, step-by-step reasoning in its outputs and can self-assess and revise its work, although it may require more explicit guidance than larger dense models due to its smaller active parameter count. While Recursive’s Mac-specific kernel optimizations provide exceptional speed, the model is also accessible through popular frameworks like Llama CPP and LM Studio, making it usable on a range of hardware.
In conclusion, Flint represents a promising step toward faster, more efficient coding models that can run locally on consumer hardware without sacrificing too much performance. While it does not yet surpass the best dense models in all benchmarks, its speed and open availability make it attractive for developers needing rapid inference. The broader vision of Recursive and their Eureka machine hints at future advances in recursive self-improvement and automated model development, potentially leading to even more powerful and efficient AI systems. The video encourages viewers to try Flint and share their experiences, highlighting the growing interest in alternatives to dense models for practical AI applications.
