Kimmy K3 is a groundbreaking 2.8 trillion parameter open-weight AI model capable of handling a one million token context window, but running it locally on consumer laptops remains extremely slow and resource-intensive despite recent optimizations like Deltafin. While impractical for real-time use, its release marks a significant milestone in open AI models, highlighting the shift from access to model weights toward overcoming deployment and inference speed challenges.
The video discusses the release and capabilities of Kimmy K3, a 2.8 trillion parameter open-weight AI model made publicly available by Moonshot AI in Beijing. This model is the largest open-weight model ever shipped, with a file size of 1.56 terabytes using a specialized 4-bit quantization format called MXFP4bit, which was integrated during training rather than applied afterward. Kimmy K3 features a mixture of experts architecture with 896 experts per layer, activating only 16 experts per token across 93 layers, enabling it to handle a massive context window of one million tokens. The model employs innovative attention mechanisms, including Kimmy delta attention and gated latent attention, to manage memory efficiently and prevent cache explosion.
Despite its impressive scale, running Kimmy K3 locally on a laptop is extremely slow due to the massive data throughput required per token—about 25.8GB of RAM and 8GB of disk reads per token. A developer project called Deltafin has made it possible to run the model on a 2021 M1 Max laptop with 64GB RAM, achieving a speed of 14.6 seconds per token, which translates to roughly one word per minute. While this speed is far from practical for real-time use, it represents an 82-fold improvement from the initial 40-minute per token runtime just two days earlier. Deltafin achieves this by streaming only the necessary experts from disk on demand, using efficient quantization and optimized kernels.
The video highlights the debate around the utility of running such a large model locally at this speed. Critics argue that the slow token generation makes it impractical for real-world applications, especially compared to cloud APIs that deliver results almost instantly and at a fraction of the cost in time. However, proponents see value in the model as a research artifact and an existence proof for streaming inference, emphasizing the importance of having a fully local, self-contained AI model that cannot be remotely switched off or controlled. The model’s license allows free use and fine-tuning but restricts commercial use above a certain revenue threshold.
Hardware constraints also limit the model’s accessibility. The full model requires around 1.7 terabytes of disk space and significant RAM—far beyond what most consumer laptops can provide. Even the smallest quantized versions demand hundreds of gigabytes of memory, and recent changes in Apple’s hardware offerings have reduced maximum RAM options, making it harder for individuals to run such models locally. Despite these challenges, the release marks a significant milestone in open-weight AI models, with other large models like SK Telecom’s AXK2 also emerging, signaling rapid progress in the field.
In conclusion, Kimmy K3’s release represents a major leap in open-weight AI models, pushing the boundaries of scale and context length. While local usage remains slow and resource-intensive, projects like Deltafin demonstrate that running these massive models on consumer hardware is becoming feasible, albeit as a research tool rather than a practical product. The video underscores that the real challenge now lies in engineering efficient interfaces and infrastructure to make these models usable, shifting the bottleneck from access to weights to the practicalities of deployment and inference speed.