TurboFieldfare, developed by Andrey Mikhaylov, is an innovative engine that enables running the 26 billion parameter Gemma 4-26B MoE language model on a base M2 MacBook Air with just 8GB of RAM by streaming most model weights from the SSD and caching a small active subset in memory. While this approach allows large models to run on low-memory devices, it faces limitations in speed, context size, energy efficiency, and model performance, making it more of a proof of concept than a practical everyday solution.
TurboFieldfare is a novel engine developed by Andrey Mikhaylov that enables running a massive 26 billion parameter mixture of experts (MoE) language model, Gemma 4-26B, on a base M2 MacBook Air with only 8GB of unified memory. The key innovation is that the engine only keeps a small portion of the model—about 2GB—in memory at any time, while streaming the rest of the weights from the SSD as needed. This is possible because the MoE architecture activates only a small subset of experts (eight plus one shared expert) per token, reducing the active parameter count to around 4 billion, which fits within the limited RAM.
The engine is hand-written from scratch in Swift and Metal, designed specifically to overcome the challenge of fetching model weights from disk fast enough to maintain a conversational speed of about five to six tokens per second. It uses a custom file format and a 16-slot per layer least-frequently-used cache to keep recently accessed expert weights in memory, exploiting the fact that routing decisions tend to be correlated between adjacent tokens. This caching strategy significantly reduces the number of costly SSD reads, making the approach feasible despite the slow speed of disk access compared to RAM.
However, the approach has limitations. The model runs strictly sequentially without batching, which means the cost of SSD reads cannot be amortized across multiple tokens or parallel requests. Additionally, the KV cache, which tracks the conversation context, must remain in memory, limiting the maximum context window size on devices with limited RAM. Furthermore, independent tests show that while the engine runs the model, the model’s performance on tasks like coding is poor, indicating that the runtime is not the bottleneck but the model quality itself.
Another important consideration is energy efficiency and hardware wear. Frequent SSD reads to fetch expert weights increase power consumption and can accelerate SSD wear, as highlighted by a 2026 research paper on the energy costs of offloading MoE weights to SSDs. Thus, while TurboFieldfare enables running large models on low-memory devices, it comes with trade-offs in battery life and hardware longevity that users should be aware of.
Ultimately, TurboFieldfare is more of a proof of concept than a practical daily tool for most users. It demonstrates that open large language models are outgrowing available memory faster than memory capacity is increasing, and that innovative engineering can push the boundaries of what is possible on consumer hardware. While the engine currently supports only one model and performs better on machines with more memory, it has garnered significant attention and community interest, highlighting the potential for future developments in efficient local AI inference.
