The video uncovers Apple’s hidden AI model, FM, embedded in Mac OS 27 Golden Gate, revealing that its decoding speed relies primarily on the Apple Neural Engine, resulting in similar performance across both Mac Mini and high-end Mac Studio despite hardware differences. It also introduces a benchmarking tool for FM, highlights the model’s current beta status on Apple Silicon Macs, and showcases emerging AI assistants like Super Nori that operate proactively.
The video reveals a hidden AI model called FM embedded within Mac OS 27 Golden Gate, accessible directly through the terminal without needing additional installations. This built-in large language model runs locally on Apple Silicon Macs and also offers a cloud-based option. The creator was curious about its performance, especially comparing the speed on a Mac Mini versus a high-end M3 Ultra Mac Studio, and set out to benchmark it using custom tools after existing benchmarks failed due to Apple’s proprietary implementation.
Initial tests showed surprisingly similar performance between the Mac Mini and the much more expensive Mac Studio, despite the latter having significantly higher memory bandwidth and GPU power. This unexpected result led to the hypothesis that the AI model might be running primarily on the Apple Neural Engine (ANE), which is consistent across these devices. However, monitoring tools failed to detect any ANE usage during FM’s operation, causing further investigation.
To resolve the mystery, the creator devised a clever experiment by stressing either the Neural Engine or the GPU while running FM. The results showed that FM’s token generation (decode stage) slowed down when the Neural Engine was hammered, while prompt processing slowed when the GPU was stressed. This confirmed that FM uses the Neural Engine for decoding and the GPU for prompt processing, explaining why the Mac Studio’s extra hardware didn’t improve overall speed.
The key takeaway is that the Neural Engine, which is similar across Apple Silicon generations, is the bottleneck for FM’s decoding speed. Therefore, a Mac Mini provides essentially the same AI performance as a top-tier Mac Studio for this built-in model, making the Mini a cost-effective choice for users leveraging FM. The video also highlights that other AI frameworks like MLX do utilize the GPU more effectively, offering different performance profiles.
Finally, the video encourages viewers to try out the Apple FM benchmarking tool available on GitHub and notes that this feature is still in beta and exclusive to Apple Silicon Macs. The presenter also briefly introduces Super Nori, a proactive family AI agent, illustrating the evolving landscape of AI assistants that act autonomously rather than waiting for user prompts. The video closes by inviting viewers to explore more about AI on Apple Silicon and stay tuned for future updates.