The video compares running the massive 2.8 trillion parameter Kimmy K3 AI model on a costly $60,000 Mac Studio cluster versus the affordable and faster cloud-based Abacus AI Supercomputer, demonstrating that while the local setup is feasible, it is significantly slower and more expensive. Ultimately, the cloud solution offers superior speed, integration, and cost-effectiveness, making it ideal for most users, whereas local clusters suit those prioritizing data privacy or hardware control.
The video explores the capabilities of running a massive 2.8 trillion parameter AI model, Kimmy K3, on a local Mac Studio cluster versus using the cloud-based Abacus AI Supercomputer. Kimmy K3 is a huge model requiring 1.56 terabytes of disk space, which cannot fit on a single Mac Studio. To tackle this, the creator connected four Mac Studios, each with 512 GB of unified memory, using Thunderbolt 5 mesh and RDMA to create a 2 TB unified memory cluster. This setup was intended to compete against Abacus AI’s cloud supercomputer in building a front-end web application from the same prompt.
Abacus AI’s Supercomputer offers a cloud-based virtual machine that is always on, eliminating cold starts and provisioning delays. It supports over 100 frontier models, including Kimmy K3, and integrates with tools like GitHub, SSH, and scheduled tasks. The video demonstrates how both the Mac cluster and Abacus AI were tasked with building a production-ready Next.js web app with features like dark mode, accessibility, and unit testing. While the Mac cluster was able to run the model and generate code, it was significantly slower, processing about 14.7 tokens per second, compared to the cloud solution.
The Mac cluster eventually completed the web app after about four hours, producing a functional and visually appealing interface with features like live filtering, charts, and scheduling. Despite the slow speed, the local setup proved that running such a large model locally is feasible, which was not possible a year ago. On the other hand, Abacus AI’s cloud solution finished the same task in just 15 minutes, delivering a similarly impressive and polished application. The cloud model’s speed and integration capabilities make it ideal for quick, high-quality results.
Cost-wise, the Mac Studio cluster was very expensive, with each machine costing around $16,000, totaling approximately $60,000 for the four-machine setup. In contrast, Abacus AI’s cloud service starts at $7 to $10 per month, making it a much more affordable option for most users. The cloud service also offers flexibility with model selection and cost optimization through custom routing of tasks to cheaper or more powerful models as needed. This makes the cloud solution more accessible and practical for many users.
In conclusion, the video highlights that while running large AI models locally on a Mac cluster is now possible and impressive, the cloud-based Abacus AI Supercomputer offers superior speed, convenience, and cost-effectiveness. The choice depends on user needs: local clusters are suitable for those who require data privacy or enjoy managing their own hardware, whereas cloud solutions excel in delivering fast, high-quality results with minimal setup. Both approaches have their place in the evolving AI landscape.