The video provides an in-depth buyer’s guide for local AI GPUs, highlighting the current market prices, performance metrics, and value propositions of various GPUs and AIO units. The presenter introduces a comprehensive tool available on digitalspaceport.com that tracks 17 key GPUs, offering insights into their pricing trends, VRAM capacities, bandwidth, and price-performance ratios. Notably, the Nvidia RTX 5090 is discussed as a highly sought-after GPU with prices significantly above MSRP, driven by international demand and FOMO, especially from buyers in China. The 5090 leads in price per gigabyte of VRAM but comes with a hefty premium, making it a high-cost option for AI workloads.
Following the 5090, the Nvidia RTX 4090 is identified as the next hot GPU, with used prices also exceeding MSRP. It offers 24 GB of VRAM and about 1 terabyte per second of bandwidth, making it a strong performer but still expensive. The older 3090 model, despite being launched years ago, remains a compelling choice due to its lower price relative to performance, especially after factoring in potential maintenance like repasting thermal paste. The 2080 Ti and 3060 are also highlighted as attractive options for buyers seeking a balance between cost and capability, with the 2080 Ti offering good bandwidth and VRAM at a reduced price compared to its original MSRP.
The guide also covers enterprise-class GPUs like the Nvidia V100 and mining GPUs such as the CMP 170HX, which offer large VRAM capacities and high bandwidth but come with caveats like cooling requirements and the need for technical expertise. The V100, despite its age, is noted for its value given its substantial VRAM and bandwidth at a fraction of its launch price. The CMP 170HX, derived from the Ampere architecture, is recommended only for skilled users due to its complexity and the need for modifications. The presenter advises caution with certain GPUs like the 4060 Ti, which is deemed overpriced relative to its performance.
On the AMD side, the video discusses GPUs like the 9060 XT, 7900 XTX, and the Pro R 9700, noting their VRAM sizes, bandwidth, and pricing relative to MSRP. The 9060 XT is praised for its solid performance at a reasonable price, while the 7900 XTX and Pro R 9700 offer larger VRAM capacities but come at higher costs. The presenter mentions plans for further testing and videos on these AMD GPUs to better understand their suitability for AI workloads. The guide also touches on AIO systems from Apple, Nvidia, and AMD, comparing their VRAM, bandwidth, and price points, with a focus on the Apple M1 Max and M4 Pro models, which offer unified memory but may struggle with deep AI model workloads due to limited compute power.
Finally, the video emphasizes the utility of the provided spreadsheet tool, which allows users to sort and compare GPUs based on various metrics such as price per gigabyte of VRAM and bandwidth. The presenter encourages viewers to subscribe and engage with the channel for ongoing updates and price alerts, highlighting the dynamic nature of the GPU market. The guide serves as a valuable resource for anyone looking to purchase GPUs for local AI applications, balancing cost, performance, and availability in a rapidly changing market.
