The video compares the AMD RX 9060 XT 16GB and Nvidia RTX 5060 Ti 16GB for budget local AI text generation, finding that both deliver solid performance with the Nvidia card offering slightly faster speeds and additional features like flash attention. While neither is ideal for video generation, the AMD card stands out for its cost-effectiveness, making both GPUs strong choices for Windows-based local AI setups.
The video compares two budget-friendly GPUs for local AI workloads: the AMD RX 9060 XT 16GB and the Nvidia RTX 5060 Ti 16GB. Both GPUs are PCIe Gen 5 with x8 lanes and have similar power requirements, making them suitable candidates for budget AI setups. The testing is conducted on a high-end rig featuring an AMD 7995WX CPU and DDR5-6400 memory to ensure the GPUs can perform at their best. The benchmarks focus on text generation performance using LM Studio with various language models, including OpenAI’s GPT-OSS 20B, Quinn 34B, and Magistral Small 259.
Starting with the AMD RX 9060 XT, the GPU achieved respectable speeds in text generation tasks. For GPT-OSS 20B, it generated about 89 tokens per second with a quick first token latency. The Quinn 34B model, which has a large context window, ran at 94 tokens per second, demonstrating good utilization of the GPU. However, the Magistral Small model, which does not fully fit into the GPU’s VRAM and requires offloading to system RAM, ran significantly slower at 16 tokens per second. This highlights the impact of VRAM capacity and offloading on performance.
Switching to the Nvidia RTX 5060 Ti, the GPU showed slightly better performance across the same models. It achieved 102 tokens per second on GPT-OSS 20B and 107 tokens per second on Quinn 34B, with a small but noticeable improvement over the AMD card. For Magistral Small, it managed 17 tokens per second, slightly faster than the AMD. Enabling Nvidia’s flash attention feature further improved performance to about 109 tokens per second, a feature not confirmed to be available on AMD cards. Overall, Nvidia holds a modest edge in raw text generation speed.
The presenter discusses use cases and practical considerations, noting that both GPUs are well-suited for budget local AI setups focused on text generation, especially on Windows. However, neither GPU is ideal for video generation workloads, where higher-end GPUs like the Nvidia 3090 are recommended due to VRAM and performance limitations. For gaming, the presenter plans to test both GPUs with Battlefield 6 to compare frame generation technologies and gaming performance. The $70 price difference between the two cards makes the AMD RX 9060 XT an attractive option for those prioritizing cost-effectiveness.
In conclusion, the performance gap between the AMD RX 9060 XT and Nvidia RTX 5060 Ti is narrower than expected, with both delivering solid results for local AI text generation tasks. The AMD card offers excellent value for money, while the Nvidia card provides some additional features and slightly better speeds. The presenter recommends both GPUs as good budget choices for local AI servers, especially for users running Windows. Future videos will explore Linux performance and image generation capabilities, providing a more comprehensive understanding of these GPUs in AI workloads.