The presentation begins by highlighting AMD’s comprehensive enterprise AI hardware options, ranging from the Helios MI450 servers designed for training large models to the AMD Instinct GPUs and Radeon AI Pro products, with EPYC CPUs positioned as a versatile solution for AI inference workloads. EPYC servers are particularly well-suited for a variety of AI tasks including generative AI with smaller models, recommendation systems widely used in social media and e-commerce, and classical machine learning applications like fraud detection and diagnostics. The key advantage of using EPYC for inference is the ability to utilize existing server capacity efficiently, enabling both batch processing during idle times and real-time, low-latency AI inference without the need for additional hardware.
A critical component enabling high-performance AI inference on EPYC CPUs is AMD’s ZenDNN software library, which offers up to a twofold performance improvement over native software stacks. ZenDNN integrates seamlessly with popular AI frameworks such as PyTorch, TensorFlow, vLLM, and llama.cpp through a plugin approach, facilitating easy adoption. The software incorporates advanced optimizations including graph optimizations, microkernels, and auto-tuning to maximize performance. Additionally, AMD is advancing low-precision model support, including 4-bit and 8-bit quantization as well as FP16 precision on the latest Venice processors, further boosting inference speed and efficiency.
The discussion then shifts to performance benchmarks, particularly the TPC-XAI benchmark, which evaluates end-to-end AI workflows across multiple real-world use cases like sales forecasting and fraud detection. AMD’s latest Venice EPYC processors demonstrate significant performance gains—up to three times better than previous generations and competitors—highlighting their suitability for agentic AI workloads that require holistic pipeline processing rather than isolated training or inference tasks. The benchmark results underscore the importance of CPUs in AI, especially as workloads diversify and demand more from both hardware and software ecosystems.
Further analysis focuses on classical machine learning methods and similarity search workloads, which underpin many everyday applications such as e-commerce recommendations and streaming content suggestions. Venice processors deliver substantial performance improvements in these areas due to a combination of hardware enhancements—higher core counts, improved instructions per cycle (IPC), faster memory and PCIe Gen 6 support—and software optimizations through AMD’s open-source AOCL libraries. These gains illustrate how CPUs are evolving to meet the demands of modern AI applications beyond traditional GPU-centric paradigms.
Finally, the presentation addresses the evolving landscape of agentic AI, where CPUs are increasingly responsible for diverse and dynamic workloads beyond merely hosting GPUs. In this new model, CPUs handle a wide range of tasks including metadata lookup, web crawling, database searches, and retrieval-augmented generation (RAG) operations, often in small, ephemeral containers tailored to specific functions. This shift leads to a more balanced CPU-to-GPU ratio and highlights AMD’s unique position in offering an end-to-end hardware and software AI solution spanning CPUs, GPUs, and DPUs. AMD’s open-source, software-first approach facilitates integration and customization, making EPYC CPUs a compelling choice for scalable, efficient AI inference in the agentic AI era.
