This episode of The AI Hardware Show explores how CPUs from AMD, Intel, IBM, Arm, RISC-V, and emerging players are evolving to support AI workloads, emphasizing their essential roles in orchestration, memory management, and system scaling alongside GPUs. The hosts highlight innovations like AMD’s Turin, IBM’s Telum 2, Intel’s Granite Rapids, and new RISC-V designs, illustrating the increasing diversity and specialization of CPUs in modern AI infrastructure.
This episode of The AI Hardware Show, hosted by Ian Cutress and Sally Fox, focuses on the evolving landscape of CPUs designed for AI workloads. The discussion covers major industry players—AMD, Intel, IBM, Arm, RISC-V, and emerging companies—highlighting how CPUs remain crucial for AI infrastructure, not just for computation but also for orchestration, memory management, and system scaling. The hosts emphasize that while GPUs often get the spotlight for AI, CPUs are essential for managing data flow, pre- and post-processing, and ensuring efficient utilization of accelerators.
AMD’s fifth-generation EPYC processors, codenamed Turin, are built on the Zen 5 architecture and target a wide range of applications from hyperscale cloud to AI model training. Turin offers up to 192 cores and 384 threads, with significant improvements in instruction throughput, power efficiency, and AI readiness. AMD positions Turin as not only a high-performance compute engine but also as a coordinator for GPU-powered AI clusters, aiming to reduce bottlenecks and improve total cost of ownership for large-scale deployments.
IBM’s Telum 2 processor continues the company’s focus on ultra-secure, low-latency transactional workloads, such as those found in mainframes. Built on Samsung’s 5nm process, Telum 2 features eight high-performance cores, a large and innovative cache structure, and an integrated AI accelerator optimized for real-time inference tasks like fraud detection. The chip also includes a dedicated on-chip data processing unit for IO acceleration, supporting IBM’s strategy of pairing CPUs with specialized AI accelerators for scalable enterprise deployments.
Intel’s Granite Rapids, part of the Xeon 6 family, leverages Intel’s advanced manufacturing and modular chiplet design to deliver up to 128 high-performance cores per socket. It is designed to serve both as a host for AI accelerators and as an AI processor itself, featuring advanced matrix extensions and AVX-512-based acceleration. Granite Rapids supports extensive memory and IO capabilities, and is intended to work alongside its efficiency-focused sibling, Sierra Forest, to address the full spectrum of AI infrastructure needs from cloud to edge.
The episode also highlights the rise of RISC-V and other open architectures, with companies like Tenstorrent and Ventana Micro (recently acquired by Qualcomm) offering licensable, customizable CPU IP for AI and edge computing. Tenstorrent’s Ascalon core and Ventana’s Veyron V2 are designed for high performance, scalability, and openness, challenging traditional x86 and Arm incumbents. Additionally, the New Reality NR1 chip is introduced as a specialized AI CPU for orchestrating data flow and maximizing GPU utilization in inference workloads, underscoring the growing diversity and specialization in AI hardware beyond just GPUs. The hosts conclude by noting the critical but often underappreciated role of CPUs in modern AI systems.