In this episode of the AI Hardware Podcast, hosts Sally Ward Foxon and Ian Cutress explore the evolving role of CPUs in AI systems, discussing advancements from major players like AMD, Intel, IBM, and emerging RISC-V startups, highlighting innovations in chip design, AI integration, and market dynamics. They also introduce novel CPU architectures such as New Reality’s NR1 and IBM’s Telum 2, emphasizing specialized functionalities aimed at improving GPU utilization, reliability, and AI workload efficiency.
In this episode of the AI Hardware Podcast, hosts Sally Ward Foxon and Ian Cutress delve into the current landscape of CPUs, focusing on their role as essential hosts in AI systems. They discuss how CPUs, traditionally general-purpose processors, are evolving to better manage data movement and orchestrate AI accelerators like GPUs. The conversation highlights AMD’s latest Turin EPYC processors based on Zen 5 architecture, emphasizing their chiplet design and advancements in frequency and core counts. AMD’s integration of Infinity Fabric for optimized CPU-GPU communication and the challenges in market share dynamics, especially against Intel, are also explored.
Intel’s approach with its Granite Rapids Xeon processors is examined, noting their blend of high-performance and efficiency cores, and the company’s efforts to regain market share by offering cost-effective solutions and strong OEM partnerships. The hosts touch on Intel’s AI-focused extensions like AMX and the ongoing challenge of improving GPU utilization rates in AI workloads. They also discuss memory bandwidth improvements with DDR5 channels and the return of simultaneous multi-threading (SMT) in Intel’s designs to boost performance, particularly for AI applications.
The episode introduces New Reality’s NR1 chip, a novel type of CPU designed primarily as a host with specialized application-specific circuits for tasks like database and data flow management. This chip aims to significantly increase GPU utilization in AI systems by handling pre- and post-processing workloads more efficiently. The hosts note its integration of IBM IP and its potential to disrupt traditional CPU roles by blending general-purpose and application-specific functionalities, highlighting its cost-effectiveness and partnership with Qualcomm.
IBM’s Telum 2 processor is discussed as a unique, high-reliability CPU designed for transaction processing with built-in AI accelerators for inline fraud detection and security. The architecture emphasizes extreme uptime and resilience, featuring advanced cache designs and integrated DPUs for enhanced data center communication. This processor caters to industries requiring near-continuous operation and low-latency AI inference, showcasing a different approach to CPU design focused on reliability and specialized workloads.
Finally, the conversation turns to the RISC-V ecosystem, spotlighting startups like Tenstorrent, Ventana Micro, and Revos. Tenstorrent is highlighted for its high-performance RISC-V CPUs and AI integration, with ambitious chiplet-based designs and open architecture efforts. Ventana Micro’s evolving Veyron CPU family and its recent acquisition by Qualcomm are noted, alongside the challenges of software ecosystem development for RISC-V. The hosts express interest in novel and unconventional CPU designs, mentioning emerging players like Ahead Computing, and invite suggestions for future coverage of innovative CPU technologies.
