The AMD Helios system’s MI455X GPU features a chiplet-based design with a reorganized cache hierarchy and redesigned compute cores to optimize AI workload efficiency, delivering enhanced tensor throughput and support for advanced numerical formats. It also incorporates improved IO subsystems, synchronization primitives, and data movement architectures, enabling high-performance, scalable AI computing through a co-designed hardware and software approach.
The presentation focuses on the architecture of AMD’s rack-scale Helios system, specifically highlighting the GPU architecture embodied in the MI455X GPU. The speaker emphasizes the evolution of AI infrastructure over the past decade, moving from traditional high-performance computing extensions to more integrated rack-scale architectures. The key to current GPU design is system-level optimization tailored for specific AI workloads, rather than just enhancing raw compute and memory capabilities. The MI455X GPU features a chiplet-based design with 12 chiplets, including fabric cache dies (FCD) and fabric cache dies with shader engines (XCD), arranged in a 3D and 2.5D package to enable global memory access and efficient communication.
A significant architectural change in the MI455X GPU is the reorganization of the cache hierarchy. Unlike previous generations where the L2 cache resided in the XCD chiplets and was backed by a third-level infinity cache, the L2 cache has been moved into the FCD chiplets, flattening the cache structure. This change allows the L2 cache to back multiple shader engines, improving cache bandwidth and reducing kernel-to-kernel latency by eliminating the need for frequent cache flushes. Additionally, the IO subsystem has been separated into dedicated IO dies, facilitating high-speed connections to the EPYC host and scale-up and scale-out networks, which are essential for cluster and pod-level GPU communication.
The compute core has undergone a major redesign from the previous CDNA4 architecture. The traditional compute unit has been replaced by a work group processor featuring four 32-lane SIMDs that natively support wave 32 execution, improving instruction dispatch efficiency and reducing register pressure and branch divergence penalties. This redesign enables finer-grain interleaving of operations and better workload efficiency. The MI455X maintains 256 work group processors, consistent with prior models, but delivers significantly increased tensor throughput and enhanced support for transcendental and vector operations, balancing training and inference workloads effectively.
In terms of numerical formats and AI performance, the MI455X continues to support a range of floating-point and mixed-precision formats, including MXFP 8, 6, and 4, with new additions such as block 16 and fractional scaling for MXFP 4 to reduce quantization error and improve convergence. The architecture also introduces programmable 4-bit tensor formats with lookup table instructions for compression and decompression, enabling more efficient memory usage. A variant of the MI400 series, the MI430, targets traditional simulation workloads with enhanced double-precision floating-point capabilities, while the MI455X focuses on AI performance.
Finally, the GPU architecture incorporates several features aimed at improving efficiency and programmability. These include clusters that group multiple work groups for better data sharing and work placement, new synchronization primitives like split and named barriers to optimize producer-consumer workflows, and advanced prefetchers to hide instruction and data latency. The front-end kernel dispatch has been optimized for lower launch latency, and a new SDMA architecture simplifies and accelerates data movement within the GPU and across the Helios system. Overall, the MI455X GPU represents a co-designed hardware and software solution that balances compute, memory, and communication to deliver high performance and efficiency for AI workloads at scale.