In this podcast episode, Dylan Patel, CEO of SemiAnalysis, discusses the biggest bottlenecks to scaling AI compute with host Dwarkesh Patel. The conversation begins by examining the massive capital expenditures (CapEx) of major tech companies like Amazon, Meta, Google, and Microsoft, which are collectively investing hundreds of billions of dollars annually into AI infrastructure. Patel explains that while some of this CapEx goes directly into compute coming online this year, much is allocated for future capacity, including long-term investments in data centers, power agreements, and semiconductor supply chains. He highlights how AI labs like OpenAI and Anthropic are racing to secure compute resources, with OpenAI being more aggressive in locking in long-term deals, giving them a margin and capacity advantage over more conservative competitors like Anthropic.
A central theme is the shifting nature of bottlenecks in the AI compute supply chain. While recent years saw constraints in areas like power, data centers, and chip packaging (CoWoS), Patel argues that the ultimate and most persistent bottleneck is in semiconductor manufacturing, particularly the production of advanced chips. The most critical constraint is the limited supply of extreme ultraviolet (EUV) lithography tools made by ASML, which are essential for producing cutting-edge chips at foundries like TSMC. Even with aggressive expansion, ASML can only increase production of these machines gradually, capping the global growth of AI compute. Patel details how each gigawatt of AI compute requires a significant share of these rare and complex machines, making them the linchpin of the entire ecosystem.
Patel also discusses the memory crunch, noting that high-bandwidth memory (HBM) is becoming a major cost and supply constraint for AI accelerators. As AI models require longer context windows and larger key-value caches, demand for DRAM and HBM has skyrocketed, driving up prices and squeezing out consumer devices like smartphones and PCs. This shift is causing consumer electronics to become more expensive and less capable, as memory manufacturers prioritize higher-margin AI customers. Patel predicts that a third of Big Tech’s CapEx will soon go toward memory, and that the lag in building new memory fabs means this crunch will persist for several years.
The conversation touches on alternative approaches to scaling compute, such as building data centers in space or using older semiconductor nodes (like 7nm) with advanced packaging. Patel is skeptical of space data centers in the near term, citing logistical challenges and the fact that chip supply, not power, is the main bottleneck. He also explains that while using older nodes or commodity DRAM could provide some relief, the performance and efficiency trade-offs are significant, and the industry’s focus will remain on pushing the limits of advanced nodes and packaging.
Finally, Patel explores the geopolitical and economic implications of these bottlenecks. He notes that the US and its allies currently have a lead in AI compute capacity, but China is rapidly working to indigenize its semiconductor supply chain. If AI progress is slower than expected, China could catch up and even surpass the West by 2035. Patel also discusses the centralization of AI compute in massive data centers, the impact of humanoid robots on chip demand, and the risks posed by geopolitical events in Taiwan. Ultimately, he concludes that the pace of AI progress will be dictated by the slowest-moving, most complex parts of the supply chain—especially the production of advanced chips and the tools required to make them.
