The Stanford MS&E435 class on the Economics of the AI Supercycle featured a deep discussion on the GPU economy and the critical role of compute in AI’s rapid growth. Brad Gersonner, founder and CEO of Altimeter, and Sunny Mudra, a serial entrepreneur and key figure behind Grok (recently acquired by Nvidia), led the conversation. They emphasized that unlike traditional software, AI applications require massive computational resources, making the economics of AI fundamentally different. The discussion highlighted how AI’s demand for compute, especially for inference tasks, is exploding due to the shift from pre-training to inference-time reasoning, which consumes vastly more tokens and compute power.
Sunny Mudra detailed Grok’s innovative chip architecture, which uses a deterministic data flow design with high-bandwidth SRAM memory, differentiating it from traditional GPUs. Grok’s approach allows for more efficient inference, and by partnering with Nvidia, they combined their strengths to significantly increase token output per unit of power. This collaboration exemplifies how competition in the AI hardware space can evolve into strategic partnerships to meet the surging demand for AI compute, which is constrained by power and memory limitations.
The conversation also covered the dramatic reduction in the cost of AI inference, which has dropped by over 90% in the past year and nearly 99% over the last two years. This cost decline is driven by innovations in chip design, supply chain improvements, and software optimizations. Despite the increasing size and complexity of AI models, the value delivered by AI intelligence is growing even faster, leading to higher willingness to pay and positive gross margins for AI companies like OpenAI and Anthropic. The rapid revenue growth of these companies confirms that the AI market is scaling sustainably rather than being a speculative bubble.
Looking ahead, both Brad and Sunny expressed optimism about the continued exponential improvements in AI capabilities and cost efficiency. They noted that leading companies like Nvidia are pushing for 100x improvements in chip performance and leveraging AI itself to design next-generation hardware. The panel also discussed the societal implications of AI’s rapid advancement, emphasizing the need for thoughtful public policy and initiatives like Invest America to ensure broad-based economic benefits and address distribution challenges in the age of abundance.
In the Q&A session, the speakers addressed concerns about job displacement, the integration of AI with consumer devices like Apple’s ecosystem, and the balance between hype and realistic expectations from AI CEOs. They underscored the importance of becoming “bionic” by augmenting human capabilities with AI and highlighted the transformative potential of AI agents that automate complex tasks. Overall, the discussion painted a picture of an AI supercycle driven by unprecedented compute demand, technological innovation, and societal transformation, with Nvidia positioned as a dominant player in this evolving landscape.
