Nvidia CEO Jensen Huang explains that soaring AI computational demands, driven by increasingly complex models and iterative inference processes, are rapidly increasing the need for powerful GPUs. To meet this demand, Nvidia is advancing its chip technology annually and has already moved its new Vera Rubin design into full production to support this year’s AI growth.
Nvidia CEO Jensen Huang discusses the rapidly increasing computational demands of artificial intelligence (AI). He explains that the need for Nvidia GPUs is soaring because AI models are growing by an order of magnitude each year. This exponential growth is driven by the complexity and size of new AI models, which require significantly more processing power to train and operate effectively.
Huang highlights a major shift in AI development, referencing what he calls an “inflection point” marked by advancements like Obi-Wan’s introduction. Instead of providing simple, one-shot answers, AI inference has evolved into a more complex, iterative thinking process. This shift necessitates reinforcement learning, where computers learn by trial and error, rather than just supervised fine-tuning or imitation learning.
As a result of these changes, the computational requirements for pre-training, post-training, and test-time scaling have exploded. Huang notes that during inference, the number of tokens generated by AI models has increased fivefold each year. The longer an AI “thinks,” the better its answers tend to be, further driving up the need for computational resources.
Meanwhile, the competitive race in AI development is intensifying. Companies are striving to reach the next frontier in AI capabilities, and as new breakthroughs are achieved, the cost of generating tokens with previous-generation AI models drops by a factor of ten each year. This rapid progress underscores the importance of faster computation, as it enables organizations to reach new milestones more quickly.
To keep pace with these demands, Nvidia is committed to advancing computational technology annually. Huang mentions that Nvidia began shipping its GB200 chips a year and a half ago and is now in full-scale manufacturing of the GB300. He announces that the new Vera Rubin design is already in full production, ensuring it will be available in time to meet this year’s AI demands.