In his TED talk, Noam Brown highlights that the recent advancements in AI are primarily due to increased scale rather than algorithmic improvements, emphasizing the importance of “system two thinking” over mere model scaling. He demonstrates that allowing AI to take more time to think significantly enhances its performance, suggesting that future AI development should focus on improving thinking time rather than solely increasing computational resources.
In his TED talk, Noam Brown discusses the significant advancements in artificial intelligence (AI) over the past five years, emphasizing that the key factor behind this progress has been scale. While there have been some algorithmic improvements, the current frontier models still rely on the Transformer architecture introduced in 2017. The main difference lies in the increased scale of data and computational resources used for training these models. For instance, while training the GPT-2 model in 2019 cost around $5,000, today’s models can cost hundreds of millions of dollars. This raises concerns about the sustainability of such scaling and whether AI will eventually plateau.
Brown shares his personal experience as a PhD student working on AI that could learn to play poker, a game that requires deep strategic thinking. Initially, the research community believed that simply scaling up existing models would lead to better performance. However, when Brown’s AI faced top human players in a competition, it lost significantly, despite having played nearly a trillion hands of poker in preparation. This highlighted a crucial difference: while the AI made decisions almost instantly, human players took the time to think through their decisions, which led Brown to explore the concept of “system two thinking” versus “system one thinking.”
Through experiments, Brown discovered that allowing the AI to think for just 20 seconds during a poker hand could enhance its performance to the equivalent of scaling the model size and training time by 100,000 times. This revelation shifted the focus of their research from merely scaling the model to incorporating longer thinking times, leading to a redesigned poker AI that ultimately triumphed over human experts in a subsequent competition. This success demonstrated the power of system two thinking in enhancing AI performance.
Brown notes that the benefits of longer thinking times are not limited to poker; they have been observed in other games like chess and Go, where AI systems like Deep Blue and AlphaGo also took time to deliberate before making moves. The relationship between scaling thinking time and model performance suggests that there is a viable alternative to simply increasing training costs. As the cost of querying these models remains low, there is potential for significant advancements by focusing on enhancing the thinking time of AI systems.
In conclusion, Brown argues that the future of AI does not have to be constrained by the current scaling paradigm. By investing in system two thinking, researchers can unlock new dimensions of AI capabilities. He encourages the audience to reconsider the common perception of AI as merely chatbots and to recognize its potential for solving complex problems. Brown believes that we are at the beginning of a new era in AI development, where scaling up thinking time could lead to revolutionary advancements in the field.