In the discussion about the newly released open-weight large language model (LLM) Kimmy 3 by Moonshot, David Gerard expresses skepticism about the significance of AI benchmarks, describing them as marketing tools rather than rigorous scientific measures. He explains that benchmarks are often vendor-funded, easily gamed, and lack transparency in methodology, making their results unreliable indicators of a model’s true capabilities. While Kimmy 3 performs well on these benchmarks, Gerard cautions that this success is more reflective of effective marketing than a genuine breakthrough in AI technology.
Gerard highlights that the performance of large language models primarily depends on the amount of training data and model size, with Kimmy 3 having around 2.8 trillion parameters compared to competitors like Fable 5’s 6 trillion. He likens LLMs to lossy compressors for text, where larger models generally yield better results but with diminishing returns. Despite its size and open-weight status, running Kimmy 3 requires substantial computational resources, making it accessible mainly to those with significant infrastructure, such as data centers equipped with high-end GPUs.
Addressing claims that Kimmy 3 is a breakthrough, Gerard argues that it is mainly a marketing success designed to generate hype and investor interest rather than a transformative technological advancement. The model’s popularity and demand, which led to a pause in new subscriptions, reflect effective promotion rather than profitability or superior quality. He notes that the AI industry is characterized by heavy subsidies and financial losses, with companies like Moonshot and Deep Seek burning money to gain market attention and position themselves for potential IPOs.
The conversation also touches on the competitive landscape, where Western AI companies like OpenAI and Anthropic dominate due to their incumbency and resources, while Chinese models like Kimmy 3 and Deep Seek serve as challengers focusing on efficiency and cost-effectiveness. Gerard points out that despite political tensions and trade restrictions, Chinese AI development continues robustly, often leveraging gray-market hardware. He suggests that while Chinese models may not surpass Western ones imminently, they contribute to a more diverse and competitive market, especially as cost pressures push enterprises to consider alternatives.
Finally, Gerard discusses the future of AI commoditization, emphasizing that true commoditization requires profitability, which is currently absent due to the industry’s reliance on subsidies and high operational costs. Running local models remains expensive, often exceeding API costs, limiting widespread adoption. He predicts that prices will eventually rise to sustainable levels, allowing for more stable market dynamics. Until then, the AI sector will continue to be marked by intense competition, marketing-driven hype, and financial losses, with Nvidia as one of the few clear winners due to its dominant position in supplying essential hardware.
