# The data black hole at the center of AI **URL:** \ **Category:** Content Creators\ **Tags:** dwarkesh-patel\ **Created:** [19 June 2026 17:17 UTC](https://www.artofsm.art/t/the-data-black-hole-at-the-center-of-ai/20384 "2026-06-19T17:17:05Z")\ **Posts on this page:** 2\ **Page:** 1
**Author:** ![artesia](https://www.artofsm.art/user_avatar/www.artofsm.art/artesia/32/36_2.png) [@artesia](https://www.artofsm.art/u/artesia)\ **Post date:** [19 June 2026 17:17 UTC](https://www.artofsm.art/t/the-data-black-hole-at-the-center-of-ai/20384/1 "2026-06-19T17:17:05Z")
[![](https://www.artofsm.art/uploads/default/original/3X/8/9/89411d1320b4a3e95f4ebc16c490cb6cf9df55ff.jpeg "The data black hole at the center of AI") ](https://www.youtube.com/watch?v=4pG3SJQPAwk) The video highlights the vast data inefficiency of current AI systems compared to human learning, emphasizing that progress has largely depended on scaling data and compute rather than improving sample efficiency, which remains a critical challenge due to the reliance on costly, specialized human-generated data. It also discusses the implications of this inefficiency for automating complex tasks, the limitations of simply increasing model size, and the need to focus on enhancing AI’s learning principles to achieve more holistic and efficient intelligence. ---
**Author:** ![artesia](https://www.artofsm.art/user_avatar/www.artofsm.art/artesia/32/36_2.png) [@artesia](https://www.artofsm.art/u/artesia)\ **Post date:** [19 June 2026 17:37 UTC](https://www.artofsm.art/t/the-data-black-hole-at-the-center-of-ai/20384/3 "2026-06-19T17:37:26Z")
The video discusses the concept of sample efficiency as a key measure of intelligence, defined by how much data is needed for a system to perform competently in a domain. Despite advances in AI, there has been limited progress in improving sample efficiency; instead, improvements have largely come from scaling up data and compute resources. Reinforcement learning (RL) plays a central role by generating synthetic data through extensive computation against evaluative rubrics, often requiring vast amounts of highly specialized human expert data to guide the learning process. This expert data is task-specific, costly, and involves numerous human contributors providing examples, rubrics, and explanations, making the data industry a multi-billion-dollar enterprise. The scale of data used to train AI models dwarfs human experience by orders of magnitude. While a human might be exposed to around 200 million tokens of language by adulthood, state-of-the-art AI models are trained on tens to hundreds of trillions of tokens—a difference of nearly a millionfold. This vast data requirement highlights the inefficiency of current AI learning compared to humans, who can learn complex tasks like driving or teleoperating robots with far less data. The video emphasizes that current AI models are more like patchworks of many specialized skills rather than holistic learners, and that open-source models can catch up quickly to frontier models primarily because data, not architectural innovations, drives progress. Common objections to these comparisons are addressed thoughtfully. One objection is that humans benefit from evolutionary pretraining, but the speaker argues that evolution likely optimized hyperparameters and loss functions rather than preloading detailed knowledge, meaning humans still build their neural connections from scratch. Another objection involves the multimodal sensory data humans receive, but examples of blind or deaf individuals with general intelligence suggest that sensory input volume alone does not explain human intelligence. Lastly, the idea that simply scaling model size will close the sample efficiency gap is refuted by scaling laws, which show that increasing parameters yields only marginal reductions in required data, indicating humans operate on fundamentally different learning principles. The video also explores why sample efficiency matters, especially for automating white-collar work and AI research itself. Many common professional tasks can be incorporated into training data, allowing AI to perform them despite inefficiencies, because AI can be trained massively in parallel and amortize costs over billions of uses. However, jobs requiring out-of-distribution thinking, such as software engineering, remain challenging. The plan is to first automate AI research to improve sample efficiency and tackle harder problems. The speaker suggests that AI progress may accelerate but cautions against simplistic views of an intelligence explosion, proposing a more nuanced understanding of AI development based on current large language models. In conclusion, the video highlights the enormous data demands and sample inefficiency of current AI systems compared to humans, the critical role of specialized human-generated data, and the limitations of simply scaling models. It underscores the economic and practical implications of these inefficiencies for AI deployment in various fields and points toward future research on how AI might overcome these challenges. The speaker invites viewers to follow their ongoing work for deeper insights into the evolving landscape of AI intelligence and learning.