In his presentation at AMLDS 2026, George Montañez, an associate professor of computer science, introduces the concept of information cost as a universal metric to quantify task difficulty across various domains. He begins by framing the challenge of measuring how much knowledge or information is necessary to accomplish a task, using examples from computational biology and automata theory. Montañez explains that traditional complexity measures focus on time or space, but for knowledge-intensive tasks, quantifying the information required offers a more insightful perspective. He illustrates this with a card-flipping example, showing how different types of information—such as yes/no questions or additional attempts—can reduce the information cost and thus the difficulty of the task.
Montañez elaborates on how information cost serves as a common currency, akin to fiat money, enabling direct comparison of diverse types of assistance or knowledge, such as data, bias, or rule knowledge. He emphasizes that information theory, through concepts like Shannon surprisal, allows researchers to measure task difficulty in bits, making it easier to reason about than probabilities. The framework can be applied to various scenarios, including selecting items from a set with or without prior knowledge, narrowing down search spaces, and improving sampling strategies. This approach also reveals a conservation of information principle, where improvements in task success probabilities correspond to changes in information cost.
The presentation extends to practical applications, such as protein design in biology, where the information cost can quantify the complexity of creating functional proteins or optimizing mutation operators. Montañez also discusses reinforcement learning systems like AlphaZero and MuZero, highlighting how prior knowledge of game rules significantly reduces the information cost and accelerates learning. By decomposing complex tasks into subtasks, researchers can calculate the cumulative information cost, providing insights into the value of different types of knowledge and learning strategies.
Montañez expresses particular interest in large language models (LLMs), posing the open question of whether these models generate more information than they consume through training data, prompting, and architecture. He suggests that the information cost framework offers a way to analyze and compare the impact of various factors on LLM performance, potentially revealing whether they are net information producers or merely consumers. This line of inquiry is ongoing, with the framework poised to deepen understanding of AI systems’ informational dynamics.
During the Q&A, Montañez addresses questions about the generalizability of the framework to other domains, its relationship to computational costs like token usage, and challenges in interpreting the meaningfulness of information cost. He clarifies that while information cost quantifies the minimum information needed, it does not guarantee the usefulness of that information. He also draws connections between information cost and classical computational complexity, such as the lower bounds on sorting algorithms, reinforcing the framework’s theoretical robustness. Overall, Montañez presents information cost as a powerful, versatile tool for researchers to rigorously assess and compare task difficulty across disciplines.
