Google’s new AI breakthroughs, Titans and Miras, introduce a novel memory architecture inspired by the human brain, enabling models to maintain genuine long-term memory and understand vast amounts of information over extended periods. This advancement allows AI to outperform larger models like GPT-4 on long-context tasks with greater efficiency, bringing us closer to achieving artificial general intelligence.
Google has potentially solved one of AI’s biggest weaknesses: memory. Current AI models like ChatGPT, Claude, and Gemini struggle with long-term memory, especially during extended conversations or when processing large texts like entire books. This limitation stems from the Transformer architecture, which becomes exponentially slower and more expensive as the context length increases. Google’s new research introduces two breakthroughs, Titans and Miras, which aim to provide AI with genuine long-term memory, enabling it to remember and understand vast amounts of information over extended periods.
Titans is a novel AI architecture inspired by the human brain’s memory systems. It features three layers of memory: persistent memory (fixed knowledge from training), core attention (short-term, in-context learning), and long-term memory (which actively learns and updates while running). Unlike previous models that store memories as simple vectors, Titans uses a multilayer perceptron—a small neural network within the larger model—to store and process memories. This allows it to connect related information across thousands of words, much like how humans remember details and themes over time.
Miras, on the other hand, is a theoretical framework that unifies all major AI sequence models, including Transformers and RNNs, by showing they all function as forms of associative memory. It breaks down sequence models into four design choices: memory architecture, attentional bias, retention gate (forgetting mechanism), and memory algorithm (how memories update). This framework reveals that most models rely on mean squared error (MSE) for attention and retention, which is sensitive to outliers. Miras opens the door to alternative approaches that improve robustness and stability, leading to new models like YAAD, Moneta, and Memoriala.
Experimental results demonstrate the power of Titans’ deep memory architecture. Unlike existing models such as Mamba, which struggle and become confused as input sequences grow longer, Titans maintains low perplexity and high accuracy even with sequences exceeding two million tokens—equivalent to multiple entire books. This depth allows Titans to compress and understand complex information more effectively, capturing not just what happened but why it matters. In practical terms, this means AI can now handle tasks requiring deep understanding of long documents, such as legal contracts, medical records, scientific research, and large codebases.
The breakthrough is significant because Titans outperforms much larger and more expensive models like GPT-4 on long-context tasks, despite having fewer parameters and lower computational costs. This advancement opens up new possibilities for AI applications that require sustained memory and comprehension over time, bringing us closer to artificial general intelligence (AGI). By mimicking human-like memory systems, Google’s research marks a major step toward AI that can truly understand and remember information as humans do.