Shlok Khemani reviews the evolution and diverse designs of memory systems in AI chatbots like ChatGPT and Claude, highlighting challenges such as managing stale information, computational trade-offs, and the current inability to integrate rich external data or reconcile conflicting memories. Despite these limitations, he emphasizes the critical role of memory in personalizing AI and encourages ongoing innovation to advance this foundational aspect of human-AI interaction.
Shlok Khemani, an independent researcher, shares insights from his year-long study of memory systems in consumer AI applications, focusing on personalization. He begins by tracing the evolution of memory in AI chatbots like ChatGPT and Claude over the past three years. Initially, ChatGPT’s memory was limited to maintaining context within a single conversation thread, with no carryover between sessions. This changed in early 2024 with ChatGPT Memory v1, which allowed users to explicitly tell the system facts about themselves, but this placed the burden of memory management on users and suffered from issues like stale or outdated memories.
In April 2025, ChatGPT introduced Memory v2, featuring a “running profile” that automatically updated every few days by synthesizing important information from past conversations. This profile was dense and extensive but still had problems with inaccuracies and staleness. Unlike ChatGPT, Claude’s memory system initially lacked a running profile and instead used on-demand search tools over past conversations. However, by August 2025, Claude released its own running profile system, which was smaller, updated daily, visible, and editable by users, marking a divergence in design philosophies between the two platforms.
Khemani highlights that despite some convergence in memory architectures—both now having running profiles and tools to search past conversations—there is no single way to implement memory in AI. Different products like Gemini and various AI agents use diverse approaches tailored to their specific needs. He emphasizes that memory systems are integral to product development and cannot be outsourced, as they must evolve alongside the product itself. Additionally, memory design is constrained by computational costs, forcing trade-offs between profile size, update frequency, and serving costs.
The talk also touches on continual learning, explaining that running profiles represent a form of continual learning outside of model weights, where new information is continuously integrated into the user profile. However, Khemani raises open questions about whether this learning will ever be incorporated directly into model weights, especially at the individual user level, given the high costs of training. He references ongoing discussions about the economics and future of personalized AI models, suggesting that much remains uncertain.
Finally, Khemani voices frustration over the current limitations of AI memory systems, particularly their inability to reconcile conflicting information or reason over rich external data sources like emails or calendars. He points out that many AI products maintain isolated memories, forcing users to repeatedly rebuild context across platforms. Despite these challenges, he remains optimistic, viewing memory in AI as a nascent but foundational field that will be crucial for the future of human-AI interaction. He concludes by encouraging continued innovation in this space, underscoring the importance of memory for personal AI’s evolution.