Agent Memory Keeps Going Stale, Studies Show

Diagram of agent memory being captured, recalled and curated across AI sessions

Agent memory is mostly solved at the saving stage and unsolved at the keeping-it-true stage. Storing facts from one AI session for the next is now routine. Making sure those facts are still correct is where agents keep failing.

The test results are blunt. After facts changed, an agent that simply kept adding notes scored 0.21 in one study, while an agent with no memory scored 0.31. Stale memory was worse than nothing.

How agent memory works

Language models don’t carry state. When ChatGPT seems to recall that you’re vegetarian, separate software stored the note and feeds it into every new chat. Agent memory usually combines databases, embedding models and plain files.

Richmond Alake describes the discipline as memory engineering, built on four steps: retain, recall, reuse and refine. Refine means fixing stale memories, merging duplicates and retiring what no longer applies. It’s the step most products handle worst.

The steps act on four types borrowed from cognitive science: working memory (the current context window), episodic memory (records of past events), semantic memory (facts about the user or environment) and procedural memory (instructions and workflows, such as the CLAUDE.md or AGENTS.md files coding agents read).

An infographic showcasing memorization techniques
Image: newsletter.chainofthought.show

A short history, and where the labs are now

Memory started as search in 2020, with retrieval-augmented generation. UC Berkeley’s MemGPT then treated the context window like RAM in October 2023. In February 2024 ChatGPT began showing users the facts it kept about them. Assistants moved to running profiles in 2025.

The 2026 theme is upkeep. Anthropic added “dreaming” to its managed agents in May, and OpenAI released Dreaming for ChatGPT in June. Each runs in the background between sessions, reviewing and rewriting stored memories. Nous Research’s open-source Hermes Agent did something similar earlier.

  • ChatGPT’s Dreaming turns “You are going to Singapore in July” into “You went to Singapore in July 2026”.
  • Since August, Claude’s chat and Cowork share a single memory system, and users can read, edit or delete any topic.
  • Claude Code keeps a MEMORY.md index with a cap of 200 lines or 25KB, plus timestamped topic files.
  • OpenAI’s Computer History, which is opt-in, converts activity in approved Mac apps and sites into memories kept locally as markdown files.
Bar chart from the TEPA full-reversal test: append-only memory 0.210, last-write-wins 0.210, no memory 0.309, retiring stale memories 0.950.
Image: newsletter.chainofthought.show

Why agent memory goes stale

The author of the analysis gave his own agent persistent memory. Two weeks later it said a vendor approval was still pending, though he had cleared it a week earlier. Two of his workflow rules also gave agents opposite instructions, and his index file outgrew its size budget without any warning.

Benchmarks now quantify the problem. STALE contains 400 scenarios where new evidence silently overturns an earlier state. Its top-scoring model managed just 55.2%. Gemini-3.1-pro identified the current state 92% of the time on same-attribute cases but rejected questions assuming the old state only 30% of the time. The authors’ prototype, CUPMem, reached 68%.

In the TEPA full-reversal test, append-only and last-write-wins memory both scored 0.210, no memory scored 0.309, and retiring stale entries lifted the score to 0.95.

Source links don’t fix it by themselves. An August study of 16 models found agents looked back at a memory’s origin in only about one episode of five, and about three times in four they still acted on the outdated memory.

agent memory image 3
Image: newsletter.chainofthought.show

Newer isn’t always truer

Defaulting to the latest record can promote a suggestion into a decision. Suppose a register lists a vendor as approved and a later chat says “we should revisit that approval”. The right update depends on intent and on who is allowed to change the status.

For fast-changing facts, the author suggests remembering a pointer instead: where the approval register lives, to be read before acting. He separates integrity (has a record changed since it was written?) from truth (should you still believe it?), with provenance linking the two. He also argues for portability, so switching models or tooling doesn’t mean starting over.

What to do next about agent memory

He boils the work down to capture, recall and curate: save the fact, bring back the right one, and make sure a changed fact changes the answer. The first two are well covered. For anyone building agents now, the open work is curation, with review rules set by a fact’s source, how fast it changes and what a wrong answer costs.

Source: newsletter.chainofthought.show

Written by

Saksham Bhargava

Saksham Bhargava runs Alteroid and has been writing about consumer technology for over five years, covering iPhone, Android, Windows, AI tools and gaming. With an MCA and six years of hands-on web development experience, he tests the features he writes about on his own devices and focuses on clear, practical guides that solve real problems. Away from the screen, he enjoys travelling to places most tourists skip.