AI Insights · Oct 22, 2026 · Verified
Agent memory is now plain files
Every major agent stores memory as files it reads and edits, with a short index loaded each time. Vendor memory benchmarks fail independent checks by 3 to 28 points.
Claim
Every major harness stores agent memory as plain files the agent reads and edits, with a short always-loaded index and on-demand detail. Structured vector or graph memory products (Mem0, Zep, Vertex Memory Bank) publish strong numbers that independent reproductions do not match. The best-evidenced result is that raw retrieved excerpts plus extracted facts beat either alone.
Evidence
- Claude Code: CLAUDE.md hierarchy (target under 200 lines, imports to
depth 4), path-scoped
.claude/rules/*.md, auto memory with aMEMORY.mdindex of which only the first 200 lines or 25 KB load, topic files on demand; subagents can have their own memory directory. Docs stress memory is "context, not enforced configuration". - Codex: opt-in memories generated from idle threads into
~/.codex/memories/, secrets redacted; "a recall aid, not a source of rules"; rules belong in AGENTS.md. - Letta (2026-02-12): MemFS, a git-versioned memory filesystem where every change is a commit, subagents edit in worktrees and merge, and a defrag skill restructures into 15 to 25 focused files. No benchmarks.
- Anthropic memory tool: a client-side
/memoriesdirectory; paired with context editing so Claude saves before tool results are cleared. - Reproductions: Redis's LongMemEval-S run (fixed GPT-4o answer model and judge, 2026-09-21) scored Vertex Memory Bank 42.4% versus about 70% in its paper; Oracle 86.0% versus 93.8% claimed; Mem0 49%; Zep/Graphiti 71.2%; Redis's own hybrid of dense plus BM25 raw chunks plus extracted facts 86.5%, pure RAG at k=20 61.5%. Maximem measured Mem0 at 57.5% then 73.8% versus a 93.4% claim. Every party has commercial interests; the reproduction gap is the robust finding.
- LongMemEval-V2 (2026-05-12): agent-experience memory over up to 500 trajectories; a coding agent reading trajectory files scored 72.5% versus 48.5% for the best RAG, at higher latency.
Open questions
- Letta's Context Constitution principle texts were not retrievable.
Sources
- https://code.claude.com/docs/en/memory — docs, Wed Oct 21
- https://developers.openai.com/codex/memories — docs, Wed Oct 21
- https://www.letta.com/blog/context-repositories/ — blog, Wed Feb 11
- https://claude.com/blog/context-management — blog, Sun Sep 28
- https://redis.io/resources/agent-memory-benchmark-report/ — paper, Sun Sep 20
- https://arxiv.org/abs/2504.19413 — paper, Sun Apr 27
- https://arxiv.org/abs/2501.13956 — paper, Sun Jan 19
- https://arxiv.org/abs/2605.12493 — paper, Mon May 11
AI-assisted research: gathered and summarised with AI from the sources above, each claim checked against them.