The agents we build today no longer fit in a single LLM call: multi-step research, coding agents, assistants that persist across sessions.
The "Claude Code" / DeepAgents style — planning, sub-agents, virtual filesystems — has become the reference pattern for "long-running" agents in 2025-2026.
Our data pipelines (ingestion, exploration, automated reporting) increasingly look like these agents: the question "where does the state live, and for how long?" becomes central.
Goal of this session: give you a precise, shared vocabulary for talking about agent memory, before we dive into code in session 2.
The two tools we'll use
LangGraph & DeepAgents, in one minute
LangGraph
LangChain's agent orchestration framework: an agent is modeled as a state graph (nodes = steps, edges = transitions), with native support for persisting that state.
DeepAgents
A "batteries-included" agent harness built on top of LangGraph, explicitly inspired by Claude Code's behavior: planning, sub-agents, virtual filesystem — shipped by default rather than something you code yourself.
Why this matters for this talk: both treat memory as a first-class citizen — that's why they run as a thread through all of Session 1, even before we revisit them in detail at the end.
The central problem
The context window is finite
Everything an LLM "knows" at a given instant t fits inside a fixed token budget, shared between:
system prompt + tool schemas + conversation history + retrieved documents ≤ N tokens
N = the context window size of the model in use — fixed for a given model (e.g. 200k tokens for Claude), but never infinite.
The longer the conversation / task runs, the more this budget fills up — you have to choose what to keep.
And it isn't just a matter of space: the "Context Rot" study by Chroma (2025) shows that response quality across 18 frontier models degrades with context length, sometimes well before the nominal limit — even on tasks that don't require "remembering everything".
"Stuffing everything into the context" is not a free memory strategy: it's a choice with a quality cost, not just a latency/$ cost.
Definition
What is agent "memory"?
An agent's memory is any mechanism that lets information persist beyond a single pass through the model's context — and brings it back at the right moment.
It is not the model's weights (that's what it learned during training — implicit, not modifiable at inference time).
It's a matter of software engineering: where to store, how to index, when to write, when to re-read, when to forget.
The academic field has a name for the reference architecture: CoALA.
Reference framework
CoALA — Cognitive Architectures for Language Agents
It's this retrieval ↔ learning cycle around a limited working memory that is, ultimately, "the" engineering problem of agent memory.
Retrieval
How memory gets retrieved
Rule-based: explicit filters (by user, by date, by type) — simple, predictable, but rigid.
Sparse search: keywords / BM25, which combines three signals:
term-frequency saturation — a word repeated 10× doesn't count 10× more,
IDF (a term's overall rarity in the corpus) — the rarer a word, the more informative it is,
document-length normalization — so long texts aren't favored.
Result: good exact recall, weak on meaning.
Dense search (embeddings): each memory is encoded as a vector at write time, the query as a vector at read time, then ranked by similarity. This is today's dominant mechanism — it's RAG applied to the agent's own past, not just to external documents.
A bit of math (1/3)
Cosine similarity
The one formula you really need to understand dense search: it measures the angle between two vectors, not their magnitude.
cos(u, v) = (u · v) / (‖u‖ · ‖v‖)
u, v = the embedding vectors of the memory and the query.
u · v = their dot product.
‖u‖ = the vector's norm (length).
Result between -1 and 1 (in practice, often between 0 and 1 for text embeddings).
1 = same direction (same meaning); 0 = no relation.
Each memory is embedded once, at write time — only the query is re-embedded at each read.
That's all the math you need to remember for 90% of vector memory systems in production.
Each memory is a timestamped event in a memory stream. To decide what to recall, three signals are combined:
score = αr·recency + αi·importance + αv·relevance
Recency — how recently the memory was accessed (details next slide).
Importance — the significance the LLM itself assigned to the memory at write time (details next slide).
Relevance — the cosine similarity between the memory and the current situation (previous slide).
(in the paper, all three α = 1; each term is min-max normalized to [0,1] before summing, so none dominates purely by scale)
A bit of math (3/3)
Recency: an exponential decay
recency = γΔt (γ = 0.995, Δt in "hours" since last access)
A memory not accessed for ~140 simulated "hours" has already lost about half its weight.
Every read of a memory resets its clock to zero — like an LRU cache.
Importance & relevance
Importance: rated 1–10 at write time, by literally asking the LLM: "how significant is this event?" ("brushing one's teeth" → 2, "a breakup" → 8).
Relevance: cosine similarity (previous slide) between the memory and the current situation.
Consolidation
Reflection: from episodic to semantic
Without consolidation, long-term memory is just a raw log that keeps growing forever — increasingly unusable.
Reflection (Generative Agents): triggered when the sum of recent importance scores exceeds a threshold (150 in the paper). The LLM receives the ~100 most recent events and generates 3 high-level questions, then synthesizes insights that cite their sources.
Result: a tree of reflections — the leaves are raw observations, the internal nodes are increasingly general abstractions (reflections can reflect on other reflections).
This is the direct conceptual ancestor of the "memory consolidation" / hierarchical summarization found in nearly every modern framework (LangGraph included, session 2).
Since the 2023-2025 papers cited above
Temporal knowledge graphs: memory that invalidates, not overwrites
Rasmussen et al. (Zep), A Temporal Knowledge Graph Architecture for Agent Memory, 2025 — arXiv:2501.13956
Reflection (previous slide) never answers: what happens when an old insight turns out to be wrong? A flat vector store just leaves the stale fact sitting there, competing on cosine similarity with the correction.
Graphiti (Zep's open-source engine) stores memory as a graph where every edge carries a (valid_at, invalid_at) pair — old facts are marked invalid with provenance, never silently deleted or silently contradicted.
Maps onto CoALA cleanly: episodic nodes ≈ episodic memory, entities/facts ≈ semantic memory — the new part is the bi-temporal validity CoALA doesn't model.
Benchmark caveat: LoCoMo leaderboard numbers for Mem0/Zep/others are self-reported under different eval setups and are not directly comparable — treat any single "SOTA" claim in this space with suspicion.
What fits in the model's window: system instructions, a window of recent messages, and blocks of "core memory" editable by the model itself.
External context (≈ disk)
Everything else, outside the window: archival memory (archived memory, facts/documents searched via embeddings) and recall memory (complete raw, searchable history).
MemGPT — mechanics
The model manages its own memory, via tools
The LLM decides on its own, through explicit function calls, what should stay in RAM or be paged out: core_memory_append, core_memory_replace, archival_memory_search, archival_memory_insert, conversation_search.
Eviction isn't a background daemon: when main context approaches its token budget, the system injects a memory-pressure warning into the conversation itself, and the LLM decides what to evict via a function call — self-regulation, not garbage collection.
MemGPT → Letta (2026): the name "MemGPT" today mostly refers to the design pattern; Letta is the maintained open-source framework/product (letta-ai/letta). 2026 addition: "Context Repositories", a git-like versioned memory for coding agents.
Comparison
Five families, one spectrum
Approach
Complexity
Latency/cost
Reliability
Good for
Raw context (stuff everything)
None
Grows with history
Degrades (Context Rot)
Very short prototypes
Memory RAG (embeddings)
Medium
Low per turn
Good, but recall silently incomplete
Multi-session personalization
MemGPT / Letta (OS-style paging)
High
Extra tool calls
Depends on the model's self-discipline
Long and multi-session runs
LangGraph Store
Low if already on LangGraph
Cheap lookup
High, explicit, testable
Per-user/org personalization
DeepAgents (filesystem + sub-agents)
Medium (opinionated harness)
More LLM calls, but smaller parent context
Strong for long single-session tasks
"Long-task" agents (research, code)
Anthropic memory tool (client-owned files)
Low
Extra tool calls, cheap
High — you own storage & validation
Multi-session agent progress, without adopting a framework
These approaches combine more than they compete — DeepAgents, for example, builds directly on LangGraph's Store (session 2).
The bridge to modern practice
"Context engineering": Anthropic's vocabulary
Anthropic formalizes context as a finite resource with diminishing returns — and names 4 techniques that show up, productized, in DeepAgents.
"Effective context engineering for AI agents" — anthropic.com/engineering
The 4 context engineering techniques
1. Compaction
Summarize a conversation approaching the context limit, then continue with the summary (+ the most recently accessed files).
2. Structured note-taking
A persistent notes file outside the context window, to survive a reset and pick back up where things left off.
3. Multi-agent architectures
An "orchestrator" agent synthesizes; specialized sub-agents work with their own, clean, isolated context window.
4. Just-in-time retrieval
Keep lightweight references (paths, identifiers) rather than the full content, and only load it when it's actually needed.
Anthropic, concretely
The memory tool: a real mechanism, not just a name
Since September 2025, Anthropic ships a concrete answer to "structured note-taking": a memory tool Claude can call directly, no framework required.
Six commands over /memories
view, create, str_replace, insert, delete, rename — the same primitives as a text editor, scoped to one directory.
Client-side by design
Claude only requests file operations; your application executes them against storage you own (disk, DB, S3 — SDKs ship a local-filesystem helper). Anthropic never sees or stores the files.
Measured impact (Anthropic's own eval): +29% from context editing alone, +39% combined with the memory tool, and an 84% token reduction in a 100-turn web-search task — this is the same "compaction + notes outside the window" idea from two slides ago, now with a number attached.
Memory tool vs. Claude.ai memory vs. ChatGPT "Dreaming"
What
Who stores it
Who decides what's written
For
API memory tool
You (client-side)
Claude requests, your app can validate/reject
Engineers building agents
Claude.ai "Memory" (Topics)
Anthropic-hosted
Automatic, user can view/edit/delete per topic
End users chatting with Claude
ChatGPT "Dreaming" (OpenAI, 2026)
OpenAI-hosted
Fully automatic — rewrites entries without being asked
End users, less transparent by design
Anthropic's explicit design choice is the opposite of "Dreaming": Claude tells you when it uses a memory and asks before writing sensitive information. That's a deliberate transparency trade-off, not a capability gap.
Why it works
It's not just intuition — it's measured
The Context Rot study (Chroma, 2025), across 18 frontier models, shows:
It's not only about length (as noted in the intro): how the context is organized matters too.
On certain tasks, models do better on a long but disordered context than on a long, logically coherent one — a sign that "having everything at hand" isn't cognitively neutral for the model.
Practical conclusion: compacting, isolating, and retrieving on demand isn't a cost optimization — it's also a quality optimization.
Transition
Where LangGraph and DeepAgents fit
DeepAgents is not a competitor to LangGraph: it's an opinionated layer on top of it — its own contribution is the filesystem metaphor + sub-agents, not a new persistence mechanism.
Summary
What to remember before session 2
Agent memory = an engineering problem around a finite context window, not an implementation detail.
CoALA taxonomy: working memory (active, volatile) vs long-term memory (episodic / semantic / procedural).
Retrieval = search (often via embeddings/cosine), possibly weighted by recency + importance + relevance.
Without consolidation (reflection), long-term memory becomes an unusable raw log.
Two dominant implementation philosophies in 2026: MemGPT/Letta (the model manages its own memory via tools) and LangGraph/DeepAgents (the engineer explicitly structures checkpointer / Store / filesystem / sub-agents).
Glossary (1/2)
Agent / LLM vocabulary
Term
Definition
Framework
A software library that imposes structure: you write your code inside the rules it defines — as opposed to a plain library you simply call from your own code.
Harness (agent harness)
The code wrapped around an LLM to turn it into a full agent: the decision loop, tool handling, memory and context management — the LLM alone doesn't know how to loop or call tools by itself.
Opinionated
Imposing default design choices rather than leaving everything configurable (the opposite of "unopinionated"/neutral) — a trade-off between getting started fast and staying flexible.
Batteries-included
Shipped with everything needed to be useful right away, without having to assemble third-party pieces yourself (a phrase borrowed from Python: "batteries included").
Model-agnostic
Works with any underlying LLM (Claude, GPT, ...) without being tightly coupled to one vendor.
Token
The unit of text an LLM processes (roughly a word or word-fragment) — it's the unit the context window is measured in.
Embedding
A numeric vector representing the "meaning" of a piece of text, produced by a dedicated model; texts that are close in meaning have vectors that are close together (see cosine similarity).
RAG
Retrieval-Augmented Generation: injecting relevant documents fetched dynamically into the prompt, instead of relying only on what the model memorized during training.
Glossary (2/2)
LangGraph / DeepAgents vocabulary
Term
Definition
Checkpointer
The LangGraph component that saves a graph's state at every step so it can be resumed later (= short-term memory).
In DeepAgents, the system that decides where virtual files actually live (ephemeral state, a persistent Store, disk...); more generally, the part of a system handling storage/processing, invisible from the user-facing side.
Grounding
An agent acting on / perceiving the real world (tool calls, environment), as opposed to pure internal reasoning.
Context engineering
The discipline (named by Anthropic) of actively managing what goes into an LLM's context window, rather than dumping everything available into it.
Offloading
Moving information out of the active context window (e.g. to a file) to free up token budget, to be re-read later only if actually needed.
Sub-agent (quarantine)
A helper agent running in its own isolated context window; only its final result "crosses over" to the parent agent — intermediate noise never pollutes the parent's context (hence "quarantine").
Bibliography (1/2)
Academic foundations
Sumers, Yao, Narasimhan & Griffiths, Cognitive Architectures for Language Agents (CoALA), TMLR 2024 — arXiv:2309.02427
Park, O'Brien, Cai, Morris, Liang & Bernstein, Generative Agents: Interactive Simulacra of Human Behavior, UIST 2023 — arXiv:2304.03442 · ACM DOI 10.1145/3586183.3606763
Packer, Fang, Patil, Lin, Wooders & Gonzalez, MemGPT: Towards LLMs as Operating Systems, 2023 — arXiv:2310.08560
Zhang, Bo, Ma, Li, Chen, Dai, Zhu, Dong & Wen, A Survey on the Memory Mechanism of Large Language Model based Agents, ACM TOIS 2025 — arXiv:2404.13501 · DOI 10.1145/3748302
Wu, Liang, Zhang, Wang, Zhang, Guo, Tang & Liu, From Human Memory to AI Memory: A Survey on Memory Mechanisms in the Era of LLMs, 2025 — arXiv:2504.15965
Du, Memory for Autonomous LLM Agents: Mechanisms, Evaluation, and Emerging Frontiers, 2026 — arXiv:2603.07670
Chroma Research, Context Rot: How Increasing Input Tokens Impacts LLM Performance, 2025 — trychroma.com/research/context-rot