agents/prod
10
Foundations · Chapter 10

Memory & state — continuity for a stateless model

A language model forgets everything the instant a call ends — its context window resets every request (Chapter 1). Yet the agents we actually want remember your preferences, resume yesterday's task, and learn from feedback. That continuity has to be engineered outside the model. This is agent memory — the basis of personalization and long-horizon work.

1 · Short-term vs long-term

Short-term (working) memory is the live context window — the thread-scoped conversation, externalized to a database via a checkpointer so a thread can pause/resume while the runtime stays stateless. Long-term memory lives outside the model — vector store, graph, key-value — and persists across sessions, organized by namespace (usually a user or org ID).

MODELstateless per call short-termthe context window long-term memory — vector DB / graph / KV (across sessions)namespaced by user/org · survives restarts
The model stays stateless; the platform holds the state — a small live window plus a large external store.

2 · Three kinds of long-term memory

Borrowed from cognitive science and now standard in frameworks like LangChain:

Episodic
specific past events — "last Tuesday you asked about refunds"
Semantic
durable facts & preferences — "prefers metric units"
Procedural
learned skills — often the system prompt itself

3 · The write path and the read path

Memory has two pipelines. The write path (often async, off the hot path) extracts structured facts from a conversation and embeds them. The read path embeds the query, searches, reranks, and injects the top results. Step the read path and watch the latency add up:

Interactive · the read path, stage by stage
A measured read path: embed the query (~2ms) → vector top-50 search (~3ms) → rerank (~20ms) → inject the top 5 into context. Colocating the memory store on the same node cut this ~90% — from 100–400ms over a network to 25–30ms local.

4 · The three architectures worth knowing

Different systems make opposite bets on when to do the work. Switch between them:

Interactive · compare memory architectures
Carry into Chapter 11

You now have the full toolkit — loop, reasoning, context, RAG, serving, scaling, isolation, memory. Next: how real companies assemble these primitives into agents that ship, and what happened when they did.