agents/prod
06
Memory · state · RAG

Retrieval latency shapes the design

Reported  This dimension was under-covered in final verification — treat as directional. Memory is optimized for fast reads (written once, read many). Latency by strategy:

memory retrieval latency · lower is faster
vector-only
10–50 ms
graph traversal
50–150 ms
multi-strategy
100–600 ms
LLM synthesis
800–3000 ms
  • Tiers: short-term/working (thread-scoped) + long-term (cross-session), plus episodic, semantic, procedural.
  • ≥8 dedicated memory frameworks compete in 2026: Mem0, Letta, Zep/Graphiti, Cognee, SuperMemory, LangMem, LlamaIndex Memory, Hindsight.
  • Index tradeoff at scale: HNSW = higher recall per latency but several× more memory; IVF = more memory-efficient at billions of vectors. Both tunable >95% recall.
  • Big context ≠ no memory: windows reset per request and can't preserve cross-session identity or learned behavior.