Ask a production AI agent what it decided three sessions ago and watch it stall. Not because the model is incapable - because nobody gave it a memory layer that survives a session boundary. Traditional LLMs operate in a stateless paradigm: each interaction exists in isolation, with no knowledge carried forward from previous conversations. That's the default. Shipping past it requires understanding four distinct memory types, how they write differently, and what they cost at inference time.
What agent memory types actually are
Memory types describe content: episodic (events), semantic (facts), procedural (rules), and working (active processing). The taxonomy comes from cognitive science, not AI marketing. CoALA (Cognitive Architectures for Language Agents, Princeton/CMU, arXiv:2309.02427) is the canonical academic reference that adapted cognitive science memory types for LLM-based agents.
LangChain's LangGraph documentation adopts the same semantic/episodic/procedural classification for long-term memory and explicitly cites CoALA as the source of the mapping.
Here is what each type actually stores and why the distinction matters in practice:
Working memory is the context window, nothing more. Whatever fits inside the current context window constitutes the agent's working memory. Baddeley's central executive plus buffer model maps neatly: the LLM is the executive, the context window is the buffer, and both share the same bottleneck - limited capacity.
In-context memory is the context window - everything the LLM processes in a single inference: system prompt, conversation history, retrieved chunks, and tool outputs. It is the only memory the model directly reasons over. All other memory types must be retrieved into the context window to influence generation.
Episodic memory is the log of what happened. It records concrete experiences: individual tool calls, conversation turns, environment observations. In the Generative Agents work, every observation lands in the episodic stream with a timestamp, an importance score, and an embedding for later retrieval.
Its value is not merely archival. Retrieved episodes can serve as concrete precedents, help the agent avoid repeating known mistakes, and supply raw material for later abstraction. Reflexion made this pattern explicit by storing reflective summaries from failed attempts as reusable experience.
Semantic memory is what the agent knows in general, stripped of the specific event that taught it. An episodic fact like "the user corrected the date format on Jan 5, Jan 12, and Feb 1" may consolidate into the semantic record "user prefers DD/MM/YYYY." This consolidation is rarely automatic; most current systems require explicit prompting or heuristic triggers. That last sentence is the gap most teams fall into: they log episodes faithfully and then wonder why the agent keeps making the same mistake.
Procedural memory is the agent's encoded rules - the instructions, routing logic, and learned workflows that govern how it acts, not what it knows. Procedural memory encodes reusable decision logic and workflows that agents execute without re-reasoning from scratch.
How each type writes - and why the write path matters
The four types don't just store different content. They write through fundamentally different mechanisms:
Working memory is written implicitly by the act of running. Episodic memory is written automatically by logging. Semantic memory is written by an extraction step - a background process that distills episodes into facts. Procedural memory is typically written by a human or a fine-tuning pass, not by the agent at runtime.
This matters because most memory bugs are write-path bugs. An agent that logs episodes but never runs the extraction step accumulates a growing episodic store it never actually learns from. The semantic record stays empty; the agent repeats known errors.
The retrieval side has its own asymmetry. Three retrieval factors govern what gets surfaced from episodic memory: recency (how recent was it?), relevance (how semantically similar is it to the current context?), and salience (how surprising or important was it when it occurred?).
Vector similarity search is most common; temporal range queries, BM25 keyword search, and graph traversal are used by more sophisticated implementations.
Each layer has different retrieval semantics - temporal proximity matters for episodic memory, semantic relevance for knowledge, and freshness for working memory - and using the same retrieval strategy for all three is a common architectural mistake.
The token cost of getting memory wrong
This is where the architecture decisions become billing decisions. Most AI agent memory systems retrieve information by maximizing context window size. That works on benchmarks but not in production, where every token adds cost.
The numbers are concrete. Mem0's algorithm achieves competitive accuracy on LoCoMo, LongMemEval, and BEAM while averaging under 7,000 tokens per retrieval call. Full-context approaches on the same benchmarks routinely consume 25,000+ tokens per query.
The tradeoff is not free, though. Two dominant approaches exist: long-context (LC) modeling, which extends the context window to include long histories, and retrieval-augmented generation (RAG), which retrieves relevant snippets from external context. Both enable question answering over large information sources, but they differ in what they optimize: LC emphasizes end-to-end reasoning over a long input, while RAG emphasizes selecting a short, relevant subset before reasoning.
The hardest open problem is temporal reasoning - knowing that a fact changed, not just what the current value is. Biggest gains from improved memory algorithms come on temporal reasoning and multi-hop questions. This is also where current LLMs most visibly fail - the Episodic Memory Benchmark shows a stark gap between entity recall and temporal reasoning.
How production systems combine all four types
No serious agent uses one memory type in isolation. The dominant production pattern is tiered: a small always-in-context core, plus a vector-store-backed retrieval layer, plus an explicit forgetting policy.
Letta (formerly MemGPT) makes this explicit with a three-tier model. Its premise is that all agent state persists in a database even after it is evicted from the context window. Memory comes in layers: memory blocks are labeled text pinned into context that the agent can edit and share, and archival memory is a searchable database the agent queries on demand. The agent decides what to write. Letta's self-editing approach is more adaptive - the agent uses its own reasoning to curate memory - but memory quality depends entirely on the model's judgment. If the model fails to save something, it's gone. Every memory operation also costs inference tokens, since the agent has to reason about what to store and how.
Mem0 flips that tradeoff. You wrap your LLM calls with Mem0's SDK; it extracts facts from conversations, stores them in a vector store, and injects relevant memories into future prompts. The agent loop, tool execution, and orchestration stay in your code.
| Approach | Who writes memory | Storage | Strength | Watch out for |
|---|---|---|---|---|
| Letta / MemGPT | The agent itself (function calls) | Tiered blocks + archival DB | High coherence over long horizons | Token cost per memory op; model judgment dependency |
| Mem0 | Passive extraction layer | Vector store | Fast to integrate, token-efficient | Weaker on deep temporal / multi-hop |
| Zep / Graphiti | Graph extraction pipeline | Temporally-aware knowledge graph | Temporal and relational reasoning | Schema overhead; self-hosting complexity |
| Raw RAG | Human-curated ingestion | Vector store | Predictable, inspectable | No cross-session learning |
A useful frame: memory systems fundamentally transform LLMs from stateless pattern processors into sophisticated agents capable of persistent learning and adaptation across extended interactions. But only if the write path, retrieval strategy, and consolidation step all match the memory type in question. A vector store alone does not give you semantic memory. It gives you episodic retrieval. The consolidation step that distills those episodes into durable facts - "user prefers DD/MM/YYYY" - is what builds the semantic layer, and it usually has to be designed explicitly.
An agent like Beagle in Slack handles this concretely: the question a teammate asks on Tuesday, the source it cites, the correction they make on Thursday - those are episodic events. The preference that emerges ("always link the primary source, not the summary doc") is semantic. Getting that consolidation right is what separates an agent that feels useful in week one from one that feels useful in month six.
Beagle in action#product, Tuesday 10:22amAgent memory types: common questions
What is the difference between episodic and semantic memory in AI agents?
Episodic memory stores specific events - a tool call at 3pm, a user correction on Tuesday - tied to time and context. Semantic memory stores what tends to be true across events: a distilled fact, preference, or rule. The practical difference is that episodic memory is written by logging; semantic memory requires an explicit extraction step that most systems do not run automatically.
How does agent memory actually persist across sessions?
Persistence requires external storage - a vector database, knowledge graph, or structured database - outside the LLM's context window. At the start of a new session, relevant memories are retrieved and injected back into context. Without this retrieval step, nothing survives a session boundary regardless of what was "saved."
Does a bigger context window replace the need for agent memory?
Not in production. Full-context approaches that stuff entire conversation histories into the window routinely consume 25,000+ tokens per query. Selective retrieval can reach comparable accuracy under 7,000 tokens per query. The cost difference compounds at scale. A larger window also cannot consolidate episodes into semantic facts - it just holds more raw history.
What is procedural memory in an LLM agent?
Procedural memory holds the agent's instructions, routing rules, and learned workflows - the "how to act" layer rather than the "what happened" or "what is true" layers. In Letta 0.5+, it's a first-class memory block the agent can update. In simpler systems, it's just the system prompt. The key property: the agent executes from procedural memory without re-reasoning the decision from scratch each time.
What is the hardest problem in agent memory right now?
Temporal reasoning - tracking how facts change over time rather than just what the current value is. Current benchmarks show this is where retrieval systems underperform most. Cross-session identity (knowing that "the user" in session 1 is the same entity as in session 40) and memory staleness (knowing a stored fact is now wrong) are the other two unsolved problems that matter most in production deployments.