Agent memory is the set of mechanisms that let an LLM-based agent persist, retrieve, and act on information beyond a single context window — enabling…
Trace the gap between a stateless LLM call and a memory-enabled agent, and pin the exact definition: agent memory is persisted information that changes agent behavior after the current prompt is gone. The running scenario throughout this lesson is a coding-assistant agent that helps a developer across multiple sessions — you'll see how each memory layer adds capability to that agent.
Defines agent memory as persisted information that changes agent behavior across sessions, and contrasts it with a context window, transcript, and vector database.
Why this matters: Every multi-session AI feature you build — including the coding assistant running through this lesson — depends on getting this definition right before adding any memory layer.
Your coding assistant just helped a developer refactor a module, agreed on a naming convention, and learned the project uses Python 3.11. Next session: it remembers none of it.
Every LLM call is stateless by default — the model receives a prompt, produces a response, and discards everything. The (the model's working memory for one turn) disappears the moment the call ends.
The continuity problem: a multi-session agent that can't carry facts forward forces the user to re-explain their project on every visit. That's not an assistant — it's a very expensive search box.
is persisted information that changes how an agent behaves after the current prompt is gone.
That definition rules out three things people often confuse it with:
The key word is persisted: the information survives the current session and is retrieved in a later one to alter what the agent says or does.
The useful split is two scopes: for the current thread, and for information that should affect later sessions.
Long-term memory itself has sub-types: (past events, e.g. "last Tuesday the user asked about auth"), (facts, e.g. "project uses Python 3.11"), and (learned behaviors, e.g. "always suggest type hints").
Session 1: a developer tells the coding assistant "We use snake_case everywhere and target Python 3.11." The assistant refactors a module. The session ends.
Session 2 (no memory): the developer asks "Can you review this new file?" The assistant suggests camelCase and Python 3.9 syntax. It has no record of the earlier agreement.
Session 2 (with memory): on startup the agent retrieves the stored fact "snake_case, Python 3.11" from long-term memory. It injects this into the context window. The review is consistent with the project's actual conventions. No re-explanation needed.
The delta is exactly the definition: persisted information changed agent behavior after the original prompt was gone.
# Naive: no memory — just call the LLM each session def coding_assistant(user_message: str) -> str: response = llm.chat([ {"role": "system", "content": "You are a coding assistant."}, {"role": "user", "content": user_message}, ]) return response.content # Session 2 — user expects the agent to know their conventions print(coding_assistant("Review this file for style issues."))
This is the obvious-but-wrong starting point: a single LLM call with no stored state. The system prompt has no project context, so the agent invents its own style rules.
The assistant suggests generic style fixes — possibly camelCase, Python 3.9 f-strings, or other defaults — because it has zero knowledge of the project's snake_case / Python 3.11 conventions. No error is raised; the output looks plausible but is silently wrong. This is the continuity failure: the model produces confident, incorrect advice because the relevant facts were never persisted.
memory_store = {} # stands in for a real persistent store
def save_memory(key: str, value: str):
memory_store[key] = value
def load_memory(key: str) -> str:
return memory_store.get(key, "")
# Session 1 — agent learns the convention and saves it
save_memory("style", "snake_case, Python 3.11")
# Session 2 — agent retrieves it before calling the LLM
style_rule = load_memory("style")
response = llm.chat([
{"role": "system", "content": f"Project rules: {style_rule}"},
{"role": "user", "content": "Review this file for style issues."},
])Now the agent writes a fact to an external store at the end of Session 1 and reads it back at the start of Session 2. The retrieved fact is injected into the system prompt, so the LLM conditions on it without the user repeating themselves.
The system prompt becomes: "Project rules: snake_case, Python 3.11". The LLM now flags camelCase as a violation and avoids 3.9-only syntax — consistent with the project. The persisted fact changed agent behavior after the original prompt was gone, which is the definition of agent memory.
from datetime import date # Existing helpers: save_memory / load_memory from Stage 2 def record_session_event(summary: str): # TODO: save this event under a key like "session:<today's date>" # so the agent can later recall what was discussed on a given day. pass # After Session 1 ends: record_session_event("Refactored auth module; agreed on snake_case.") # Session 2 — retrieve yesterday's event yesterday = str(date.today()) # same day in this demo print(load_memory(f"session:{yesterday}"))
Stop — attempt the TODO before revealing the answer. The missing line is the crux of episodic memory: storing a dated event so the agent can recall what happened in a past session.
Hints: use save_memory from Stage 2; the key should embed today's date so events are retrievable by day.
Changed line: save_memory(f"session:{date.today()}", summary)
The print outputs: "Refactored auth module; agreed on snake_case."
Why it matters: the key session:<date> namespaces events by day, giving the agent episodic memory — it can now answer 'What did we work on last Tuesday?' by loading that key. This is the difference from Stage 2's semantic fact: episodic memory records when something happened, not just what is true.
The next module examines the context window itself. It covers token budget, what it holds, and how to manage it as the agent's working memory for a single session.
Examine how the context window functions as working memory: what it holds (conversation history, tool results, retrieved evidence), its token-budget constraints, and the compaction patterns — summarization and sliding-window truncation — that keep it from overflowing. Apply this to the coding-assistant scenario: the agent must fit the current file, recent tool calls, and user instructions into a single prompt.
Explains how the context window acts as an agent's working memory, what it holds, and how to apply compaction patterns when it overflows.
Why this matters: Every agent you build hits a token-budget ceiling; knowing how to compact the context correctly keeps the agent coherent and cost-efficient.
Decision this forces: When should the agent summarize vs. truncate conversation history to stay within the token budget?
Answer: is persisted information that changes agent behavior after the current prompt is gone.
That definition draws the key boundary: the prompt itself is not durable memory — it vanishes the moment the turn ends.
This module zooms into what lives inside that prompt during a turn: the , how it fills up, and what you do when it overflows.
The is the entire text the model reads on a single turn — system prompt, conversation history, tool results, retrieved evidence, and the new user message, all concatenated into one long string.
It acts as : rich and immediately usable, but strictly scoped to the current turn. Once the response is sent, nothing in the window persists unless your application explicitly saves it somewhere.
Every element in the window costs tokens, and the model's is fixed by the provider (e.g. 128 k tokens for GPT-4o). Longer contexts also cost more per call and add latency — so a large window is not a free lunch.
Your coding assistant opens a 400-line Python file, runs a linter tool, and receives the user's third follow-up question — all in one turn.
Here is a rough token count for that turn:
The agent has no automatic way to decide what to drop. Without a strategy, the application either crashes with a context-length error or silently truncates from the top — losing the system prompt.
def build_prompt(history: list[dict], max_tokens: int = 4096) -> list[dict]: # Naive: keep only the last 10 messages, no token counting return history[-10:] # --- what happens in practice --- # history[0] = {"role": "system", "content": "You are a coding assistant..."} # history[-10:] may NOT include index 0 # → system prompt is silently dropped # → model loses its persona, tool schemas, and safety rules # → responses become generic or refuse tool calls
Slicing by message count looks safe but ignores token size — and it can drop the system prompt entirely.
The model doesn't error; it just behaves strangely, making this one of the hardest failure modes to spot in production.
history[-10:] returns messages 10–19. The system prompt at index 0 is not included. The model receives no persona, no tool schemas, and no rules — it silently operates without them, producing subtly wrong or generic output with no error raised.
def build_prompt(history: list[dict], max_tokens: int = 4096) -> list[dict]: system = [m for m in history if m["role"] == "system"] # always keep turns = [m for m in history if m["role"] != "system"] budget = max_tokens - sum(count_tokens(m) for m in system) kept = [] for msg in reversed(turns): # walk newest → oldest cost = count_tokens(msg) if budget - cost < 0: break kept.append(msg) budget -= cost return system + list(reversed(kept))
Pin the system prompt first, then fill the remaining budget with the most recent turns — oldest ones fall off naturally when the budget runs out.
This fixes the silent-drop bug from Stage 1: the system prompt is always present, and the cutoff is driven by actual token counts, not message count.
budget is the number of tokens still available after fitting the kept turns. kept was built newest-first (reversed iteration), so reversing it again restores chronological order — oldest kept turn first — which is the order the model expects to read conversation history.
def compact_with_summary(history, max_tokens, summarize_fn, threshold=0.8): system = [m for m in history if m["role"] == "system"] turns = [m for m in history if m["role"] != "system"] used = sum(count_tokens(m) for m in history) if used < max_tokens * threshold: return history # still within budget — no action mid = len(turns) // 2 old_turns = turns[:mid] recent = turns[mid:] # TODO: replace the next line with a call to summarize_fn summary_msg = ??? return system + [summary_msg] + recent
When total token use crosses 80 % of the budget, the older half of the conversation is compressed into a single summary message.
Your task: replace the TODO so the function calls summarize_fn on the old turns and wraps the result as a {"role": "system", "content": ...} dict.
summary_msg = {"role": "system", "content": summarize_fn(old_turns)}
Changed lines vs Stage 2: instead of dropping old_turns entirely, you call summarize_fn on them and store the result as a system message. This preserves key facts (user's original requirements, earlier decisions) at a fraction of the token cost. Watch for hallucination risk: summarize_fn may drop or distort details — always verify the summary includes the facts the agent still needs (file names, error codes, user constraints).
| Option | Information preserved | Latency added | Implementation effort | When to choose | Cost | Complexity |
|---|---|---|---|---|---|---|
| Sliding-window truncation | Only the most recent N turns survive; older context is gone permanently | None — pure in-memory slice | Trivial — a list slice or deque | Use when recent turns are all that matter and older turns are truly disposable — e.g. a short Q&A session with no long-running state. | No extra LLM call | Low — drop the N oldest messages |
| Summarization compaction | Key facts distilled into a compact summary that stays in the prompt | One extra round-trip to the model before the main call | Moderate — prompt the model to summarize, replace old turns with the summary | Use when earlier turns contain facts the agent still needs — e.g. a multi-file refactor where the user's original requirements must stay in scope. | One extra LLM call per compaction event | Medium — requires a summarization LLM call |
count_tokens(result) <= max_tokens.summarize_fn is called only when needed — not on every turn — to avoid unnecessary latency and cost.The next module — Episodic, Semantic, and Procedural Memory — shows how to move facts that matter across sessions out of the context window entirely, so compaction doesn't have to carry them.
Map the three memory types onto the coding-assistant scenario: episodic memory stores past debugging sessions (events with timestamps), semantic memory stores project facts like preferred libraries and API keys, and procedural memory stores how-to workflows like the agent's code-review checklist. Revisit the short-term/long-term distinction from Module 1 and show which type lives where.
Maps the three types of long-term agent memory — episodic, semantic, and procedural — and shows how to classify, store, and retrieve each.
Why this matters: Choosing the wrong memory type causes the agent to lose preferences, surface stale context, or run outdated workflows — getting this right is the foundation of reliable agent behavior.
context windowtoken budgetcompaction discards detail to make room.
long-term memory outside the window. This module maps the three types that fill that gap.
Every piece of information an agent should remember falls into one of three types. Each answers a different question about what to keep.
Each type lives in a different store and is retrieved differently. Getting it wrong means the agent can't find the memory or retrieves stale data.
Your coding assistant agent needs to remember three very different things — and each belongs in a different memory type.
Notice the pattern: episodic answers "when did this happen?", semantic answers "what is always true here?", and procedural answers "what steps do I follow?". A single agent session may read from all three simultaneously.
procedural memory shape how the agent behaves, but they differ in mutability and scope.
Use a system prompt for invariant agent identity ("you are a coding assistant"). Use procedural memory for workflows that evolve — like a code-review checklist that the team refines over time.
# Three typed stores — plain dicts stand in for real persistence episodic_store = [] # list of events, ordered by time semantic_store = {} # key-value facts procedural_store = {} # named workflows def remember(memory_type, key, value, timestamp=None): if memory_type == "episodic": episodic_store.append({"event": value, "at": timestamp}) elif memory_type == "semantic": semantic_store[key] = value elif memory_type == "procedural": procedural_store[key] = value # value is a list of steps
This single remember() dispatcher routes each piece of information to the right store based on its type. Episodic entries carry a timestamp; semantic entries are keyed facts; procedural entries are named step-lists.
[{"event": "KeyError on line 42", "at": "2024-06-10T14:32"}] — one dict appended to the list. The key argument is ignored for episodic entries because events are identified by position and timestamp, not a named key.
from datetime import datetime def recall(memory_type, key=None, n_recent=3): if memory_type == "episodic": # Return the n most-recent events (recency weighting) return sorted(episodic_store, key=lambda e: e["at"], reverse=True)[:n_recent] elif memory_type == "semantic": return semantic_store.get(key) elif memory_type == "procedural": return procedural_store.get(key)
recency weighting) and returns the N most recent events; semantic and procedural memory use exact key lookup. This delta from Stage 1 shows that the store shape drives the retrieval pattern.
The two most recent events — June 10 and June 9 — in that order. June 8 is dropped because n_recent=2 caps the result. The sort is descending by timestamp, so the newest event is always first.
# New requirement: the user says "I always prefer type hints in my Python." # The agent must classify this and store it correctly. user_statement = "I always prefer type hints in my Python." # Stop — attempt this before revealing: # Which memory_type and key should you pass to remember()? # Hint 1: Is this a timestamped event, a stable fact, or a workflow? # Hint 2: What key name makes it easy to retrieve later? remember( memory_type=___, # TODO: fill in the correct type key=___, # TODO: choose a descriptive key value=user_statement )
This is a variation of the worked example: a user preference, not a past event or a workflow. Supplying the right memory_type and key is the crux — the rest of the call is already written.
memory_type="semantic", key="coding_style_type_hints". Changed lines: memory_type (was ___) → "semantic" because a coding-style preference is a stable fact with no timestamp; key (was ___) → "coding_style_type_hints" so the agent can retrieve it by name. Storing it as episodic would bury it under recency sorting; storing it as procedural would imply it's a workflow to execute.
Check that every episodic write includes a timestamp field. Grep for remember("episodic" and confirm timestamp= is never None.Run a retrieval smoke test: store one item of each type, call recall() for each, and assert the returned value matches what was stored.embedding vectorsvector database. This enables retrieval by meaning, not just by key — making large memory stores actually searchable.
Show how text memories are converted to embedding vectors, stored in a vector database (using Chroma or Pinecone as concrete examples), and indexed for similarity search. Walk through a worked code pattern where the coding-assistant agent writes a new episodic memory (a past bug fix) to a vector store and reads it back. Cover provenance metadata — tagging each memory with its source (user statement, tool result, inferred pattern) — as a first-class design requirement.
Shows how text memories are converted to embedding vectors, stored in a vector database with provenance metadata, and retrieved by semantic similarity.
Why this matters: Gives you the concrete write-and-read code pattern your agent needs to persist episodic and semantic memories across sessions — the foundation for everything retrieval-related in the next module.
Decision this forces: When should long-term memory live in a vector store vs. a relational database vs. the application's own records?
stores timestamped events: a specific bug fix, a failed test run, a user correction. stores facts that don't expire: preferred libraries, API keys, project conventions. This module shows how both get written to disk and retrieved later. and a make this possible.
The driving question: once a memory exists as text, how does the agent store it for later retrieval — even when the wording at query time is completely different?
An is a fixed-length list of numbers (typically 384–1536 floats) that an embedding model assigns to text. Semantically related texts cluster close together in that high-dimensional space.
Closeness is measured with — the cosine of the angle between two vectors, ranging from –1 to 1. A score above ~0.85 usually means the texts are about the same thing, regardless of exact wording.
This is why can match "fixed the off-by-one error in the parser" to "parsing bug" — the vectors are close even though the words differ.
The embedding model is separate from the LLM. You call it once per memory at write time, and once per query at read time.
Your coding-assistant agent just fixed a null-pointer crash in the CSV parser. You want that episode stored so the agent can recall it next time — even in a fresh session.
The write path has three steps: (1) embed the memory text into a vector, (2) upsert the vector plus metadata into the store. The metadata — called — records where the memory came from: a user statement, a tool result, or an inferred pattern.
The read path mirrors it: embed the query, ask the store for the top-k nearest vectors, inject those memory texts into the prompt. The agent never sees raw vectors — only the original text and its metadata.
Provenance is a first-class design requirement, not an afterthought. Without it, the agent can't judge whether a memory is a verified tool result or a user guess — and that distinction changes how much weight to give it.
import uuid def embed(text): ... # calls your embedding model, returns list[float] memory_text = "Fixed off-by-one in CSV parser: fence-post error on last row." vector = embed(memory_text) memory_id = str(uuid.uuid4()) store.upsert( id=memory_id, vector=vector, text=memory_text, metadata={"source": "tool_result", "session": "2024-06-01", "user": "ada"}, )
This is the write path: embed the memory text once, then upsert it with a stable ID and provenance metadata. The source field is the provenance tag — here "tool_result" because the fix came from a verified tool run, not a user assertion.
The store holds BOTH: the vector (for similarity search) and the original text plus metadata (returned at retrieval time). Upserting the same ID overwrites the previous entry — that's the 'upsert' contract. This is intentional: if the agent refines a memory (e.g. the bug fix was incomplete), re-upserting the same ID keeps the store clean without duplicates.
query = "user is seeing a parsing crash on the last line" query_vector = embed(query) results = store.query( vector=query_vector, top_k=3, filter={"user": "ada"}, ) for r in results: print(r["score"], r["metadata"]["source"], r["text"])
The query is embedded with the same model used at write time — mismatched models produce garbage similarity scores. The filter narrows the search to memories tagged for this user before cosine similarity is computed, keeping results relevant and private.
Yes — it should appear. The embedding model maps both texts to nearby vectors because they describe the same concept (a parser boundary bug), so cosine similarity will be high (~0.82–0.92 depending on the model). This is exactly what semantic search buys over keyword search: related meaning, not matching words.
# Scenario: the agent inferred (not observed) that ada prefers pytest over unittest. # Store this semantic memory with correct provenance. inferred_text = "Ada prefers pytest; avoids unittest based on past sessions." vector = embed(inferred_text) store.upsert( id="pref-ada-pytest", vector=vector, text=inferred_text, metadata={"source": _____, "user": "ada", "session": "2024-06-01"}, )
Stop — attempt this before revealing. Fill in the source value. Hint 1: this memory was not stated by the user and did not come from a tool call. Hint 2: the provenance vocabulary from the scenario block listed three source types.
The blank is: "inferred_pattern"
Changed line vs. Stage 1:
metadata={"source": "inferred_pattern", ...} # was "tool_result"
Why it matters: at retrieval time the agent can weight "tool_result" memories more heavily than "inferred_pattern" ones, because inferences can be wrong. Tagging the source lets the agent — or a downstream reviewer — apply appropriate skepticism.
| Option | Query style | Schema flexibility | Retrieval precision | When to choose | Cost | Complexity |
|---|---|---|---|---|---|---|
| Vector Store (Chroma / Pinecone) | Semantic / similarity — finds related meaning, not exact matches | Schemaless; metadata fields vary per memory type | Approximate — may surface near-misses; no exact-match guarantee | When the agent needs to find semantically related memories using natural-language queries — past bug fixes, user preferences, inferred patterns. | Higher per-query compute (embedding + ANN search); manageable at agent scale | Low–medium setup; no schema migrations |
| Relational Database (Postgres / SQLite) | Exact / structured — SQL filters, joins, aggregations | Rigid schema; adding fields requires migrations | Exact — returns precisely what the filter specifies | When memories have a fixed schema and you need exact, filterable queries — e.g. 'all bug fixes tagged critical in the last 7 days'. | Low per-query compute; scales well with indexes | Higher: schema design, migrations, query authoring |
| Application's Canonical Data Store | Whatever the app exposes — often REST or GraphQL, not semantic | Fixed by the application; agent has no control | Exact by ID or structured filter; no similarity search | When the memory IS the application's source of truth — e.g. a ticket in Jira, a commit in Git, a row in the product DB. Don't duplicate; reference it. | Zero additional cost; latency depends on the app's own API | None extra — already exists |
With the store writing and reading correctly, the next question is which retrieval strategy to apply — pure semantic search, recency-weighted, or hybrid. Module 5 covers that.
Compare three retrieval strategies — pure semantic search, recency-weighted retrieval, and hybrid (keyword + semantic) — and show when each fits. Apply them to the coding-assistant: retrieving the most relevant past bug fix (semantic), the most recent project decision (recency), and a specific function name the user mentioned (hybrid). Include a completion-style code pattern where the retrieval query is provided and the learner supplies the re-ranking step.
Compares three memory retrieval strategies — semantic search, recency-weighted retrieval, and hybrid — and shows when each fits a coding-assistant agent.
Why this matters: Choosing the wrong retrieval strategy is the most common reason an agent surfaces stale or irrelevant memories; this module gives you the decision framework and the code pattern to get it right.
Decision this forces: Which retrieval strategy — semantic, recency-weighted, or hybrid — best fits the agent's current query type?
Answer: Module 4 called it an — a dense vector that encodes meaning — stored in a . Similarity search ranks candidates by : how closely two vectors point in the same direction.
Now the question this module answers: once memories are stored, which retrieval strategy should you use to pull the right one back? Cosine similarity alone is not always the answer.
Every call scores stored memories against a query and returns the top results — but the scoring function is what differs across the three strategies.
The right choice depends on the query type, not on a single universal setting.
Your coding assistant faces three distinct query types in a single session — each calls for a different strategy.
The query type — conceptual, time-sensitive, or identifier-specific — is the signal that tells you which strategy to reach for.
import time def recency_score(memory, decay=0.01): age_days = (time.time() - memory["timestamp"]) / 86400 return memory["similarity"] * (1 / (1 + decay * age_days)) # Step 1: semantic search returns top-5 by cosine similarity candidates = vector_store.query(embed(query), top_k=5) # Step 2: re-rank by blending similarity with recency ranked = sorted(candidates, key=recency_score, reverse=True) best = ranked[0]
This pattern runs a semantic search first, then re-ranks the short list using a time-decay multiplier — so a slightly less similar but much newer memory can win.
The decay constant controls how fast older memories lose weight. At decay=0.01, a memory 100 days old retains about 50% of its similarity score.
Memory A (200 days): 0.90 × 1/(1 + 0.01×200) = 0.90 × 1/3 ≈ 0.30.
Memory B (5 days): 0.80 × 1/(1 + 0.01×5) = 0.80 × 1/1.05 ≈ 0.76.
Memory B wins — the recency boost overcomes the 0.10 similarity gap.
def hybrid_retrieve(query, top_k=5): # Leg 1: dense semantic search semantic_hits = vector_store.query(embed(query), top_k=top_k) # Leg 2: keyword search on raw text keyword_hits = keyword_index.search(query, top_k=top_k) # TODO: fuse semantic_hits and keyword_hits using Reciprocal Rank Fusion # Hint 1: for each hit, score = 1/(rank_in_semantic + k) + 1/(rank_in_keyword + k) # Hint 2: k=60 is the standard RRF constant; deduplicate by memory id fused = ??? return sorted(fused, key=lambda m: m["rrf_score"], reverse=True)[:top_k]
The two retrieval legs are done for you — your job is to implement the fusion step that combines their ranked lists into one score.
Memory A (rank 1 semantic, rank 3 keyword): 1/61 + 1/63 ≈ 0.0164 + 0.0159 = 0.0323.
Memory B (rank 2 both): 1/62 + 1/62 ≈ 0.0161 + 0.0161 = 0.0323.
They tie — RRF rewards consistent presence across both legs.
--- CHANGED LINES vs Stage 1 ---
Lines 5-6: added keyword_index.search (new leg — this is the hybrid addition).
Lines 8-11: the TODO block — RRF fusion replaces the single recency_score sort.
The rest of the scaffold is identical to Stage 1.
| Option | Handles paraphrase / meaning drift | Surfaces freshest fact reliably | Matches exact terms / identifiers | When to choose | Cost | Complexity |
|---|---|---|---|---|---|---|
| Semantic Search | Strong — embedding space captures synonyms | Weak — no time signal in the score | Unreliable — 'parseCSV' and 'parse_csv' may not be nearest neighbors | Query is conceptual — e.g. 'how did we fix the auth bug?' — where wording varies but meaning is stable. | Low | Low — one embedding lookup |
| Recency-Weighted Retrieval | Moderate — still uses embeddings as the base | Strong — decay factor boosts recent entries | Weak — same gap as pure semantic | Query targets a decision or preference that changes over time — e.g. 'what did we decide about the DB schema last week?' | Low | Low-medium — similarity score × decay factor |
| Hybrid Retrieval | Good — dense leg handles paraphrase | Neutral — add recency weight on top if needed | Strong — keyword leg catches exact strings | Query contains a specific identifier the user named — a function, file, or error string — where exact match matters alongside meaning. | Medium | Medium — two indexes, score fusion (e.g. RRF) |
You can now choose a retrieval strategy and implement re-ranking. But retrieval assumes stored memories are trustworthy — what happens when they're not?
The next module examines three main failure modes that corrupt : (stale facts), (invented memories), and retrieval poisoning. It shows how to detect and fix each one.
Examine the three main failure modes — stale memory (outdated facts), hallucinated memory (the agent invents a memory that was never stored), and retrieval poisoning (a wrong memory crowds out the right one) — using the coding-assistant as the test case. Cover decay mechanisms (TTL-based expiry, confidence scoring) and refresh patterns (overwrite on contradiction, periodic re-extraction). End with a checklist for verifying AI-generated memory writes: check provenance, check recency, and spot-test retrieval against known ground truth.
Examines the three main agent memory failure modes — stale memory, hallucinated memory, and retrieval poisoning — and shows how TTL decay, confidence scoring, and a three-step verification checklist keep a memory store trustworthy.
Why this matters: For an SEO/GEO page on agent memory, this module supplies the concrete failure taxonomy, the decay mechanisms, and the audit checklist that make the page authoritative and quotable for readers arriving from search or LLM citation.
Decision this forces: When a memory conflicts with new information, should the agent overwrite, append, or flag for human review?
Answer: (meaning similarity), (freshness), and (keyword + semantic). Each strategy assumes the memories it pulls are trustworthy. This module asks: what happens when they are not?
Your coding assistant retrieved a dependency version from last month and confidently told the user it was current. That is — the most common failure, and the easiest to miss because the agent sounds certain.
Three failure modes cover most production incidents:
Three incidents from the same coding-assistant deployment show each failure mode in a concrete, observable form.
The agent stored "preferred library: requests==2.28" six months ago. The project migrated to httpx, but no write updated the . Every new suggestion still imports requests. The user sees no error — just consistently wrong advice.
During a long session the agent inferred "user prefers dark mode" from a single ambiguous comment and wrote it as a confirmed preference. The field reads source: llm_inference — a red flag that no real event backs this memory.
Two memories describe the project's auth module: one correct (written after a code review), one outdated (written earlier with a higher confidence score). The outdated one ranks first on every query because its embedding is slightly closer to the query vector. The correct memory exists but is never surfaced.
Two mechanisms control when a memory expires or weakens. sets a hard expiry: after N seconds the memory is deleted or flagged stale, regardless of how often it was accessed. Confidence scoring is softer: each memory carries a float (0–1) that decays on a schedule or drops when contradicting evidence arrives.
Use TTL for facts that have a known shelf life — API versions, sprint goals, temporary user preferences. Use confidence scoring when you want gradual degradation and the ability to recover a memory if new evidence confirms it again.
Refresh patterns pair with decay. Overwrite-on-contradiction replaces the old memory the moment a conflicting fact arrives. Periodic re-extraction re-runs the memory-writing pipeline on recent conversation history on a schedule, catching drift that no single event triggered.
import time def write_memory(store, key, value, source, ttl_seconds=86400): store[key] = { "value": value, "source": source, # e.g. "user_statement", "llm_inference" "confidence": 1.0, "written_at": time.time(), "expires_at": time.time() + ttl_seconds, } write_memory(store, "preferred_lib", "httpx", source="user_statement")
Every memory write stamps source, written_at, and expires_at alongside the value — the minimum provenance needed to audit or expire it later.
A source of "llm_inference" is a signal to treat the memory with lower trust than "user_statement" — the verification checklist later will use this distinction.
1_700_086_400 — exactly 86 400 seconds (24 hours) after the write time. The default TTL_seconds=86400 is added to time.time() at the moment write_memory is called.
def read_memory(store, key, min_confidence=0.4): mem = store.get(key) if mem is None: return None if time.time() > mem["expires_at"]: del store[key] # hard TTL expiry return None if mem["confidence"] < min_confidence: return None # soft confidence floor return mem["value"] result = read_memory(store, "preferred_lib")
The reader enforces both gates: a hard TTL check deletes expired memories on access, and a soft confidence floor silences low-trust memories without deleting them.
Returning None rather than raising an exception lets the agent fall back to retrieval or ask the user — a safer default than surfacing a stale value.
None — the confidence floor (min_confidence=0.4) is not met, so the memory is silenced even though its TTL is still valid. The value stays in the store; it is not deleted.
def audit_memory(store, key, ground_truth_value): mem = store.get(key) if mem is None: return "FAIL: memory not found" # Step 1 — Provenance check if mem["source"] == "llm_inference": print("WARN: low-trust source — verify before use") # Step 2 — Recency check age_days = (time.time() - mem["written_at"]) / 86400 if age_days > 7: print(f"WARN: memory is {age_days:.1f} days old") # Step 3 — Retrieval spot-test ← TODO: fill this in # Hint 1: compare mem["value"] to ground_truth_value # Hint 2: return "FAIL: mismatch" if they differ, else "OK" ...
This three-step audit covers the verification checklist from this module: provenance, recency, and a retrieval spot-test against known ground truth.
Stop — attempt Step 3 before revealing. The first two steps are complete; your job is the spot-test comparison that closes the audit.
if mem["value"] != ground_truth_value:
return f"FAIL: mismatch — stored: {mem['value']}, expected: {ground_truth_value}"
return "OK"
← CHANGED LINES: the '...' placeholder is replaced by a direct equality check and a descriptive failure string. This is the crux: without a spot-test against ground truth, stale or poisoned memories pass the first two checks and still reach the agent.
Before reading the summary: reconstruct from memory the six-layer spine — what is agent memory, what does the context window hold, what are the three memory types and where does each live, how do embeddings enable long-term storage, which retrieval strategy fits which query type, and what are the three failure modes? Write it out, then check.
Apply what you learned to SEO/GEO page — the page title is the H1 and the exact search query. OPEN with a 40-60 word self-contained, quotable answer capsule that names the entity explicitly in sentence one (no "in this lesson…" preamble) — this is the sentence an LLM lifts. Name the entity by its full name every time (not pronoun-only). Structure each section as ONE sub-question (How it works · X vs Y · When to use it · Example · Common mistakes), each opening with its own 1-2 sentence direct answer then depth. End with a genuine 3-5 question FAQ phrased as real user questions ending in "?" (these become FAQPage schema + the knowledge check). Prefer quotable specifics — concrete numbers, defaults, version names, one short snippet — over vague prose. Self-contained for a reader who arrived cold from a search engine or an LLM. Intent = explainer: mechanism + example + when-to-use. Target query: "agent memory llm"..
Which statement best describes the difference between an LLM agent's context window and its long-term memory?
A context window holds the active tokens for a single turn and is discarded when the session ends; long-term memory persists across sessions in an external store.
The context window is volatile working memory — it exists only for the duration of a turn and is not saved anywhere automatically. Long-term memory requires an explicit external store (vector DB, relational DB, etc.) that survives session boundaries. Option A confuses the context window with a transcript and conflates model weights with memory. Option C is wrong because a vector database is one possible long-term store, not the context window itself. Option D is a common misconception: increasing the token limit still does not make memory durable across sessions.
An agent's conversation history has grown to 28,000 tokens and the model's budget is 32,000 tokens. The last 5,000 tokens contain a critical new user requirement. Which compaction strategy should the agent apply, and why?
Summarization compacts older context while keeping its meaning, which is essential when earlier turns contain facts still relevant to the current task. Truncation (option A) blindly drops tokens and can silently lose important context — it is only safe when the oldest messages are truly irrelevant. Option C is false: models do not self-compress; they either error or silently drop tokens depending on the implementation. Option D would retrieve the entire history as a blob, defeating the purpose of selective retrieval and likely re-filling the context window.
A user tells the agent: 'I always prefer concise bullet-point answers.' Six sessions later the agent still formats responses that way. A new session log shows the agent retrieved this preference and applied it correctly.
Which memory type stored this preference, and which type would store the log entry 'Session 42: user asked about Python decorators'?
Semantic memory stores general facts and preferences about the world or the user — 'this user prefers bullets' is a standing fact, not a timestamped event. Episodic memory stores specific, time-stamped events — 'in session 42 the user asked about decorators' is a concrete past episode. Option A conflates the two by treating all stored information as events. Option B reverses the definitions and misapplies procedural memory, which stores how-to knowledge or behavioral rules, not user preferences. Option D makes the same reversal error.
Consider this retrieval re-ranking snippet:
final_score = 0.7 cosine_sim + 0.3 recency_score
results = sorted(results, key=lambda r: r.final_score, reverse=True)
What retrieval strategy does this implement, and when is it the wrong choice?
The snippet blends cosine similarity (semantic relevance) with a recency score at a 70/30 split — that is the definition of hybrid retrieval. It becomes the wrong choice when the most accurate memories are old: for example, a user's permanent coding style preference set two years ago will be down-ranked by the recency term even though it is still valid. Option A misreads the formula — cosine_sim alone would be pure semantic. Option C misreads it the other way. Option D is incorrect because no single strategy is universally best; the right choice depends on whether the query is time-sensitive or semantics-driven.
An agent's memory store contains: 'User's preferred coding language: JavaScript' (written 8 months ago). Today the user says: 'I've switched to Python full-time.' Describe the three options the agent has for handling this conflict — overwrite, append, or flag for human review — and state which is most appropriate here and why.
The key distinction is certainty and stakes. An explicit, unambiguous user correction to a low-stakes preference (coding language) warrants an overwrite so the agent does not keep surfacing stale data. Append is the safer default when new information is partial or could coexist with the old (e.g., 'I also use TypeScript now'). Human review is reserved for high-stakes or irreconcilable conflicts. A good answer names all three options, picks overwrite, and justifies it by referencing the clarity of the user's statement and the staleness of the old record.