AI Engineering Interview Questions: Agent Memory

Vector databases (e.g., Pinecone, Chroma, Weaviate) persist embeddings of past interactions or knowledge on disk. An AI agent can query this store at the start of or during a new session to retrieve relevant memories from prior sessions, effectively giving it long-term memory that survives context resets.

Agent memory is most commonly categorized into short-term (in-context) memory — information held within the LLM's active context window that disappears when the session ends — and long-term (persistent) memory — information saved to external storage such as databases or files and retrievable across multiple sessions. Understanding these two scopes is foundational to designing memory-aware agent architectures.

The distinction that matters is durability. Within-run context is working state that dies with the run; persistent memory is the small set of conclusions worth carrying forward. Teams usually get this wrong in one of two directions — persisting nothing, which produces an agent with amnesia between sessions, or persisting everything, which produces a store too large to retrieve from and full of superseded facts. The staleness point is the one most often missed: a remembered preference that was true in March and is quoted back confidently in September is worse than having remembered nothing.

Agent memory architectures are typically decomposed into four distinct tiers that serve different temporal and functional purposes. In-context (working) memory is the active prompt window — cheap to read but bounded by context length. External/episodic memory (e.g., a vector store of past episodes) allows retrieval across unlimited history but requires an explicit lookup step. Semantic/knowledge memory stores structured facts the agent can query. Procedural memory encodes learned skills or tool-use policies, often baked into weights or fine-tuned adapters — NOT the in-context window, which is stateless across sessions. The statement that in-context memory persists across independent agent sessions is false: a fresh invocation starts with an empty context unless explicitly reconstructed from an external store.

The bug is on line 10. results is a list of Document objects returned by LangChain's similarity_search; each document's text content lives in doc.page_content, not doc.content. Accessing .content returns None (or raises AttributeError depending on the version), so memory_context is assembled from None values and the agent receives no useful context. The fix is to replace doc.content with doc.page_content.

Agent memory architectures must make a deliberate choice about when to write to long-term memory. Writing after every single LLM turn (option A) floods the store with low-signal noise and degrades retrieval precision. Writing only at task completion (option C) risks losing context if the task is interrupted. Periodic summarisation with a dedicated memory-consolidation step (option B) is the established best practice: it mirrors the human sleep-consolidation model, reduces store size, and keeps retrievable memories semantically rich. Storing only tool-call logs (option D) omits the reasoning and observation content needed for rich episodic recall.

Long-running agents survive by managing the window rather than enlarging it. Two complementary moves dominate: (1) compaction — replace older turns with a running summary, keeping recent turns verbatim; (2) externalize + retrieve — persist the full history to an outside store and pull back only what's relevant to the current step (RAG over your own transcript). Supporting tactics: prune/evict low-signal turns, and pin durable facts (goal, decisions, constraints) in a small always-present block. Simply 'use a model with a bigger context window' defers the problem and raises cost/latency; the win is compressing and selecting what the model actually needs to see.

Real, documented multi-agent failure modes are coordination overhead — extra tokens and rounds spent on hand-off can outweigh any parallelization gain, especially on tasks that are fundamentally sequential rather than genuinely parallelizable (a); error compounding — a downstream agent trusts an upstream agent's wrong or hallucinated result without re-verifying it, so mistakes propagate and can amplify rather than cancel out (b); and context pollution — shared or forwarded history carries one agent's mistaken assumptions or stale state into another agent's context (c). (d) is false: more agents does not guarantee higher accuracy — it can just as easily amplify errors, and reliably improving quality requires explicit structure like verification steps, not headcount. (e) is false and dangerous: assuming 'the next agent will implicitly catch it' is precisely the assumption behind the error-compounding failure mode in (b) — an implicit downstream glance is not an actual validation step, so chaining agents with no explicit check is exactly how bad results silently propagate.

Memory architecture in autonomous agents involves several distinct tiers, each with different latency, persistence, and capacity profiles. In-context (working) memory is fastest but ephemeral and token-limited; external vector/semantic stores provide large-scale fuzzy recall; relational/graph stores provide structured, exact-lookup long-term memory; and a parametric (fine-tuned weights) tier bakes knowledge directly into the model but is expensive to update. The correct ordering from fastest/most ephemeral to slowest/most persistent is: in-context working memory → external semantic/vector store → structured relational/graph store → parametric (fine-tuned weights). Options that place vector stores before in-context memory, or parametric memory before structured stores, reflect fundamental misunderstandings about write latency and retrieval cost.