Agentic RAG Interview Questions — AI Engineering

A defining property of agentic RAG is that retrieval is a tool the agent chooses to call, not a mandatory first step. For a question needing no external knowledge — small talk, a pure arithmetic/reasoning task, or something already present in the conversation — a well-designed agent answers directly and skips retrieval. That avoids a wasted round-trip and, more importantly, avoids stuffing irrelevant retrieved passages into context (which can introduce errors). The classic retrieve-then-generate pipeline always retrieves exactly once; making retrieval optional, model-controlled, and repeatable is precisely what 'agentic' adds — the agent can retrieve zero times, once, or several times.

Tool/function calling works as a two-turn protocol: the model emits a structured tool_call (not prose), the host runs the function, and the result is injected back to the model. Providers name that result message differently — OpenAI uses a tool role message, while Anthropic returns a tool_result content block inside the user turn — but the protocol is identical. This separation keeps execution outside the model—the model only decides what to call and with what arguments, never running the code itself. Validating those arguments against a schema before execution is a key safety guard.

Classic RAG always retrieves once; agentic RAG makes retrieval a decision the model takes inside an agent loop. The upside is adaptivity on hard queries; the downside is more round-trips, so it's a deliberate trade, not a free upgrade.

Classic RAG is a static pipeline: every query triggers exactly one retrieval, the top-k chunks are stuffed into the prompt, and the model generates. Agentic RAG puts retrieval under the model's control (b) — retrieval (often across several indexes/tools) becomes something the agent can invoke zero, one, or many times, formulating and reformulating the query, decomposing multi-hop questions, judging whether the retrieved context is sufficient and re-querying if not, then deciding when to stop and answer. It is not about memorizing the corpus into weights (a), streaming (c), or a bigger embedding model (d). The cost of the flexibility is more LLM calls — higher latency, cost, and non-determinism — so it pays off on multi-hop or ambiguous queries rather than simple single-fact lookups.

The query contains two distinct information needs — our tier's pricing/SLA and Competitor X's pricing/SLA — that likely live in different documents. A single embedding of the composite query produces a vector that averages both needs, which means retrieval tends to under-serve both: the top-k chunks satisfy neither sub-question well. An agentic retriever recognizes this (via a planning LLM step or a query-decomposition tool) and fires separate retrievals: one for each sub-question, potentially against different indexes (internal docs vs. a competitor intelligence store), then passes the combined context to the generator for a proper comparative answer (c). A better embedding model (a) helps marginally but cannot fix the fundamental issue that one vector is trying to do two jobs. Raising top_k to 50 (b) adds noise from irrelevant chunks and inflates cost without guaranteeing that both sub-topics are represented proportionally. Fine-tuning (d) is the wrong tool for frequently-changing competitor data and still cannot produce real-time citations.

Agentic RAG's value is deciding about retrieval. Multi-hop questions (a) need the iterative retrieve→read→retrieve loop a single-shot pipeline cannot do. Ambiguous queries (b) benefit from query reformulation/decomposition before searching. Multiple sources (c) require a routing decision the agent makes by choosing a tool. Skipping retrieval when none is needed (e) avoids a wasted round-trip and — more importantly — avoids injecting irrelevant passages that can introduce errors. The case where it does not pay (d) is the simple single-fact lookup a static top-k already nails: there the extra agent turns add latency, cost, and non-determinism for no accuracy gain. Match the retrieval architecture to query complexity.

Agentic RAG iterates retrieval, which introduces failure modes a single-shot pipeline cannot have. Compounding errors (a) occur because a bad first retrieval shapes the query the agent issues next, snowballing into a completely off-track trajectory. Runaway loops (b) happen when the agent's stopping condition is absent or misconfigured, triggering repeated retrievals at cost with no answer produced. Tool over-calling (c) wastes LLM calls and latency when the answer is already in context but the agent issues another retrieval anyway — often diagnosed by tracing tool-call sequences. Latency amplification (e) is intrinsic: each reasoning turn + retrieval round-trip stacks, and a 4-hop question can be 4× slower than a single pass. The claim that the context window is never large enough (d) is false — modern large-context models handle multi-hop results comfortably, and window size is a solvable engineering constraint, not an inherent agentic failure mode.

The win is model-controlled, iterative, multi-source retrieval that adapts to the query — deciding whether to retrieve, reformulating, decomposing, and re-querying until the context is sufficient. The price is extra LLM round-trips: latency, token cost, non-determinism, and harder eval/tracing, with a risk of looping. Match the architecture to query complexity; static retrieve-then-generate stays the right default for simple lookups.

The loop is decide → retrieve → assess → (re-retrieve) → answer. The agent first chooses to retrieve and formulates a query as a tool call (a); the tool returns candidates (b); the agent evaluates whether they are sufficient (c) — the step a static pipeline lacks entirely; on finding a gap it reformulates and retrieves again (d), the iterative multi-hop behavior; and only once the evidence is adequate does it generate the grounded answer (e). Real agents may repeat the retrieve→assess sub-loop several times; the defining contrast with static retrieve-then-generate is the assess-and-decide step that can trigger another retrieval.

This deliberately hands the candidate an architecture rather than a blank page, because accepting a proposed design uncritically is the failure mode worth detecting — and so is rejecting multi-agent reflexively after hearing it is usually overkill. The task genuinely has independent sub-work, so the right answer engages: keep the fan-out, drop the planner, since a known competitor list needs no model to decide it. The budget probe is the strongest discriminator, because per-agent caps feel like control and are not — five agents at twenty steps is a hundred steps of worst-case spend, at rising per-step cost, with no cap violated and nothing to alert on. The injection probe tests whether the candidate scopes authority rather than trusting instructions; read-only researchers make it structurally boring, which is the correct answer. And the citation and trace criteria are what separate a demo from something a team would actually rely on: without a path from a claim back to the page it came from, nobody can ever check the thing the product exists to produce.