AI Engineering Interview Questions: Agents & Agent Systems

In a standard LLM agent loop, the correct execution order is: (1) the LLM reasons over the current context and selects an action, (2) that action or tool call is executed, (3) the agent receives the observation or tool result, (4) a stopping condition is checked to see if the task is complete, and (5) if not done, the observation is appended to context and the next iteration begins. This repeating think-act-observe cycle is the core pattern behind frameworks like ReAct.

The agent runtime (or orchestrator) that wraps the LLM is responsible for actually executing tool calls like web searches or API requests. The LLM itself only reasons about which action to take and what arguments to pass — it cannot run code or call external services directly. The runtime performs the action, then feeds the result back to the LLM as an observation to continue the loop.

Valid stopping conditions for an LLM agent loop include the LLM emitting a designated "Final Answer" signal, a cap on the maximum number of steps or iterations being reached, and a programmatic check detecting that the task objective is satisfied. Any of these terminates the loop, which would otherwise run indefinitely. Reaching a minimum token count, by contrast, is not a practical or meaningful stopping criterion.

True. A vector database used as an agent's external memory persists embeddings of past interactions to disk. This means the agent can retrieve relevant information from previous conversation sessions — not just the current one — effectively providing long-term memory that survives context window resets.

To prevent an agent loop from running indefinitely, developers should set a maximum-step limit on the number of iterations, and the loop should also exit when the agent emits a terminal action (e.g., finish or final_answer).

An agent runs an iterative loop: it plans a next step, acts (often via a tool call), observes the result, and decides what to do next toward a goal, rather than making one call and stopping. A workflow orchestrates LLMs and tools through a predefined code path; an agent is a system where the model dynamically directs its own process based on what it just observed. A single tool call followed by returning the result, with no model-decided branching, is a workflow of depth one, not an agent. Using multiple LLM providers or streaming tokens are unrelated implementation details.

You give the model a set of tool definitions with names, descriptions, and JSON-schema parameters. When the model decides a tool is needed it doesn't run anything — it returns a structured tool-call request with arguments matching the schema. Your application code executes the function, then sends the result back to the model as a tool-result message, and the model continues, optionally calling more tools, until it produces a final answer. The model only chooses and fills in arguments; your code is the one that actually executes the side effect.

MCP is an open protocol that standardizes how LLM applications connect to external tools, data sources, and prompts through a uniform client/server interface. Instead of writing a bespoke integration for every tool and every model host, you build an MCP server that exposes capabilities — tools (callable functions), resources (readable data/context), and prompts (reusable templates) — and any MCP-compatible client (host app) can discover and use them. It decouples integrations from any single vendor: write the server once, reuse it across hosts.

A fixed pipeline of three deterministic steps — a workflow — fits best, because the control flow never actually depends on the model's own judgment about what to do next. When a task's steps are fixed and predictable ahead of time, a deterministic pipeline gives the same result with lower cost, lower latency, and fully reproducible behavior; an agent's autonomy buys nothing when there's no real decision to make, and adds a new failure surface instead. A multi-agent supervisor adds coordination overhead for no benefit here, and letting a single prompt decide whether to skip steps just reintroduces the same non-determinism.

Enforce a hard cap on iterations or tool calls, and/or a token or dollar budget, per task, forcing the agent to stop and return its best partial answer once the cap is hit. Maximum iteration limits are a core agent safeguard because agent errors can compound and autonomy needs a hard external ceiling, not a hope that the model self-regulates. Lowering temperature makes output deterministic but doesn't bound iteration count — a deterministic 'not sufficient yet' judgment will loop forever without an external stop. A bigger context window only delays the crash, and removing the tool disables legitimate functionality instead of bounding misuse.

False. A hard iteration or token budget bounds cost and runtime — it forces the agent to stop — but it does not make the agent's reasoning or tool use correct. An agent can exhaust its entire budget on a flawed plan or a misread tool result and still be wrong when the cap hits, and enlarging the budget only lets more unproductive work happen before the forced stop rather than fixing why the agent went wrong. Budgets protect cost and latency; correctness needs its own guardrails, like sufficiency checks and evaluation.

False. More agents means more inter-agent hand-offs, more tokens spent on coordination, and more opportunities for one agent's error or stale context to propagate downstream — documented failure modes that can make a multi-agent system worse than a single agent, especially on tasks that are fundamentally sequential rather than genuinely parallelizable. Multi-agent decomposition pays off when sub-tasks are large, independent, and benefit from focused per-agent context and tools; it is not an automatic quality win for every task.

Planning every step upfront without looking at results defeats the reason to use an agent at all — it collapses back into a fixed pipeline of predetermined steps decided before any evidence exists, so it can't adapt when a tool returns something unexpected: an empty search result, an error, or a value that changes what the next question even should be. The point of the plan-act-observe loop is that each observation feeds back into the model's context before the next planning step, so the agent can revise its plan, retry with different arguments, ask a follow-up, or decide it already has enough information and stop early. Batch-planning N calls upfront only works when the calls are genuinely independent and none of them could change what the others need to look like; the moment step 2 might depend on what step 1 returned, planning both before observing either throws away the model's ability to react — which is exactly what separates an agent from a scripted workflow.

Tool-calling is a capability — the model can emit a structured request for your code to execute. An agent is an architecture built on top of that capability: the model runs an iterative loop where it plans a next step, observes the result of acting on it, and decides on its own what to do next, potentially many times, until it judges the task done. A single tool call followed by one response is much closer to a scripted workflow than an agent — the control flow is still effectively fixed. The distinction matters architecturally because agents trade predictability for flexibility: more LLM calls, more latency, non-deterministic behavior, and a need for guardrails (iteration caps, budgets, human checkpoints) a single call or fixed pipeline doesn't need. Reach for an agent only when the task's path genuinely can't be predetermined; otherwise a workflow of one or more scripted tool calls is simpler, cheaper, and more predictable.

The loop prints 7000 3000, billing 1000, 1500, 2000 and 2500 input tokens across its four iterations, summing to 7000, while the transcript itself only reaches 3000 tokens. The gap is the whole point: every earlier token is paid for again on every later step, so the run costs more than twice the final context size. Counting only the tokens each step adds gives 4000 — the single-call intuition that makes agent bills surprising. At 10 steps the sum reaches 32,500 against a 6000-token transcript.

The orchestrator-workers pattern fits: a lead agent decomposes the task into sub-tasks, delegates each to a worker agent scoped to one domain's tools and context, then synthesizes the workers' results into the final report. This suits tasks whose subtasks can't be predicted in advance, letting each worker hold a focused, uncluttered context. Loading every tool into one generalist agent causes tool-selection confusion and context overload; running parallel copies of a generalist doesn't decompose the task by domain; and chaining domains as sequential turns in one conversation is still a single agent with no real specialization.

Genuine guardrails against a runaway agent are a hard cap on iterations or tool calls (and/or a token or dollar budget) with a forced stop, a human-in-the-loop checkpoint before irreversible or high-stakes actions, an overall wall-clock timeout independent of iteration count, and least-privilege scoping of the agent's tools and permissions so a runaway loop can only touch what it's allowed to. Running at temperature 0 is a trap: determinism only means the same input reliably produces the same output, not that the process converges — a deterministic 'not sufficient yet' judgment will loop forever without an external cap.

Genuine multi-agent failure modes are coordination overhead — extra tokens and rounds spent on hand-off can outweigh any parallelization gain, especially on sequential work — error compounding, where a downstream agent trusts an upstream agent's wrong result without re-verifying it, and context pollution, where shared or forwarded history carries one agent's mistaken assumptions into another agent's context. Splitting a task across more agents does not guarantee higher accuracy — it can just as easily amplify errors — and multi-agent chains do not implicitly validate each other's output; assuming the next agent will catch a mistake is exactly how errors compound silently.

Layer several independent guardrails rather than relying on the model to self-regulate: a hard cap on iterations/tool calls per task; a token or dollar budget that forces a stop when exceeded; an overall wall-clock timeout independent of iteration count, since a slow loop could still run forever in real time without one; and a human-in-the-loop checkpoint before any irreversible or high-stakes action. When any cap is hit, the agent should return its best partial answer rather than erroring out or silently continuing. The model's own 'not confident yet, let me check once more' judgment can't be the only backstop because it's exactly the mechanism that failed here: if the model's internal sufficiency check never concludes 'yes, I have enough,' it will keep looping indefinitely by its own logic, no matter how reasonable that logic looked on the first few iterations. The guardrail has to be an external, deterministic circuit-breaker the agent cannot reason its way around — not another judgment call made by the same model that just got stuck.

Orchestrator-workers: a lead/supervisor agent decomposes a task into sub-tasks it couldn't fully predict in advance, delegates each to a worker agent scoped to focused context and tools, then synthesizes the workers' results — fits open-ended tasks like multi-domain research or multi-file code changes, where the shape of the work depends on what earlier steps discover. Sequential handoff (a pipeline of specialist agents): each agent completes its stage and passes a structured result to the next, with no shared free-for-all context — fits tasks with a genuinely fixed stage order (e.g. draft → fact-check → format), where each stage's specialist only needs its input, not the others' full history. Both trade single-agent simplicity for parallelism or specialization, at the cost of coordination overhead and the risk that one agent's error or stale context propagates to the next.

An orchestrator-workers pass with a supervisor and specialist workers runs in this order:

1. Orchestrator receives the task and decomposes it into domain-scoped sub-tasks (legal risk, financial risk)
2. Orchestrator delegates each sub-task to the worker agent scoped to that domain's context and tools
3. Each worker agent runs its own plan/act/observe loop independently and returns a structured result
4. Orchestrator collects the workers' results and checks them for consistency and completeness
5. Orchestrator synthesizes the results into a single combined answer for the user

The bug is on line 17.

An agent's trajectory — which tools it calls, in what order, with what intermediate reasoning — can differ run to run and still land on a correct final answer, so scoring only the final output can't tell a sound path from a lucky or risky one, and a single bad step early on can silently corrupt every step after it. Each step is its own LLM sample, so stochasticity compounds, and several different valid tool-call sequences can reach the same correct answer, leaving no single 'expected trajectory' to grade against. This is why agent evals score the trajectory itself alongside outcome correctness. Agents are not guaranteed more accurate than single calls, and they do produce a final answer worth grading in addition to the trajectory.

Three compounding problems, not just general complexity. First, non-determinism compounds across steps: every tool call and re-plan is its own LLM sample, so the same task can take a different path on different runs, and a small divergence early on can snowball into a completely different trajectory by the end — a single pass/fail run tells you little about the distribution of outcomes. Second, there's no single ground-truth trajectory: several different sequences of tool calls can all reach the same correct final answer, so scoring against one 'expected path' penalizes valid alternatives, while scoring only the final answer misses whether the path taken was efficient, safe, or just lucky. Third, failures are spread across many components and can be silent: a bad intermediate step, like a misread tool result or an unnecessary call, can still end in a textually correct final answer, hiding a real problem a final-answer-only eval would never catch. The practical fix is to evaluate the trajectory itself — which tools were called, in what order, whether intermediate steps were sound — in addition to final-answer correctness, and to run each task multiple times rather than once given the non-determinism.

Assume a few thousand runs a day, a refund ceiling of $200, and 30-day trace retention. Most tickets follow a handful of shapes, so I would route: a classifier sends the common ones down fixed workflows and only the tail reaches the agent, which cuts cost and shrinks the surface where things can go wrong.

The loop terminates on four conditions, not one: the model returns a final answer that passes a goal check; a cumulative token budget for the run is exhausted; a step cap fires as a backstop; or a repeated-call detector sees the same (tool, arguments) hash twice with nothing new in context. Budget is the primary control because steps are a poor proxy for cost — each iteration re-sends the transcript, so late steps cost far more than early ones. On any stop without an answer the agent hands the ticket to a human with its partial findings attached, rather than replying with something it is not sure of.

Tool failures go back into the context as tool results flagged as errors, so the model can adapt — try a different lookup, or tell the user. Retry policy is per-tool, not global: reads retry with exponential backoff, and the refund call never retries blindly, because a 5xx there is ambiguous rather than failed. Refunds carry an idempotency key so a duplicate collapses server-side.

For a 40-iteration run I pin the system prompt, tool schemas and goal, summarise older turns into a running recap once the transcript passes a threshold, clear stale tool results while keeping the calls that produced them, and keep large invoice payloads in storage behind a reference the agent re-reads on demand.

Refunds are gated in code. The loop intercepts the call before execution, verifies the amount against the invoice server-side, and requires human approval above a threshold — the model's authority is whatever the loop grants it, and the system prompt is a hint, not a control. Invoice text is untrusted: it reaches the context as data, and instructions inside it have no privilege, which is exactly why enforcement cannot live in the prompt.

Every run writes a trace keyed by run id and ticket id, one span per iteration, capturing the rendered prompt, the tool calls and arguments, results, token counts, and the stop reason. A week later I can open the ticket, find the run, and see which iteration went wrong and what the model was looking at when it did.

The main tradeoff is autonomy against friction: a lower approval threshold means fewer bad refunds and more human load. I would start conservative, measure the approval queue, and raise the threshold only against evidence.