AI Engineering Interview Questions: Agents & Tool-Calling

Modern agent frameworks (e.g., OpenAI function-calling, LangChain agents, ReAct-style loops) allow the model to iteratively call tools over multiple steps. After each tool response is appended to the conversation context, the model can decide to call another tool or produce a final answer. This multi-step tool-use is a core capability that separates agents from simple single-turn LLM calls.

The standard tool-calling loop has four stages: (1) the developer supplies tool schemas so the model knows what tools are available; (2) the model emits a structured tool call (not prose) when it decides a tool is needed; (3) the orchestration layer runs the tool and feeds the result back as a special 'tool' role message; (4) the model uses that result to craft its final answer. Understanding this loop is foundational to building LLM-powered agents.

none is the mirror image of forcing a tool: instead of guaranteeing a call, it guarantees no call, which matters whenever you need the model to communicate rather than act on a specific turn without having to strip and re-add tool definitions for that one turn.

In agentic LLM systems, a tool (or function) is an external capability registered with the model. When the model decides to use it, it emits a structured call (e.g., JSON with a function name and arguments). The orchestrating framework executes that call and injects the result back into the conversation context, allowing the model to reason over the outcome. This is distinct from fine-tuning, system prompts alone, or RAG indexes.

Tool call results are NOT written into the model's weights. LLMs are stateless between inference calls; the tool's response is simply appended to the conversation context (the prompt) for the current session. If the information needs to persist across sessions it must be stored externally (e.g., in a database or memory module) and explicitly retrieved later—it is never baked into the weights at inference time.

The modern OpenAI tool-calling API (introduced mid-2023) uses the tools parameter to pass tool definitions, not the legacy functions parameter. Passing functions=[...] uses the deprecated interface and will be rejected or ignored depending on the model version. The correct call should use tools=[tool_definition] together with tool_choice="auto". Everything else in the snippet—the schema structure, required placement, and message format—is valid.

Parallel tool calls are a latency win when the calls are genuinely independent, but the API's flexibility about execution order puts the correctness burden on the application: every emitted call needs a matching result, correctly IDed, or the conversation state becomes invalid.

Tool calling is a structured-output contract: the model emits a request (tool name plus schema-validated arguments) rather than running code itself. The host application executes the function, returns the result as a tool message, and the loop repeats. This separation is the whole security model — the model never has direct execution authority, so you can validate arguments, enforce permissions, and sandbox effects before acting. It is also the foundation of agent loops and MCP.

Tool/function calling works as a two-turn protocol: the model emits a structured tool_call (not prose), the host runs the function, and the result is injected back to the model. Providers name that result message differently — OpenAI uses a tool role message, while Anthropic returns a tool_result content block inside the user turn — but the protocol is identical. This separation keeps execution outside the model—the model only decides what to call and with what arguments, never running the code itself. Validating those arguments against a schema before execution is a key safety guard.

Line 3 calls tool.execute with no try/catch, so when it throws — a network timeout, a downstream 500, a bug in the tool itself — the exception propagates straight up and kills the entire request instead of becoming a tool result the model can react to. The fix wraps the call in try/catch and, on failure, still pushes a tool message with an error flag and short description, so the model learns the call failed and can decide what to do next — retry, try something else, or tell the user — instead of the user seeing an unhandled server error with no explanation.

The tool-choice modes trade the model's autonomy for reliability along a spectrum: auto gives it full discretion (including replying with no call at all), forcing any tool removes the "no call" option but not which tool, and forcing a specific tool removes both, guaranteeing exactly one call with schema-conformant arguments. That last mode is the backbone of using tool calling purely as a structured-output mechanism — you never actually run side effects from the "tool", you just exploit the schema-constrained decoding to get reliable JSON instead of parsing prose. None is the escape hatch for turns where you want the model to talk without the option to act.

The line between 'tool calling' and 'agent' is architectural, not a marketing label: the loop the model runs and the decisions it's allowed to make on its own is what separates them, and that difference is exactly what should drive the build-vs-simplify call on a new feature.

40 schemas at 120 tokens is 4,800 tokens on every single call, and twelve iterations makes 57,600 tokens for the run — before a word of the actual conversation. That is the part teams miss: tool definitions are not sent once at the start, they are part of every request the loop makes, so a large tool surface is a fixed tax multiplied by the iteration count. It is also the cheapest thing to cut, because most runs need a handful of those forty. Narrowing the tool set per run reduces this directly and improves selection accuracy at the same time — one change, two wins. Note the interaction with caching: tool definitions sit near the front of the prompt and are stable, so they cache well when their order is deterministic; build the list from an unordered map and you lose both the cache and any hope of diagnosing why.

When an LLM is given a set of tools, it returns a structured 'tool_call' object (or equivalent) rather than a plain text answer when it decides a tool should be invoked. The caller is then responsible for executing the tool, collecting the result, and passing it back to the model as a 'tool' role message. The model does NOT execute the tool itself, and it does not simply embed the result inline — the conversation must include a follow-up turn with the tool's output before the model produces the final answer. This round-trip is fundamental to how OpenAI-style function/tool calling works.

Parallel tool calling allows a model to emit multiple tool_call entries in a single response, which the agent can execute concurrently before returning all results. The key subtlety is that each tool result must be returned as a separate role: "tool" message keyed by its tool_call_id. If any result is missing from the follow-up request the model has incomplete context. Additionally, if tools share side-effects or sequential data dependencies (e.g., tool B needs tool A's output), parallel execution is unsafe and the agent must serialize them. 'Strict' JSON schema validation prevents the model from generating malformed arguments, but does not force sequential execution.

The bug is on line 29. After appending the tool result message to conversation, the follow-up API call on line 29 re-passes the original messages list instead of the updated conversation list. This means the model never sees the assistant's tool_call message or the tool result, causing it to behave as if no tool was ever called. The fix is to pass conversation to the second client.chat.completions.create call so the full conversation history — including the assistant's tool invocation and the tool's response — is sent to the model.

Models can and do emit parallel tool calls (supported natively in the Anthropic and OpenAI APIs), but they have no reliable way to express or enforce data dependencies between those calls. If tool B depends on the output of tool A, the model may still emit both in one response — and executing them in parallel produces incorrect results or a runtime error. As the orchestrator, you must detect dependencies (e.g. one tool's argument references another's output placeholder) and sequence or group calls accordingly. A safe default is to treat co-emitted calls as parallel only if their inputs are independent of each other, and always validate before executing.

Agentic RAG treats each retriever as a tool with a descriptive contract, and the model performs query routing: it reads the tool descriptions plus the user intent to choose the right source(s) and tailor the query to each. Querying every source unconditionally is wasteful and counter-productive — extra cost/latency, and irrelevant passages crowd the context and degrade the answer. Tool-mediated routing also enables iteration (retrieve, assess sufficiency, retrieve again from a different source only if needed). The cost is the dependence on well-written tool descriptions and a routing decision that the model can get wrong, which is why routing quality belongs in your eval set.

The loop is: present the model with the tools it may call, the model decides to emit a tool call with structured arguments, your code executes that tool (the model never runs it), you return the result as a tool/function message appended to the conversation, and the model then synthesizes the final answer from it. For multi-step tasks the middle three steps repeat until the model stops requesting tools.

Reliable tool calling comes from giving the model unambiguous signal at both selection time and argument-filling time. Specific descriptions (a) and tight argument typing (b) are exactly that signal, and a narrow, non-overlapping tool set (c) removes the ambiguity that causes wrong-tool selection in the first place — reinforced by disambiguating detail where two tools could plausibly be confused (e). Defining many overlapping tools "just in case" (d) is the opposite: it increases the chance the model picks the wrong one and makes every description's job harder, since it now has to differentiate itself from near-duplicates instead of just describing its own job.

Schema validation only checks shape and type (this argument is a string, that one is an integer) — it says nothing about whether a string is safe to interpolate into a query or a shell command, so treating schema-passing as "safe to execute" (d) is a real vulnerability, not a genuine practice. The model's arguments can be influenced by untrusted content it read earlier in the conversation (c), so they deserve the same scrutiny as any other external input: validate/sanitize before executing (a), run with the least privilege the task needs so a bad or manipulated call can't do more damage than necessary (b), and keep an audit trail (e) so you can detect and investigate when something did go wrong.

When an LLM agent receives a tool call response, the message history must follow the provider's expected role sequence. For OpenAI's chat completions API with parallel tool calls, the assistant message containing the tool_calls array must come first, then one tool role message per tool call (each referencing the matching tool_call_id), before the model can generate a follow-up. Inserting a plain user message or omitting the assistant tool_calls message before the tool results causes a 400 validation error. Option C describes the only valid sequence.

The pattern only holds its trust boundary if the untrusted content is read exclusively by the tool-less component (b) after the task is issued (a), and only a narrow, typed extract crosses into the privileged side (c) — passing the raw page text across would recreate the exact vulnerability the split exists to avoid. The privileged LLM then makes the tool-call decision from that sanitized variable alone (d), and because even a correctly-designed extraction step or a flawed controller can still misfire, the resulting tool call is still checked against an independent hard rule before it takes effect (e) rather than being trusted purely because it came from the privileged side.