Ruby on Rails interview questions

An agent loop (also called a ReAct loop or think-act loop) follows a repeated cycle: the agent observes the current state/context, reasons or thinks about what to do, takes an action (e.g., calls a tool), receives an observation back, and repeats until a stopping condition is met. This is the fundamental pattern behind virtually all LLM-based agent frameworks.

Agent memory architectures are typically decomposed into four distinct tiers that serve different temporal and functional purposes. In-context (working) memory is the active prompt window — cheap to read but bounded by context length. External/episodic memory (e.g., a vector store of past episodes) allows retrieval across unlimited history but requires an explicit lookup step. Semantic/knowledge memory stores structured facts the agent can query. Procedural memory encodes learned skills or tool-use policies, often baked into weights or fine-tuned adapters — NOT the in-context window, which is stateless across sessions. The statement that in-context memory persists across independent agent sessions is false: a fresh invocation starts with an empty context unless explicitly reconstructed from an external store.

LLM-as-a-judge is a popular evaluation technique, but it has well-documented systematic biases. Studies (e.g., from Zheng et al. 2023 on MT-Bench) show that GPT-4 and similar judges exhibit (1) position bias — preferring the first answer shown, (2) verbosity bias — favouring longer answers regardless of correctness, and (3) self-enhancement bias — rating outputs from models of the same family higher. Calibration drift and sycophancy are real but are properties of the target model, not the judge. The key insight for senior engineers is that using a single LLM judge without positional swapping, length normalization, or multi-judge consensus will silently skew evaluation results.

Chain-of-thought (CoT) prompting consistently improves reasoning on multi-step arithmetic and symbolic tasks. However, it does NOT reliably improve — and can sometimes harm — performance on tasks that require only simple factual recall (where extra reasoning steps add noise), highly creative open-ended generation (where structured thinking can constrain diversity), or tasks where the model lacks the background knowledge needed to reason correctly (garbage-in, garbage-out). Retrieval-augmented generation addresses knowledge gaps; CoT alone does not. The key insight is that CoT helps when intermediate steps genuinely decompose a hard problem, not when the bottleneck is elsewhere.

When evaluating build-vs-buy for an LLM-powered feature in production, total cost of ownership must account for hidden operational costs on the build side (GPU infrastructure, MLOps tooling, on-call burden, model versioning) and hidden risks on the buy side (vendor lock-in, data-privacy contractual obligations, latency SLA mismatches, and unpredictable API pricing changes). Options A and C each capture only one dimension. Option D is wrong because open-source weights still require substantial infra and ops investment. Option B correctly identifies that neither pure build nor pure buy eliminates risk, and that the decision must be framed around long-term TCO including ops, data governance constraints, and switching costs — the canonical senior AI engineering framing.

Faithfulness (a), attribution/citation recall (c), and NLI-based entailment (e) all directly ask: 'Is the model's claim supported by the retrieved context?' — the core definition of groundedness. ROUGE-L (b) measures lexical overlap against a reference answer, not against the retrieved context, so it detects recall-style quality, not hallucination relative to sources. Perplexity (d) is a language-modelling metric that has no connection to retrieval grounding whatsoever. A system can have low perplexity and still hallucinate facts not in any retrieved passage.

The Model Context Protocol (MCP) defines a strict client-server architecture where the host application (e.g., an IDE or chat client) spawns or connects to MCP servers. The LLM itself never directly calls MCP servers; instead, the MCP client (embedded in the host) mediates all communication. Tool results flow back to the host, which appends them to the context before re-querying the model. Option C is incorrect because MCP servers expose capabilities via JSON-RPC 2.0, not REST. Option D is incorrect because MCP servers are stateful — they maintain a session with the client. Option A is the only accurate statement.

When an LLM is given a set of tools, it returns a structured 'tool_call' object (or equivalent) rather than a plain text answer when it decides a tool should be invoked. The caller is then responsible for executing the tool, collecting the result, and passing it back to the model as a 'tool' role message. The model does NOT execute the tool itself, and it does not simply embed the result inline — the conversation must include a follow-up turn with the tool's output before the model produces the final answer. This round-trip is fundamental to how OpenAI-style function/tool calling works.

Paged attention's block table design deliberately makes KV pages non-contiguous, so the classic OS external-fragmentation narrative (needing a large contiguous region) does not apply to vLLM's allocator. The real mechanisms behind high TTFT despite apparent available memory are: (a) internal fragmentation — each sequence wastes slots in its last partially filled block, inflating allocated-byte counts without providing usable capacity; and (b) vLLM's watermark-based scheduler policy, which queues or preempts requests when the free-block count drops below a threshold, even if raw byte utilization looks low. Effective mitigations therefore target internal fragmentation (smaller block size) or the admission policy (watermark tuning), both of which are orthogonal levers available at the serving layer without model changes.

Rotary Position Embeddings (RoPE) encode position by rotating query and key vectors in pairs within the attention computation. The key property is that the dot-product between a query at position m and a key at position n depends only on the relative offset (m − n), not their absolute positions — this emerges naturally from the rotation mathematics. This is why RoPE-based models generalize better to sequence lengths beyond their training context: the relative distances are inherently represented. ALiBi (Attention with Linear Biases) achieves a similar effect differently — it subtracts a linear position bias directly on attention logits rather than modifying Q/K vectors. The critical distinction is that RoPE operates in the Q/K embedding space, while ALiBi operates on the attention score matrix post-QK dot-product. Extended context methods like YaRN and LongRoPE scale the RoPE frequency base (theta) — NOT the embedding dimension — to accommodate longer sequences without full fine-tuning.

Hybrid search (dense + BM25 + RRF) improves recall and precision by combining lexical and semantic signals — this directly affects which chunks are retrieved. Query rewriting and HyDE improve retrieval by making the query embedding closer to relevant document embeddings in vector space. Re-ranking with a cross-encoder is a post-retrieval step that re-scores candidates for relevance before passing them to the LLM — still part of the retrieval quality pipeline. Increasing LLM temperature affects generation diversity, not retrieval. Reducing the context window affects inference cost/latency, not retrieval quality.

Cross-encoder rerankers (a) re-score each candidate passage with full query–passage attention, directly targeting precision by surfacing only the most relevant chunks for the LLM context. Metadata pre-filtering (c) narrows the candidate pool to semantically plausible documents before vector search, eliminating irrelevant domains entirely—assuming the filter criteria are aligned with the query's intent so that relevant documents are not accidentally excluded. A similarity score threshold (e) acts as a precision gate, discarding low-confidence passages from the final context sent to the LLM—though thresholds must be tuned per corpus since bi-encoder scores are not well-calibrated across queries. Larger chunks (b) reduce precision by packing more off-topic content into each passage, making the problem worse rather than better. HyDE (d) generates a hypothetical document to improve embedding alignment for underspecified queries, targeting recall rather than precision; it does not filter irrelevant passages and may even surface additional false positives if the synthetic document drifts from the true answer.

temperature scales the logits before sampling; at 0 the model becomes (near) greedy — it picks the highest-probability token at each step — which is what you want for deterministic, reproducible extraction. Note the honest caveat: even at temperature 0 you can still see small variation in practice from non-deterministic floating-point reduction order on GPUs, batching, and model updates, so treat it as low variance, not a hard guarantee. Raising temperature (a) does the opposite — it flattens the distribution and increases randomness. Raising top_p (c) widens the nucleus of candidate tokens, again adding variability rather than removing it. frequency_penalty (d) only discourages token repetition; it changes what is generated but does nothing for determinism.

Cosine similarity is the cosine of the angle between two vectors: it is the dot product divided by the product of the magnitudes, so it depends only on direction, not length. That is why it works well for embeddings — two passages about the same topic point the same way regardless of how long each text is. Euclidean distance (a) is a different metric that is sensitive to magnitude (though on length-normalized vectors the two ranking orders coincide). "Shared dimensions" (c) is not a real similarity measure for dense embeddings, whose dimensions are not independently interpretable. Token overlap (d) describes lexical/keyword matching (e.g. BM25), which is exactly what dense embeddings are meant to go beyond.

Provider structured-output features (e.g. JSON Schema-constrained decoding) restrict the token sampler at each step to tokens that keep the output valid against the schema, so the result is guaranteed-parseable and schema-conformant by construction. A free-text instruction (a) is best-effort: the model can still emit prose, trailing commas, or extra keys. Temperature 0 (c) only reduces variability — a deterministic output can be deterministically malformed. Retry-on-error (d) is a reasonable fallback and improves reliability, but it adds latency and cost and still has no hard guarantee on any single attempt; constrained decoding removes the failure mode at the source.

This is prompt injection: an LLM processes its entire context as one token stream and has no robust, built-in boundary between trusted instructions and untrusted data, so adversarial text embedded in fetched content can hijack behavior. The defense is architectural — keep untrusted content clearly delimited, apply least-privilege to tools, and require human approval or hard policy checks for dangerous actions like sending email or exfiltrating secrets — not a single magic setting. Temperature (a) controls randomness, not whether instructions are followed. Lengthening the system prompt (c) does not create a real trust boundary; a sufficiently crafted injection can still override it. Embedding dimensionality (d) is about retrieval quality and is unrelated to the model obeying injected commands.

Prompt caching stores the processed key/value state for a stable prefix (your unchanging system prompt + policy doc); on later requests the model reuses it instead of re-processing those tokens, which lowers input cost and time-to-first-token while leaving outputs identical — ideal when a large prefix is constant and only the tail varies. Changing temperature (b) affects randomness, not cost. Retrieving the system prompt via vector search (c) adds machinery and a retrieval step to fetch text you already have verbatim — pure overhead. Raising max_tokens (d) only sets an upper bound on output length; it does not reduce the cost of re-reading the same input every call and can increase cost if it lets responses grow.

For open-ended generation, LLM-as-judge over a fixed, versioned eval set scales and correlates reasonably with human judgment when you give the judge a clear rubric and, ideally, a reference answer. The essential caveat is that the judge is itself a fallible model — it shows known biases (position bias, verbosity/length bias, self-preference) — so you must validate it against a sample of human labels and re-check when you change the judge model. Exact match (a) fails by design for free-form text where many wordings are equally correct. BLEU against a single reference (c) was built for machine translation and penalizes valid paraphrases; its weakness is poor correlation with quality on summaries, not speed. Reading every output manually (d) does not scale to weekly iteration; spot-checking belongs alongside an automated eval, not as the primary gate.

The same request can yield different outputs across time — sampling is non-deterministic, a model alias can be repointed to a new version, and retrieval depends on an index that changes — so the only way to replay a past response is to have logged the exact inputs that produced it: the resolved prompt (post-templating, post-retrieval), the model and version, the sampling params (temperature/top_p/seed), and the raw completion (b). Throughput (a) and IP/session data (c) are operational and UX metrics that never let you re-run a single call. Option (d) throws away the input — and the model is not stateless with respect to its prompt: the prompt is the input, so without it there is nothing to reproduce.

A golden set is a versioned, labeled dataset of representative inputs with known-good expectations; running both prompt versions against it and comparing scores catches regressions before they ship and localizes which cases changed (c). Watching the live thumbs-down rate (a) is post-hoc, slow, and noisy — it only tells you something broke after real users hit it, and can't isolate the regressed cases. Eyeballing a handful (b) doesn't cover the long tail you were already passing. Temperature (d) is unrelated to whether the new prompt is actually better.

Model routing (a cascade) matches each request to the cheapest model that can handle it: a fast classifier sends the easy 90% to a small model and escalates only the hard 10% to the expensive one, so you pay the premium where it actually buys quality (d). Small-model-for-everything (a) sacrifices the 10% that needed the big model. Capping max_tokens (b) just truncates outputs — it hurts exactly the open-ended cases that need room. Caching (c) only helps repeated inputs and does nothing for the volume of distinct queries.

Classic RAG is a static pipeline: every query triggers exactly one retrieval, the top-k chunks are stuffed into the prompt, and the model generates. Agentic RAG puts retrieval under the model's control (b) — retrieval (often across several indexes/tools) becomes something the agent can invoke zero, one, or many times, formulating and reformulating the query, decomposing multi-hop questions, judging whether the retrieved context is sufficient and re-querying if not, then deciding when to stop and answer. It is not about memorizing the corpus into weights (a), streaming (c), or a bigger embedding model (d). The cost of the flexibility is more LLM calls — higher latency, cost, and non-determinism — so it pays off on multi-hop or ambiguous queries rather than simple single-fact lookups.

RAGAS-style frameworks score the two stages separately (a). Retrieval metrics judge the chunks the retriever returned: context precision (are the retrieved/ranked passages actually relevant — signal vs. noise) and context recall (did retrieval fetch all the passages needed to answer, typically checked against a reference answer). Generation metrics judge what the model did with that context: faithfulness (is every claim in the answer supported by the retrieved context — grounded, not hallucinated) and answer relevance (does the answer actually address the question). The point of the split is attribution: a wrong answer might be retrieval's fault (the needed passage was never fetched → low context recall) or generation's fault (the passage was present but the model ignored or contradicted it → low faithfulness) — a single end-to-end score can't tell you which.

Privacy and residency are enforced by the data path, not by the model's behavior. Redacting/tokenizing PII before it is embedded, indexed, or logged (a) keeps raw identifiers out of every downstream store. A zero-retention, no-training, in-region endpoint (b) is what actually satisfies the residency and 'no third-party training' clauses contractually. Per-user retrieval scoping (d) stops the system from surfacing tickets a user shouldn't see. Lowering temperature (c) is a sampling knob with no bearing on data handling — it neither redacts data nor controls where it is processed. A system-prompt instruction (e) is bypassable (prompt injection) and addresses neither retention nor residency, so it cannot be the primary safeguard.

The system prompt is the instruction contract every conversation inherits. Defining persona and scope (a) establishes what the model is for so it can apply consistent judgment. Specifying out-of-scope behaviour (b) prevents the model from improvising unhelpfully when a query is outside its remit. In-context examples in the system prompt (c) are a form of few-shot instruction and reliably shape tone and format. Stating the output format (e) reduces variance and surprises. Omitting constraints (d) is the opposite of good practice: without guardrails the model defaults to broad open-ended behaviour, which is rarely appropriate for a product assistant.

Few-shot prompting has well-documented sensitivities. Recency bias (a) is real — models over-weight the distribution of the last few examples, so shuffling or balancing order matters. Representativeness (b) determines whether the model generalises correctly; examples from an unrelated domain can mislead. Format demonstration (d) is an underappreciated effect: examples silently teach schema, length, and punctuation. Adding off-topic examples on a task the model already handles well (e) injects noise that can hurt accuracy. The claim that more examples always help (c) is false: beyond a model-dependent saturation point, additional examples consume context budget without benefit and can crowd out the actual query.

The context window bounds the total tokens the model attends to: system prompt + retrieved context + conversation history + the tokens it generates. If you stuff the input close to the limit, you starve the output and risk truncated completions. Budget explicitly — reserve headroom for max_tokens of output, and remember that long outputs cost both latency and window space.

Chunk size is a retrieval precision vs. recall trade-off, not a monotone dial. Very large chunks mean each chunk covers more ground, so retrieval is more likely to include the relevant sentence — but it also stuffs the context window with a lot of irrelevant surrounding text, which dilutes the signal, wastes tokens, and can cause the model to lose or contradict the key fact buried in a dense passage. Very small chunks maximise precision but risk splitting the supporting evidence across chunks that may not all rank highly. The right chunk size depends on document structure, embedding model characteristics, and query type. Common practice is to experiment across chunk sizes with a retrieval eval (e.g. context precision and recall from RAGAS) before picking a value — not to default to bigger.

Streaming is purely a latency-perception win: tokens are flushed incrementally so the user gets immediate feedback (low time-to-first-token), which matters a lot for long generations. It does not reduce total generation time, total tokens, or cost — those are fixed by the output length. It does add engineering cost: you must handle partial output, parse incrementally, and deal with mid-stream errors and cancellation.

Cosine similarity measures the angle between vectors, so it is invariant to vector length — useful because embedding magnitude often carries little semantic meaning. When embeddings are L2-normalized, cosine similarity, dot product, and (inverse) Euclidean distance rank results identically, which is why many vector stores normalize on ingest and then use a plain dot product internally.