AI Engineering Interview Questions: RAG Basics
Reviewed by Mark Dickie · Last updated
Retrieval-Augmented Generation (RAG) is a technique where a language model is grounded with documents retrieved from an external knowledge base at inference time, rather than relying solely on its pretrained parameters. For an AI engineering interview, you should understand the end-to-end pipeline: how text is chunked and embedded, how a vector store is queried for relevant passages, and how those passages are fed into the prompt context. Interviewers also expect you to reason about failure modes — stale indexes, poor chunk boundaries, retrieval that misses the right document — and to name concrete metrics for measuring retrieval and generation quality.
What does a RAG interview actually test?
Most RAG questions fall into four areas. Here is how they map to what you should prepare:
| Area | What gets asked | What you should know |
|---|---|---|
| Chunking & indexing | How to split documents, trade-offs in chunk size | Token-based vs. semantic splitting, overlap windows, how chunk size affects recall |
| Embeddings & retrieval | Distance metrics, top-k selection, re-ranking | Cosine similarity vs. L2, approximate nearest neighbor indexes (HNSW, IVF), cross-encoder re-ranking |
| Prompt construction | How context is assembled and ordered | Context window limits, lost-in-the-middle effects, system prompt vs. retrieved context separation |
| Evaluation | How to measure RAG quality | Faithfulness, answer relevance, context precision/recall, frameworks like RAGAS or Trulens |
How should I structure a basic RAG pipeline?
A minimal RAG system follows these stages, and you should be able to whiteboard each one:
- Ingest — Load source documents (PDFs, HTML, markdown) and split them into chunks small enough to fit in a context window with room for the model's response.
- Embed — Pass each chunk through an embedding model to produce a fixed-dimensional vector.
- Store — Insert the vectors into a vector database (Pinecone, Weaviate, pgvector, FAISS) with metadata for filtering.
- Retrieve — At query time, embed the user's question, run a similarity search against the store, and collect the top-k matching chunks.
- Generate — Assemble the retrieved chunks into a prompt with the user's question, then call the LLM to produce an answer grounded in that context.
What are the most common RAG failure modes in interviews?
Interviewers like to probe where RAG breaks. A few concrete ones come up often:
- Chunk boundary problems. If a key fact spans two chunks, neither will retrieve well on its own. Overlap windows and semantic splitting help but don't fully solve it.
- Embedding mismatch. A question phrased as "How do I reset my password?" may not match a chunk titled "Credential Recovery" under cosine similarity. Query rewriting or hyde (hypothetical document embeddings) can bridge that gap.
- Context pollution. Retrieving too many chunks dilutes the signal. The model may hallucinate or pull from the wrong passage. Tuning top-k and adding a re-ranking step is the standard fix.
- Stale data. If the source documents update but the index doesn't, the model answers from outdated information. You need an ingestion pipeline with incremental or scheduled re-indexing.
The quiz below covers these topics and more — work through it to find where your gaps are before the real interview.
Key facts
- Tarmac's AI Engineering interview questions cover 24 questions at difficulty 1–5 of 5.
- Tarmac tracked 4,175 job postings asking for AI Engineering in August 2026.
- Roles asking for AI Engineering advertise a median base salary of US$182,450, across 806 job postings as of August 2026.
- Tarmac last reviewed these AI Engineering interview questions on 31 August 2026.
At a glance
| Questions | 24 |
|---|---|
| Difficulty | 1–5 of 5 |
| Formats | Multiple choice, True / false, Ordering, Fill in the blank, Multiple answer, Short answer, Code output, Design exercise |
What you'll review
- rag basics
- rag
- chunking
- retrieval quality
- agentic rag
Practice questions
Try one before you open the answer. Pick an option and press Check; it's marked on the spot.
AI Engineering/rag/rag-basics
In a Retrieval-Augmented Generation (RAG) pipeline, what is the primary purpose of the retrieval step?#
Options
Show answer
The retrieval step fetches relevant documents or passages from an external knowledge source (such as a vector database) and injects them as context into the LLM's prompt. This grounds the model's response in up-to-date or domain-specific information without requiring expensive fine-tuning of the model weights.
In RAG, the retrieval step queries an external knowledge store (e.g., a vector database) using the user's query to fetch semantically relevant documents. Those documents are then injected into the LLM's context window so the model can ground its answer in up-to-date or domain-specific information. The other options describe fine-tuning, reranking, and query compression — separate concepts unrelated to the core retrieval step.
AI Engineering/rag/rag-basics
In a basic RAG system, user queries are converted into vector embeddings and compared against pre-computed embeddings of documents using similarity search (e.g., cosine similarity) to identify the most relevant passages.#
Options
Show answer
True. In a basic RAG system, both documents and user queries are converted into vector embeddings using the same embedding model. Cosine similarity (or a similar metric) is computed between the query vector and each document vector, and the top-k most similar documents are retrieved to provide relevant context to the LLM.
This is a core mechanic of RAG. Both documents (at index time) and the incoming query (at query time) are passed through the same embedding model to produce dense vector representations. A similarity metric such as cosine similarity or dot product is then used to rank documents by relevance, and the top-k results are retrieved to augment the prompt.
AI Engineering/rag/rag-basics
Arrange the following steps of a basic RAG pipeline in the correct order, from start to finish.#
Put these in order
Show answer
The correct order is: (1) Split source documents into chunks → (2) Embed each chunk and store in a vector database → (3) Receive the user's query → (4) Embed the query and perform similarity search → (5) Pass retrieved chunks + query to the LLM to generate an answer. The first two steps form the offline indexing phase; the last three form the online query phase.
A RAG pipeline has two distinct phases. The indexing phase runs offline: (1) source documents are chunked, then (2) each chunk is embedded and stored in a vector database. The query phase runs at inference time: (3) the user's query arrives, (4) it is embedded and used to retrieve the most similar chunks via similarity search, and finally (5) those chunks are appended to the prompt so the LLM can generate a grounded response.
AI Engineering/rag
The acronym RAG stands for _____-Augmented Generation, a technique that supplements a language model's parametric knowledge with externally retrieved information at inference time.#
Show answer
The acronym RAG stands for Retrieval-Augmented Generation, a technique that supplements a language model's parametric knowledge with externally retrieved information at inference time.
RAG stands for Retrieval-Augmented Generation. The retrieval step fetches relevant passages from an external knowledge source (e.g., a vector database) and includes them in the prompt context so the model can ground its answer in up-to-date, source-specific information.
AI Engineering/rag/rag-basics
Which of the following best describes the primary motivation for using a Retrieval-Augmented Generation (RAG) architecture instead of a plain LLM?#
Options
Show answer
The primary motivation for RAG is to ground the LLM's responses in up-to-date or domain-specific knowledge without retraining the model. By retrieving relevant documents at inference time and injecting them into the prompt, RAG gives the model access to information beyond its training cutoff or outside its original training data, making answers more accurate and current.
RAG was specifically designed to address the knowledge-cutoff limitation of LLMs by retrieving up-to-date or domain-specific documents at inference time. It does NOT fine-tune the model weights, does NOT eliminate hallucinations entirely (it reduces them), and does NOT remove the need for a prompt — in fact it adds context to the prompt. The key benefit is grounding the model's response in retrieved, external knowledge.
AI Engineering/rag/rag-basics
During the document indexing phase of a RAG system, which of the following steps are typically performed? Select all that apply.#
Options
Pick every one that applies.
Show answer
During the document indexing phase of a RAG system, three steps are performed: splitting documents into smaller chunks, generating vector embeddings for each chunk via an embedding model, and storing those embeddings along with the source text in a vector database. The LLM is not used during indexing, and similarity search only happens at query time when a user submits a question.
At indexing time in a RAG system, source documents are split into smaller pieces (chunked), each chunk is passed through an embedding model to produce a dense vector, and those vectors are stored in a vector database. The original text of each chunk is also stored alongside the vector so it can be retrieved and placed into the LLM prompt later. The LLM itself is NOT involved during the indexing phase — it is only used at query time to generate the final answer.
AI Engineering/rag/rag-basics
What is retrieval-augmented generation (RAG) and what problem does it solve over just prompting the model?#
Show answer
RAG retrieves relevant documents from an external knowledge source (usually via a vector search over embeddings) and injects them into the prompt as context, so the model answers grounded in that retrieved data. It solves the problem that a model's parametric knowledge is frozen at training time and can hallucinate — RAG lets you supply fresh, private, or domain-specific facts at query time without retraining, and lets you cite sources.
RAG decouples knowledge from weights: the model stays general, while a retrieval layer (embed the query, search a vector store, pass the top hits as context) supplies current and proprietary facts. The big wins are freshness, the ability to cite sources, and reduced hallucination — but retrieval quality becomes the bottleneck, so chunking, the embedding model, and reranking matter as much as the LLM itself.
AI Engineering/rag
In a basic RAG (Retrieval-Augmented Generation) pipeline, source documents are first split into smaller passages through a process called _____. Each passage is then converted into a numerical vector using an _____ model and stored in a vector database so that relevant passages can be found via similarity search at query time.#
Show answer
In a basic RAG (Retrieval-Augmented Generation) pipeline, source documents are first split into smaller passages through a process called chunking. Each passage is then converted into a numerical vector using an embedding model and stored in a vector database so that relevant passages can be found via similarity search at query time.
The two foundational preprocessing steps in a RAG pipeline are chunking (dividing documents into manageable passages) and embedding (converting each passage into a dense vector). These vectors are what the vector database indexes and compares against the query vector at retrieval time.
AI Engineering/rag
In a RAG system, cosine similarity is commonly used to rank document chunks against a user query. Trace the following Python code that computes cosine similarity between a document vector and a query vector. What is printed?#
import numpy as np
def cosine_similarity(a, b):
return np.dot(a, b) / (np.linalg.norm(a) * np.linalg.norm(b))
doc = np.array([3, 4, 0])
query = np.array([3, 0, 0])
print(round(cosine_similarity(doc, query), 4))Show answer
0.6
The dot product of [3,4,0] and [3,0,0] is 33 + 40 + 0*0 = 9. The norm of the doc vector is sqrt(3²+4²+0²) = sqrt(25) = 5, and the norm of the query vector is sqrt(3²+0²+0²) = 3. Cosine similarity = 9 / (5 * 3) = 9/15 = 0.6. Rounding to 4 decimal places gives 0.6.
AI Engineering/rag/rag-basics
Arrange the following steps of a basic RAG pipeline in the correct execution order, from document ingestion through answer generation:#
Put these in order
Show answer
The correct order is: (1) Split documents into chunks → (2) Embed chunks and store vectors in a vector database → (3) Receive a user query → (4) Embed the user query → (5) Retrieve the top-k most similar chunks → (6) Pass retrieved chunks + query to the LLM and return the answer. Steps 1–2 are an offline indexing phase; steps 3–6 happen at inference time for every user request.
A RAG pipeline has two distinct phases. The indexing phase (offline) consists of: (1) chunking documents into manageable pieces, then (2) embedding those chunks and storing the vectors in a vector database. The inference/retrieval phase (online, per query) consists of: (3) receiving the user query, (4) embedding that query using the same embedding model, (5) querying the vector store for the top-k nearest-neighbor chunks, and finally (6) constructing a prompt with the retrieved context and the query, then calling the LLM to generate the answer.
AI Engineering/rag/rag-basics
Your support bot must answer from a knowledge base that changes daily and must cite the source document for each answer. Which approach is the most appropriate primary strategy?#
Options
Show answer
Use retrieval-augmented generation (RAG): retrieve the relevant documents at query time and pass them into the context. RAG is the right default when knowledge is large, changes frequently, and answers need attribution, because updates are just re-indexing and you can cite the chunks you retrieved. Nightly fine-tuning and one-time LoRA adapters bake facts into weights that go stale and cannot cite sources; stuffing the whole knowledge base into the prompt blows the context window and dilutes quality.
RAG is the right default when knowledge is large, changes frequently, and answers need attribution: you index the documents, retrieve the relevant chunks per query, and the model answers from them — so updates are just re-indexing, and you can cite the chunks you retrieved. Nightly fine-tuning (a) is slow, expensive, hard to attribute (the model can't reliably cite which training example produced an answer), and it bakes facts into weights where they go stale between runs. Stuffing the whole KB into the system prompt (c) blows the context window and cost, and degrades quality as irrelevant text dilutes the signal. A one-time LoRA on a snapshot (d) is immediately stale for daily-changing data and still can't cite sources. Fine-tuning earns its place for behavior/format/tone, not volatile facts.
AI Engineering/rag/chunking
This is a fixed-size sliding-window chunker with overlap (the pattern used to split documents before embedding). It prints the number of chunks produced. What does it print?#
def chunk(text, size, overlap):
step = size - overlap
chunks = []
i = 0
while i < len(text):
chunks.append(text[i:i + size])
i += step
return chunks
print(len(chunk("abcdefghij", 4, 1)))Show answer
4
The string is 10 characters and the stride is size - overlap = 4 - 1 = 3. Windows start at indices 0, 3, 6, 9, yielding "abcd", "defg", "ghij", and a final "j" — four chunks. Note the trailing single-character chunk: a naive while i < len(text) loop emits a runt window at the end. Production chunkers usually drop or merge tiny tail chunks, since a 1-token chunk embeds poorly and pollutes retrieval.
AI Engineering/rag/rag-basics
Order the stages of building and serving a retrieval-augmented generation (RAG) pipeline, from raw documents to a grounded answer.#
Put these in order
Show answer
Ingestion runs first, then serving at query time:
- Chunk the documents into retrievable passages
- Embed each chunk into a vector
- Store the vectors (and metadata) in a vector index
- Retrieve the top-k chunks for the user's query
- Generate the answer with the retrieved context in the prompt
Ingestion runs first — documents are chunked, each chunk is embedded, and the vectors plus metadata are stored in the index. At query time you retrieve the most similar chunks for the question, then generate the answer with that context injected into the prompt. (A reranking step often sits between retrieve and generate, but the ingest-then-serve order is fixed.)
AI Engineering/rag
A minimal RAG retrieval step computes cosine similarity between a query vector and a list of document vectors, then returns the top-k document texts. Trace the code below and determine what it prints.#
import math
def cosine_sim(a, b):
dot = sum(x*y for x, y in zip(a, b))
na = math.sqrt(sum(x*x for x in a))
nb = math.sqrt(sum(x*x for x in b))
return dot / (na * nb)
def retrieve(query, docs, k=2):
scored = [(cosine_sim(query, d['vec']), d['text']) for d in docs]
scored.sort(key=lambda x: x[0], reverse=True)
return [text for score, text in scored[:k]]
docs = [
{'text': 'alpha', 'vec': [1, 0, 0]},
{'text': 'beta', 'vec': [0, 1, 0]},
{'text': 'gamma', 'vec': [1, 1, 0]},
]
query = [1, 1, 0]
print(retrieve(query, docs, k=2))Show answer
['gamma', 'alpha']
Cosine similarities with query [1,1,0]: gamma = 2/(√2·√2) = 1.0; alpha = 1/(√2·1) ≈ 0.707; beta = 1/(√2·1) ≈ 0.707. Sorting descending puts gamma first, then alpha and beta tie at ≈0.707. Python's sort is stable, so alpha (originally before beta) stays ahead. Taking k=2 yields ['gamma', 'alpha'].
AI Engineering/rag
In a two-stage RAG pipeline, a fast first-stage retriever (such as a bi-encoder) pulls a broad candidate set of N documents, then a _____ model re-scores those candidates and narrows them to the final top-k. The first stage is optimized for _____ (casting a wide net), while the second stage is optimized for _____ (surfacing the most relevant results at the top).#
Show answer
In a two-stage RAG pipeline, a fast first-stage retriever (such as a bi-encoder) pulls a broad candidate set of N documents, then a cross-encoder model re-scores those candidates and narrows them to the final top-k. The first stage is optimized for recall (casting a wide net), while the second stage is optimized for precision (surfacing the most relevant results at the top).
Two-stage retrieval is a standard RAG optimization. The first stage uses a cheap, high-throughput model (bi-encoder or BM25) to maximize recall — ensuring relevant passages enter the candidate pool. The second stage uses a more expensive but more accurate cross-encoder (a reranker) to re-score candidates and maximize precision, so only the most relevant passages reach the LLM's context window.
AI Engineering/rag/rag-basics
In a production Retrieval-Augmented Generation (RAG) pipeline, which of the following techniques directly improve retrieval quality (i.e., the relevance of retrieved chunks), as opposed to improving generation quality or system throughput?#
Options
Pick every one that applies.
Show answer
The techniques that directly improve retrieval quality in a RAG pipeline are: hybrid search with BM25 + dense vectors + RRF, query rewriting / HyDE, and cross-encoder re-ranking. Hybrid search combines lexical and semantic signals; HyDE/query rewriting closes the embedding gap between query and relevant documents; and cross-encoder re-ranking refines the candidate list before generation. Temperature and context-window size affect generation, not retrieval.
Hybrid search (dense + BM25 + RRF) improves recall and precision by combining lexical and semantic signals — this directly affects which chunks are retrieved. Query rewriting and HyDE improve retrieval by making the query embedding closer to relevant document embeddings in vector space. Re-ranking with a cross-encoder is a post-retrieval step that re-scores candidates for relevance before passing them to the LLM — still part of the retrieval quality pipeline. Increasing LLM temperature affects generation diversity, not retrieval. Reducing the context window affects inference cost/latency, not retrieval quality.
AI Engineering/rag/rag-basics
A senior AI engineer is building a RAG pipeline from scratch. Arrange the following steps in the correct indexing-time order — i.e., the sequence that transforms raw source documents into a queryable vector index.#
Put these in order
Show answer
The correct indexing-time order is: Load → Clean → Chunk → Embed → Store. You first parse raw source files into text, then normalize/clean that text, then split it into chunks sized for the embedding model, then convert each chunk to a dense vector, and finally persist those vectors (with metadata) into a vector database for ANN retrieval at query time.
The canonical RAG indexing pipeline proceeds as follows: (1) Load — parse raw files (PDF, HTML, etc.) into raw text; (2) Clean — normalize the text, strip boilerplate, fix encoding; (3) Chunk — split into appropriately-sized segments (with optional overlap) so they fit within the embedding model's context; (4) Embed — run each chunk through an embedding model to produce a dense vector representation; (5) Store — persist the vectors along with chunk text and metadata into a vector database for efficient ANN retrieval at query time.
AI Engineering/rag/retrieval-quality
Your RAG assistant gives confident but wrong answers, and you suspect the passages it pulls into context are the problem. Walk through how you would debug this, from the user's question through to what finally reaches the model.#
Show answer
First make it observable: log and inspect exactly what was retrieved for the failing question. Check whether the right source documents are even in the candidate set the vector store returned — if not, the problem is upstream (poor chunking, a weak or mismatched embedding model, or the document was never indexed). If the right passages are retrieved but ranked below the cutoff, add or tune a reranker and/or raise top-k. Then confirm the selected chunks are actually placed in the final prompt and not truncated out by the context budget. Also sanity-check that the query is embedded with the same model as the index, and try hybrid (keyword + vector) retrieval for questions that hinge on exact terms or IDs. In short, trace question → candidates → rerank → context and find the stage where the right evidence drops out.
Debugging retrieval means making each stage observable and finding where the right evidence falls out. (1) Inspect what was actually retrieved for the failing query — are the correct documents even in the candidate set? If not, the fault is upstream: chunking, the embedding model, or a missing/under-indexed doc. (2) If good passages are retrieved but ranked too low, add/tune a reranker or raise top-k. (3) Confirm the chosen chunks actually land in the final prompt and aren't truncated by the context budget. (4) Verify the query uses the same embedding model as the index, and consider hybrid keyword+vector retrieval for exact-term/ID questions. The mental model is a pipeline — question → candidates → rerank → context — so you isolate the stage that drops the needed passage rather than blaming the generator.
AI Engineering/rag
You are tuning a production RAG pipeline that uses HNSW (Hierarchical Navigable Small World) as the ANN index in your vector database (e.g., FAISS HNSW, Qdrant, pgvector with HNSW). During evaluation you observe that recall@10 is too low but query latency is well within budget. Which HNSW parameter should you increase to directly improve query-time recall at the cost of higher latency?#
Options
Show answer
Increase ef_search. In HNSW, ef_search controls the size of the dynamic candidate list explored during layer traversal at query time — a larger value improves recall by examining more neighbors at the cost of additional distance computations and higher latency. ef_construction and M are build-time parameters that shape the graph structure, and ml governs layer-assignment probability; none of them can be tuned at query time to trade recall for speed.
ef_search (called ef in FAISS) is the parameter that controls how many candidates HNSW explores in the dynamic list during each layer traversal at query time. Increasing it widens the search frontier, improving recall at the cost of more distance computations and higher latency. ef_construction affects index build quality and the resulting graph structure, but once the index is built it does not change query-time behavior. M (max connections per node per layer) also affects graph connectivity but is set at build time; while a low M can limit achievable recall, the parameter you tune at query time to trade recall for latency is ef_search. ml (the level normalization factor) controls the exponential probability distribution for layer assignment and is not a query-time tunable.
AI Engineering/rag
A RAG retrieval pipeline uses Maximal Marginal Relevance (MMR) to diversify retrieved chunks. Trace the following MMR selection code and determine the exact output printed to stdout.#
import numpy as np
def mmr_select(query_sim, doc_sim, k=3, lambda_=0.5):
selected = []
candidates = list(range(len(query_sim)))
for _ in range(k):
best = None
best_score = -1
for d in candidates:
relevance = lambda_ * query_sim[d]
if selected:
redundancy = (1 - lambda_) * max(doc_sim[d][s] for s in selected)
else:
redundancy = 0
score = relevance - redundancy
if score > best_score:
best_score = score
best = d
selected.append(best)
candidates.remove(best)
return selected
query_sim = [0.9, 0.8, 0.3, 0.7]
doc_sim = [
[1.0, 0.6, 0.1, 0.2],
[0.6, 1.0, 0.1, 0.5],
[0.1, 0.1, 1.0, 0.1],
[0.2, 0.5, 0.1, 1.0],
]
print(mmr_select(query_sim, doc_sim, k=3, lambda_=0.5))Show answer
[0, 3, 1]
The MMR score for each candidate is λ·Sim(d,query) − (1−λ)·max_{s∈selected} Sim(d,s). With λ=0.5:
Round 1 (selected=[], no redundancy term): scores are d0=0.45, d1=0.40, d2=0.15, d3=0.35. Select d0. selected=[0].
Round 2 (selected=[0], redundancy = 0.5·doc_sim[d][0]):
- d1: 0.40 − 0.5·0.6 = 0.40−0.30 = 0.10
- d2: 0.15 − 0.5·0.1 = 0.15−0.05 = 0.10
- d3: 0.35 − 0.5·0.2 = 0.35−0.10 = 0.25
Select d3 (highest). selected=[0,3].
Round 3 (selected=[0,3], redundancy = 0.5·max(doc_sim[d][0], doc_sim[d][3])):
- d1: 0.40 − 0.5·max(0.6,0.5) = 0.40−0.30 = 0.10
- d2: 0.15 − 0.5·max(0.1,0.1) = 0.15−0.05 = 0.10
Tie at 0.10; the code uses strict > so the first candidate encountered (d1) wins. Select d1. selected=[0,3,1].
The function returns [0, 3, 1].
AI Engineering/rag/rag-basics
In a RAG system, you observe that increasing top-k beyond 5 consistently hurts end-to-end answer quality (measured by RAGAS faithfulness and answer relevance scores), even though retrieval recall@20 is much higher than recall@5. Explain the specific mechanism responsible for this degradation and name at least two architectural interventions — beyond simply lowering k — that allow the system to benefit from higher recall without the quality penalty.#
Show answer
The degradation is caused by lost-in-the-middle attention dilution: LLMs attend most strongly to content at the beginning and end of the context window; passages injected in the middle of a long context receive proportionally less attention weight. When k grows, the relevant passage may be retrieved but buried among many lower-relevance passages, causing the model to ignore or under-weight it. Additionally, irrelevant retrieved passages introduce noise that can confuse the generator (semantic interference). Two architectural interventions: (1) Cross-encoder re-ranking — after retrieving top-k with a bi-encoder, run a more powerful cross-encoder (e.g., a fine-tuned BERT-style model) to re-score all k passages against the query and keep only the top-m (m << k) for generation, combining high recall with high precision context. (2) Contextual compression / LLM-based filtering — pass each retrieved passage through a compressor (e.g., LLMLingua or a dedicated extractive model) that strips irrelevant sentences before concatenation, reducing total context length while preserving relevant signals. Other valid answers include: fusion-in-decoder architectures, self-RAG with critic scoring, or hierarchical summarisation of retrieved chunks before synthesis.
The 'lost-in-the-middle' phenomenon (documented by Liu et al., 2023) shows LLM performance on multi-document QA degrades when relevant information appears in the middle of the context. More retrieved chunks raise recall but worsen the signal-to-noise ratio and trigger this positional bias. The two canonical fixes are cross-encoder re-ranking (precision after recall) and contextual compression (reduce noise before generation), both of which are standard components in production RAG stacks.
AI Engineering/rag/agentic-rag
What most fundamentally distinguishes agentic RAG from a classic 'retrieve-then-generate' pipeline?#
Options
Show answer
In agentic RAG, retrieval becomes a tool an LLM agent decides whether, when, and how to call — reformulating queries and retrieving iteratively — rather than one fixed retrieval that always runs before generation. Classic RAG fires exactly one retrieval per query, then generates. Agentic RAG instead lets the agent invoke retrieval zero, one, or many times, decompose multi-hop questions, and re-query when context is insufficient. It is not about memorizing the corpus, streaming, or a bigger embedding model.
Classic RAG is a static pipeline: every query triggers exactly one retrieval, the top-k chunks are stuffed into the prompt, and the model generates. Agentic RAG puts retrieval under the model's control (b) — retrieval (often across several indexes/tools) becomes something the agent can invoke zero, one, or many times, formulating and reformulating the query, decomposing multi-hop questions, judging whether the retrieved context is sufficient and re-querying if not, then deciding when to stop and answer. It is not about memorizing the corpus into weights (a), streaming (c), or a bigger embedding model (d). The cost of the flexibility is more LLM calls — higher latency, cost, and non-determinism — so it pays off on multi-hop or ambiguous queries rather than simple single-fact lookups.
AI Engineering/rag
In an HNSW (Hierarchical Navigable Small World) index used for dense vector retrieval in a RAG pipeline, which statement correctly describes the roles of the three key parameters M, ef_construction, and ef_search?#
Options
Show answer
M, ef_construction, and ef_search in HNSW serve distinct roles: M sets the maximum number of graph connections per node per layer, governing memory and navigability; ef_construction sets the candidate list size during index building, governing graph quality and build time; ef_search sets the candidate list size during querying, governing the recall-vs-latency tradeoff. The number of layers is determined probabilistically, not by any of these three parameters.
In HNSW, M is the maximum number of bidirectional connections per node per layer, directly impacting memory usage (more edges = more storage) and graph navigability. ef_construction governs the size of the dynamic candidate list used while inserting nodes during index build—higher values produce a better-quality graph at the cost of slower build times. ef_search governs the same dynamic list during query traversal—higher values improve recall at the cost of higher query latency. Option (b) swaps these roles; (c) and (d) assign each parameter to an incorrect concern. The number of graph layers in HNSW is determined by a probabilistic decay function (exponential level assignment), not by M directly, so option (d) is wrong on that point as well.
AI Engineering/rag
Design a production RAG system for a multinational law firm with the following requirements:#
Show answer
We would build a multi-stage RAG pipeline with the following components:
Chunking & Citation Provenance: Documents are parsed into a hierarchical structure (document → section → paragraph). Each paragraph becomes a chunk with metadata: {doc_id, doc_version, paragraph_id, section_path, jurisdiction, matter_id, client_id, language, effective_date}. Chunks are embedded using a legal-domain-tuned embedding model. At generation time, the LLM is prompted to include [doc_id, paragraph_id] citations for every claim. A post-processing validation step checks that every cited chunk was actually in the retrieved context set and is accessible to the user—if a citation references a chunk not in the retrieved set, the answer is flagged for review.
Access Control: Access control is enforced at the vector store level using metadata pre-filters. Each query includes the user's authorized matter_id and client_id list as filter predicates on the ANN search (supported by Pinecone, Weaviate, Milvus via metadata filtering). We use a single HNSW index with metadata filtering rather than separate indexes per matter, since the matter count is large and many matters share reference documents (e.g., public statutes). Unauthorized documents are never returned by the search—they are excluded by the filter before the ANN traversal produces candidates.
Document Versioning: When a document is updated, new chunks are embedded and inserted with an incremented version number and a new effective_date. Old version chunks are soft-deleted (marked with tombstone=true and superseded_by= new_version) rather than immediately removed, preserving the audit trail. At query time, a metadata filter (tombstone=false AND version=latest) ensures only current versions are retrieved. A background compaction job periodically rebuilds index shards to physically remove tombstoned chunks and reclaim memory.
Scale & Retrieval Quality: At 50M+ documents, we shard the index by jurisdiction (reducing per-shard size and enabling jurisdiction-aware routing—queries specifying a jurisdiction hit only the relevant shard). Retrieval uses hybrid search: dense HNSW retrieval (top-100) fused with BM25 sparse retrieval (top-100) via reciprocal rank fusion. A legal-domain fine-tuned cross-encoder re-ranks the fused top-50 down to the final top-10 chunks passed to the LLM. HNSW parameters (M=32, ef_construction=400, ef_search=64) are tuned against a held-out recall@10 benchmark set of 5,000 annotated legal queries, with ef_search adjusted dynamically based on shard size.
Multi-Jurisdiction & Multilingual: We use a multilingual embedding model (e.g., multilingual-e5-large) so queries in one language retrieve documents in another without a translation step. Jurisdiction is stored as chunk metadata and used as an optional explicit filter when the user specifies a jurisdiction. For cross-jurisdiction comparison queries (e.g., 'Compare force-majeure clauses in NY and German law'), the query is routed to multiple jurisdiction shards in parallel; results from each jurisdiction are fetched separately (top-k per jurisdiction) and merged before cross-encoder re-ranking, ensuring balanced representation from each jurisdiction in the final context.
This rubric evaluates whether the candidate can architect a RAG system that addresses the five hardest production concerns in a legal-domain setting: citation provenance, pre-retrieval access control, document versioning, retrieval quality at scale, and cross-lingual/multi-jurisdiction handling. Each criterion tests a distinct architectural decision; an answer that omits any one leaves a critical production gap. The sample answer satisfies all five: hierarchical chunking with paragraph metadata (c1), metadata pre-filtering at the vector store (c2), version increment with tombstoning and query-time filtering (c3), hybrid search + cross-encoder re-ranking + jurisdiction sharding + HNSW parameter tuning (c4), and multilingual embeddings with parallel multi-jurisdiction routing (c5).
Related interview questions
Job market
See ai-engineering salaries and hiring demand from live job postings.
Practise these until they stick
That's every question we hold on this topic, and the page marks what you pick. What it can't do is remember. A free account keeps every answer, and what you miss comes back until it's right: after a day, then at longer gaps.
Free · the whole bank · 100 marked answers per 30 days · written feedback on the paid plan