RAG Retrieval Quality Interview Questions

Context Recall measures whether the retrieved chunks contain all the pieces of information required to answer the query — i.e., how much of the ground-truth answer is covered by the retrieval set. BLEU measures text similarity between generated and reference text, perplexity measures a language model's uncertainty, and throughput is a latency/performance metric, none of which directly evaluate retrieval coverage.

Increasing top-k does not always improve answer quality. While a higher k increases the chance of retrieving relevant chunks (improving recall), it also introduces more irrelevant or noisy chunks into the context window. This can confuse the language model, dilute the relevant signal, and even cause the model to ignore key information — a phenomenon sometimes called 'lost in the middle'. The optimal k balances recall and precision.

Precision and recall are the two fundamental retrieval quality metrics. Precision = (relevant retrieved) / (total retrieved), telling you how much of what was fetched is actually useful. Recall = (relevant retrieved) / (total relevant), telling you how much of the available useful information was found. A good RAG system needs both: low precision means noisy context, low recall means missing key information.

Increasing chunk size is a trade-off, not a guaranteed improvement. Larger chunks can dilute the dense vector representation, causing the embedding to capture a broad mix of topics rather than the specific content relevant to a query. This lowers retrieval precision: the retrieved chunk may contain the answer but also a lot of irrelevant content, reducing the signal-to-noise ratio. Smaller, focused chunks often yield higher precision, while a hybrid approach (e.g., 'small-to-big' retrieval or hierarchical chunking) is commonly used to balance precision and context.

Each stage strictly depends on the output of the one before it. You must first ingest raw documents (A) before you can split them (B). You can only embed chunk text after chunks exist (C). The vectors, text, and metadata are upserted into the vector store as a single operation once the embeddings are produced (D). Finally, the ANN index is built on the already-stored vectors so that similarity search can be performed at query time (E). No pair of adjacent steps can be reordered without breaking a hard data dependency.

Cross-encoder rerankers (a) re-score each candidate passage with full query–passage attention, directly targeting precision by surfacing only the most relevant chunks for the LLM context. Metadata pre-filtering (c) narrows the candidate pool to semantically plausible documents before vector search, eliminating irrelevant domains entirely—assuming the filter criteria are aligned with the query's intent so that relevant documents are not accidentally excluded. A similarity score threshold (e) acts as a precision gate, discarding low-confidence passages from the final context sent to the LLM—though thresholds must be tuned per corpus since bi-encoder scores are not well-calibrated across queries. Larger chunks (b) reduce precision by packing more off-topic content into each passage, making the problem worse rather than better. HyDE (d) generates a hypothetical document to improve embedding alignment for underspecified queries, targeting recall rather than precision; it does not filter irrelevant passages and may even surface additional false positives if the synthetic document drifts from the true answer.

High Recall@10 and MRR@10 confirm that the correct passage is being retrieved and ranked well, so the failure is not a coverage or ranking problem. The faithfulness drop after an embedding model change strongly implies the new model retrieves semantically adjacent but factually divergent passages — a qualitative mismatch invisible to rank-based metrics. Good answers identify this nuance and propose attribution-level or qualitative inspection steps rather than simply reverting the model blindly.

Line 9 embeds the query with text-embedding-3-small while the documents were indexed with text-embedding-3-large (line 1/4). Different embedding models produce vectors in different, incompatible spaces — they have different dimensionalities and geometry, so cosine similarity between a small-model query vector and large-model document vectors is meaningless. The query must be embedded with the exact same model (and version) used to build the index. The fix is to reuse DOC_MODEL on line 9.

A cross-encoder reranker (option A) scores each of the top-100 candidate passages against the full query jointly, dramatically improving ranking precision without changing recall@100 because the recall set is untouched. Fine-tuning the bi-encoder with hard negatives on in-domain data (option B) directly teaches the embedding model to separate relevant from near-miss documents, lifting both precision and recall. Option C reduces ef_search, which shrinks the ANN candidate set and hurts recall@100 — the opposite of desired. HyDE (option D) can improve recall for ambiguous queries but does not selectively improve precision and may introduce noise from hallucinated hypothetical docs. Increasing chunk size (option E) often hurts precision because a large chunk is more likely to contain irrelevant passages that dilute relevance scoring.

A high MRR means the relevant document is found, but faithfulness can still fail for at least two independent reasons. First, LLMs exhibit 'lost-in-the-middle' behaviour (Liu et al., 2023): when multiple retrieved chunks are concatenated, information not at the very beginning or end is often ignored, causing the model to generate unfaithful completions even when the right chunk is present. Second, retrieval relevance is a coarser signal than answer sufficiency — a chunk can be on-topic but lack the exact supporting evidence, leading the LLM to hallucinate the missing detail. Both are diagnosable with targeted ablations and decomposed metrics (e.g., RAGAS splits context relevance from answer faithfulness and groundedness).