AI Engineering Interview Questions: Evaluation & Safety

High accuracy on a single test set does not guarantee production safety. The test set may not be representative of real-world inputs, the classifier may have blind spots for underrepresented demographic groups, and accuracy alone can mask severe class-imbalance issues (e.g., flagging very few inputs as toxic). Additional evaluation — such as subgroup fairness analysis, adversarial testing, and monitoring for drift — is required before deployment.

Recall = TP / (TP + FN), measuring the proportion of actual harmful prompts that the classifier correctly blocks. When false negatives are the dominant cost, maximizing recall directly minimizes the rate of harmful prompts that slip through. Precision focuses on false positives (lower priority here), accuracy is dominated by the large class of benign prompts and can mask poor harmful-prompt detection, and F1 weights precision and recall equally rather than reflecting the asymmetric cost structure.

Red-teaming is the standard term for adversarial probing of LLMs. Testers (human or automated) attempt to bypass safety guardrails using techniques like jailbreaks, prompt injections, and edge-case inputs. The goal is to surface failure modes before real users encounter them, then use the findings to improve alignment, filters, or system prompts. This contrasts with benign evaluation, which tests standard capability and quality on representative — not adversarial — inputs.

A held-out, independently curated test set with novel attack vectors avoids both data contamination and overfitting to the training distribution, giving a genuine estimate of generalization to unseen harmful inputs. Reusing training data means the model has already been optimized on those exact prompts — good performance there reflects memorization, not robustness. Using a well-known public safety benchmark without checking whether those prompts appeared in the training data risks benchmark contamination: if the public benchmark was seen during pre-training or fine-tuning, scores are inflated and regressions on truly novel prompts would be missed. Evaluating only on benign prompts and measuring the false-refusal rate measures over-refusal of benign requests, which is a complementary safety metric but tells you nothing about whether harmful requests are actually refused.

Red-teaming is the standard term for adversarially probing an LLM to surface safety failures before deployment. It borrows the cybersecurity concept of a 'red team' that simulates attacks. In practice, red-teams craft jailbreaks, prompt injections, and socially engineered requests to measure attack success rate (ASR) and identify guardrail weaknesses. The accept list covers the hyphenated, spaced, and standalone variants since all are in common use.

LLM-as-a-judge is a popular evaluation technique, but it has well-documented systematic biases. Studies (e.g., from Zheng et al. 2023 on MT-Bench) show that GPT-4 and similar judges exhibit (1) position bias — preferring the first answer shown, (2) verbosity bias — favouring longer answers regardless of correctness, and (3) self-enhancement bias — rating outputs from models of the same family higher. Calibration drift and sycophancy are real but are properties of the target model, not the judge. The key insight for senior engineers is that using a single LLM judge without positional swapping, length normalization, or multi-judge consensus will silently skew evaluation results.

The FIND_THE_BUG here targets a subtle but critical mistake: the moderate function calls openai.Moderation.create but then checks results[0]['flagged'] using dict-style access on a Pydantic model object returned by the newer OpenAI Python SDK (v1+). The correct attribute access is results.results[0].flagged (dot notation on the model object). Additionally, the guard short-circuits on None return but the caller never checks for None, meaning a flagged message silently disappears without raising an error or returning a meaningful response — but the primary code-level bug is the results[0]['flagged'] dict access which raises a TypeError at runtime with the v1 SDK. The buggy line is line 6.

This question tests deep knowledge of layered guardrail architecture. Input guardrails run before the LLM call to block malicious or off-topic prompts. Output guardrails run after the LLM response to catch hallucinations, PII leakage, or policy violations before delivery to the user. A correct defence-in-depth pipeline therefore needs both layers. Relying only on input guardrails misses prompt-injection-induced unsafe outputs; relying only on output guardrails still allows prompt injection to reach the model, wasting compute and risking data exfiltration. Grounding checks (RAG relevance) are an output-side concern. Rate limiting is an infrastructure concern orthogonal to content safety. A well-designed system applies: (1) input validation → (2) LLM call → (3) output validation, with each stage having independent failure modes.

Self-preference bias (documented by Panickssery et al., 2024, and others) is the tendency for an LLM judge to prefer outputs produced by models in its own family, because shared training data and stylistic patterns make same-family outputs feel more natural or higher-quality to the judge. The scenario controls for the two most common confounds — position bias and length bias — by stating there is no significant difference in ordering or length. Calibration drift refers to threshold instability across a batch, which is unrelated to the model-identity correlation described. The pattern of same-family leniency and cross-family strictness is the hallmark signature of self-preference bias.

The judge model sharing architectural lineage and training data with the model under test is correct: when the judge and the model-under-test share training data and architecture, they tend to have correlated failure modes — content that the model produces unsafely is also content the judge fails to flag as unsafe. This is a well-documented concern in LLM-as-judge safety evaluation. The eval prompt distribution being static and curated while production inputs exhibit distribution shift is correct: static curated eval sets cannot cover the full distribution of production inputs, and novel adversarial techniques (multi-turn jailbreaks, encoding tricks, persona-based attacks) that emerge after the eval set was created will not be caught. The harness evaluating only single-turn interactions is correct: single-turn evaluation misses multi-turn attack patterns where each individual turn appears benign but the conversation trajectory progressively erodes the model's safety guardrails — a known limitation of single-turn safety benchmarks. Setting the judge model's temperature to 0 for deterministic scoring is incorrect: lowering the judge's temperature to 0 makes its scoring deterministic but does not guarantee correctness. A temperature-0 judge can still systematically fail to identify unsafe content due to capability gaps, prompt-engineering flaws, or shared blind spots with the model under test — temperature controls sampling stochasticity, not accuracy. The eval set being drawn from public benchmarks that the model under test may have encountered during pretraining is correct: benchmark contamination is a well-recognized problem. If the model saw the exact safety test prompts during pretraining, it may learn to refuse those specific strings without generalizing the underlying safety principle, causing inflated eval scores that don't reflect robustness to paraphrased or restructured variants of the same attack.