DevOps interview questions

Canary releasing exists precisely for this scenario: you expose the new version to a small slice of real traffic (say 1–5%), compare its error rate, latency, and business metrics against the stable version, and only ramp up if it holds. A regression is therefore detected while it harms a fraction of users, and you can roll back by shifting that slice back. Blue-green (b) is excellent for instant cutover and instant rollback, but at the moment of cutover the new version takes the entire load, so a load-sensitive regression hits everyone — it minimizes downtime, not blast radius under live traffic. Rolling updates (c) do gradually replace instances, but the claim is wrong: as soon as a new instance is in the pool it serves real traffic, and rolling gives you no metric-gated traffic split. (d) is false — the whole point of these strategies is progressive exposure without a downtime window.

True, and this decoupling is the main strategic reason teams adopt feature flags. Deploying ships code to the servers; releasing exposes behaviour to users. A flag is a runtime switch wrapping the new code path, so you can merge to trunk and deploy continuously — keeping branches short-lived — while the feature stays off ('dark') in production. You then enable it on your own schedule: for internal users first, for a 5% canary cohort, or for everyone, and you can kill it instantly without a rollback deploy if it misbehaves. This is what makes trunk-based development and continuous deployment practical for large, in-progress features, and it underpins experimentation (A/B tests) and progressive delivery. The cost is real and worth naming: flags are conditional logic that accumulate, so undisciplined teams drown in stale flag debt — short-lived flags should be removed once a feature is fully rolled out.

Object spread merges left to right, and on a key collision the last spread wins. { ...defaults, ...process.env } therefore lets any variable present in the real environment override the committed default — so DATABASE_URL resolves to postgres://prod-db/app. This is the twelve-factor 'store config in the environment' principle in miniature: the same artifact ships everywhere, and per-environment values (prod DB, secrets, feature toggles) come from the environment at runtime, while a committed defaults file gives a sane local-dev fallback that never contains real secrets. Option b inverts the precedence (committed defaults are the fallback, not the override). Option c is wrong — duplicate keys across the two sources is the normal, intended case, not an error. Option d is false: process.env is a readable map here and nothing about reading it from a file changes merge order. The interview point is that environment beats baked-in defaults, which is what keeps one build deployable to every environment.

Blamelessness shifts the postmortem's question from 'who caused this?' to 'what about our systems and processes made this failure possible and its blast radius large?'. It rests on the assumption that people generally do their best with the context they had, so a human error is treated as a symptom of a system that allowed it (a missing guardrail, a confusing UI, an alert that fired too late), not as the cause to be punished. The reliability payoff is mechanistic, not soft: blame creates fear, fear suppresses honest reporting, and suppressed reports hide the real contributing causes — so the latent fault recurs. Remove blame and you get psychological safety, which produces candid, detailed accounts that reveal the true systemic gaps, which in turn become concrete, tracked action items (automation, safer defaults, better detection) that actually prevent recurrence. A strong answer stresses systems-over-individuals, psychological safety enabling honest reporting, and that accountability is redirected to fixing the system, not removed.

Incident response follows a deliberate order. First you must detect the problem — ideally an automated alert wired to an SLI breach, not a customer tweet. Then triage and declare: size up impact and severity, formally declare an incident so the right people engage, and name an incident commander to coordinate (separating coordination from hands-on debugging). Next comes the phase juniors most often get wrong: mitigate before you diagnose. The first duty is to stop user pain by the fastest safe lever — roll back the last deploy, fail over to a healthy region, or disable the feature flag — even if you don't yet understand the root cause. Only once the bleeding stops do you resolve: verify the SLI is back in bounds and stand the incident down. Finally, after the adrenaline fades, the postmortem extracts lasting value through a blameless review that identifies contributing causes and produces tracked action items so the same failure can't recur. The two classic mistakes are chasing root cause before mitigating, and skipping the postmortem so nothing is learned.

This is the foundational IaC distinction. A declarative approach states what you want — 'three web servers behind a load balancer with this config' — and the tool diffs that desired state against reality and figures out how to converge (create what's missing, change what differs, leave correct things alone). Because the spec is the end state, re-applying it is naturally idempotent and the tool can detect and correct drift. An imperative (or procedural) approach states how: an ordered list of commands ('create server, then install package, then edit file') that you must make safe to re-run yourself, since the script doesn't inherently know the current state. Most modern IaC and orchestration tools are declarative for exactly these reasons — idempotency, drift detection, and reproducibility — while shell-style provisioning scripts are imperative. Knowing which model a tool uses tells you how it will behave on a second run.

An error budget is derived directly from the SLO: it is 100% minus the objective, so a 99.9% target permits 0.1% unreliability over the measurement window (a). That budget is a currency. While it is unspent the team has room to move fast — deploy often, take calculated risk — and when it runs out, the agreed error-budget policy kicks in: stop shipping risky changes and redirect effort to reliability until the service earns budget back (b). Its real organizational value is making the eternal velocity-vs-stability tension an objective, shared number instead of a turf war between developers who want to ship and ops who want stability (c). Option d is the classic trap: a budget that is never spent means your SLO is too loose (or you're over-investing in reliability users don't perceive) — you're leaving velocity on the table, so consistently spending zero is a signal to recalibrate, not a victory. Option e is wrong: the budget is computed from the SLO/SLI math, not by tallying incident counts.

Idempotency means running the operation any number of times leaves the system in the same end state as running it once — the contract a convergent config tool depends on, since it re-applies on every run. This script breaks that contract twice. useradd app is a create that fails with a non-zero exit when the user already exists; under set -e that aborts the whole converge on the second run. And echo ... >> /etc/app.env appends, so each run adds another identical CONFIG_DIR=/etc/app line, growing the file and potentially changing how the app parses it. The fix is to make every step check-then-act so it's safe to repeat: guard the user creation (id app >/dev/null 2>&1 || useradd app) and add the config line only if it isn't already present (grep -qxF 'CONFIG_DIR=/etc/app' /etc/app.env || echo 'CONFIG_DIR=/etc/app' >> /etc/app.env). Option b is backwards — set -e correctly surfaces the latent failure; deleting it just hides a broken converge. Option c fixes only half the problem and wrongly calls the append idempotent. Option d is plainly false on both counts.

The interview point is the pull-based reconciliation loop versus push-based deployment. In GitOps, declarative manifests in git define desired state and an in-cluster controller continuously diffs live state against the repo and converges to it — making git the audit log, the rollback mechanism (revert the commit), and the drift-correction engine all at once. Traditional CI/CD pushes: the pipeline ends by pushing artifacts/changes into the environment using credentials the external system holds. Strong answers name (1) git as single source of truth, (2) a controller that reconciles continuously, (3) the security win of pull-from-inside vs handing prod creds to CI, and (4) automatic drift detection. A common follow-up is the limitation: GitOps assumes a declarative target and a healthy reconciler, so it maps cleanly onto Kubernetes but awkwardly onto imperative provisioning.

This design exercise tests the candidate's ability to architect a multi-layered deployment-gate system that balances safety with team autonomy. A strong answer covers stage-specific gates with concrete pass/fail criteria, canary rollback signals with numeric thresholds, cross-service dependency validation with an explicit block condition, and a policy-enforced baseline that prevents teams from disabling safety gates while still allowing extension. The rubric was corrected so that c1 (stage taxonomy) and c3 (cross-service dependency gating) no longer both credit consumer-driven contract tests: c1's integration-stage example now references integration tests against an ephemeral environment, while c3 exclusively owns contract tests, schema-compatibility checks, and dependency-graph mechanisms, and additionally requires the candidate to specify the exact block condition — making it discriminating against shallow answers that merely name a technique.