System Design Interview Questions: Reliability & Availability

When components are arranged in series (all must be up), the combined availability is the product of each component's availability. With three components each at 99% (0.99), the overall availability is 0.99 × 0.99 × 0.99 = 0.970299, or approximately 97.03%. This is why reducing the number of serial dependencies or improving individual component reliability is critical in system design.

While adding parallel replicas generally improves availability because the component only fails if ALL replicas fail simultaneously, it does not help indefinitely. In practice, there are diminishing returns — the improvement becomes negligible after a few replicas. More importantly, if the replicas share a common failure domain (e.g., same power supply, same network switch, same availability zone), a correlated failure can bring all of them down at once, making redundancy ineffective. The statement is therefore false in general.

Standard SRE incident response is governed by 'stop the bleeding first,' but initial communication is embedded in the declaration step rather than treated as a separate later phase. Step 1 (a): monitoring fires, an incident is declared, an incident commander is assigned, and an immediate 'we are investigating' notice goes out to stakeholders — communication begins here, not after mitigation. Step 2 (b): the team applies the fastest available remediation (rollback, failover, traffic rerouting) to minimise MTTR; this is the primary focus until service is restored. Step 3 (c): with the immediate impact contained and the service stabilising, the team pivots to diagnosing the root cause; earlier exploratory diagnosis may have run in parallel, but deep analysis becomes the primary focus only after mitigation succeeds. Step 4 (d): once the root cause is understood and fully resolved, the incident is formally closed, the status page is updated to 'resolved', and a final all-clear is sent. Step 5 (e): after full resolution, the team conducts a blameless post-mortem, documents contributing factors, and prioritises long-term corrective actions to prevent recurrence. By folding the initial stakeholder notification into the declaration step, the ordering avoids ambiguity about where communication sits relative to mitigation and diagnosis.

The calculation is straightforward: 30 days × 24 h × 60 min = 43,200 total minutes. 99.9% uptime leaves 0.1% for downtime: 43,200 × 0.001 = 43.2 minutes. The correct SRE term for this allowance is the 'error budget.' Understanding error budgets is critical for mid-level system design because they translate abstract SLO percentages into concrete operational guidance—when the budget is nearly exhausted, feature releases are slowed to protect reliability.

99.9% ('three nines') leaves 0.1% of a 8,760-hour year unavailable, which is about 8.76 hours per year. Each additional nine cuts allowed downtime by 10x — four nines (99.99%) is ~52 minutes/year, five nines (99.999%) is ~5 minutes/year. Knowing these conversions lets you sanity-check whether a proposed SLA is realistic given the architecture's redundancy and failover design.

Active-active multi-region failover (a) eliminates single-region SPOFs and keeps the service up during region-level outages. Circuit breakers (c) prevent a failing dependency from cascading and taking down the entire service. Idempotent async messaging (d) decouples availability of the caller from the availability of downstream services, making partial outages invisible to end users. A single Redis primary (b) is itself a SPOF—losing it loses sessions globally—so it hurts availability. A monolithic rolling restart (e) can improve deploy uptime, but a monolith is not inherently more available than microservices and does not directly address the 99.99% target; it is neutral-to-negative for this goal.

The canonical sequence is: (1) detect failure via health-check, (2) immediately stop sending traffic to the failed node so user impact is minimised, (3) notify humans and redirect traffic to healthy capacity, (4) provision replacement and let it catch up, (5) only re-introduce the node after it is verified healthy and in sync. Skipping detection before routing removal would send requests to a broken node; re-introducing a lagging replica before confirming lag ≈ 0 can serve stale or inconsistent data.

A latency-only SLO is a classic availability blind spot: when the service is completely down, requests fail fast (often with a TCP reset or immediate 5xx) rather than timing out, so they record near-zero latency or are excluded from percentile calculations altogether. The P99 therefore looks healthy during an outage. The fix is to pair every latency SLO with an error-rate/availability SLO so that both slowness and unavailability are captured in the error budget.

This question tests deep knowledge of availability math and SLA composition. When services are chained sequentially, their availabilities multiply: 0.9999 × 0.9995 × 0.9990 = 0.9984005. Expressed as nines, 0.9984 ≈ 99.84%, which is between two-nines (99%) and three-nines (99.9%). The composite is only ~99.84% available, meaning roughly 1.4 hours of downtime per month — far less than any single component's SLA. This is a critical trap: engineers often assume the 'best' component's SLA dominates, but in series, the worst combination wins and you always lose nines.

In a 5-node Raft cluster, a quorum of 3 nodes is required to commit any log entry. Option A is wrong: N1 cannot commit new entries because it can only hear acknowledgments from N2 (2 nodes < quorum of 3); leadership alone does not grant commit authority. Option B is correct: {N3, N4, N5} form a majority of 3 and will hold an election after their election timers fire without receiving heartbeats from N1, electing a new leader capable of committing writes. Option C is correct: N1 cannot commit new entries because quorum is unreachable with only N2. The Raft paper (Figure 2) does not mandate a proactive step-down on replication failure, but many production implementations (e.g., etcd's check-quorum) do cause N1 to step down after an election timeout of missed quorum acknowledgments. The statement correctly reflects both behaviors — N1 cannot commit regardless, and the precise step-down timing is implementation-dependent. Option D is correct: clients requiring linearizable reads routed to N1 or N2 cannot be safely served; a correct implementation must return an error or unavailable response because N1 cannot confirm it holds current leadership without quorum, and serving reads without that confirmation would violate linearizability. Option E is wrong: {N1, N2} has only 2 nodes and cannot form a quorum of 3, so no leader election in that partition can succeed.