Kafka Interview Questions: Practice Real Technical Interview Quiz

A Kafka topic is a named, ordered, append-only stream of records that producers publish to and consumers read from. It is physically split into one or more partitions — each its own independently ordered log — which is what lets Kafka parallelize both writes and reads across a topic instead of serializing everything through one log.

400

False. Kafka only guarantees ordering within a single partition — messages within one partition are read back in exactly the order written, but there is no ordering guarantee across a topic's different partitions. Anything needing per-entity ordering must key its messages so all of that entity's records land on the same partition.

One independently-ordered, append-only log that a topic is split into. Every record within a single partition has a strictly increasing offset and is read back in the exact order it was written; there is no ordering guarantee across different partitions of the same topic. Partitions are the unit of parallelism (different partitions can be produced to and consumed from independently) and of replication (each partition is copied across a set of brokers).

Kafka supports three real delivery semantics: at-most-once, at-least-once and exactly-once. Each is a different tradeoff of commit timing and producer configuration. At-most-once commits offsets before processing, so a crash between the two loses the message but never reprocesses it. At-least-once commits after processing, so a crash between the two reprocesses the message but never loses it — this is the default behaviour and the reason consumer logic needs to be idempotent. Exactly-once layers an idempotent, transactional producer and read_committed consumers on top to get neither loss nor duplicates. "Best-effort-once" and "eventual-once" are not Kafka terminology.

A consumer group is a set of consumer instances that share a group.id and cooperatively consume a topic, splitting its partitions between them so each partition is owned by exactly one consumer in the group at a time — this is how Kafka gets parallel consumption of one topic across multiple processes/machines while still processing each partition's messages in order. The group coordinator (a broker) tracks membership and triggers a rebalance whenever a consumer joins, leaves, or is considered dead (missed heartbeats) — at which point partitions are redistributed among the remaining/new members. Different consumer groups are fully independent: each group gets its own copy of every message on the topic, tracked by its own offsets, which is how one topic can feed multiple unrelated downstream applications.

A Kafka message's path is: the producer sends it (optionally keyed), the partitioner assigns it to a partition, the partition leader appends it to its log, followers replicate it into the in-sync replica set, the broker acknowledges per the acks setting, and only then does a subscribed consumer poll, read, and commit its offset for that record. With acks=all, the acknowledgment is deliberately held until replication completes, which is what makes it mean the write is durable rather than merely written somewhere.

A Kafka producer accumulates records into a batch per partition. It sends that batch immediately once it reaches batch.size bytes; if the batch is still not full, the producer waits up to linger.ms milliseconds for more records before sending it anyway.

Line 5 — key: null gives the partitioner nothing to route on, so it spreads messages across all partitions instead of pinning a user to one, and a given user's events land on different partitions with no ordering guarantee between them

Assume events average 1 KB, ordering matters per order and not across orders, and the business will pay for 35 days of retention to give the 30-day replay some headroom.

One topic, order-events, carrying all four event types, keyed by order id. One topic keeps a single order's lifecycle in one partition, which is the only ordering Kafka can actually give me; splitting by event type would scatter one order's created and paid across topics and make ordering impossible to reason about. Order id is deliberately chosen over tenant id: tenant id looks attractive because it would order a tenant's whole history, but with 2% of orders concentrated in a handful of enterprises it would drive those tenants onto one or two partitions and leave the rest idle — a hot partition that no amount of extra consumers can relieve, because one key always maps to one partition.

At 50k/sec and roughly 1 KB per event that is about 50 MB/sec. A partition comfortably handles some tens of MB/sec, but sizing to the theoretical minimum leaves no headroom, and partition count can be raised and never lowered — so I would start around 24 to 32 partitions, measured against a real load test rather than guessed. Over-provisioning is cheap; discovering later that I need more is not, because raising the count re-hashes keys and an order that has already emitted created on the old partition can emit paid on a new one, breaking exactly the guarantee the key existed to provide. If I ever do need to widen, the safe path is a new topic at the new count with consumers cut over once the old one has drained.

Three consumer groups, three different guarantees. Search runs at-least-once and does nothing special: reindexing the same document twice is a no-op, so it commits after processing and moves on. Billing is the hard one, and the key point is that Kafka's exactly-once is Kafka-to-Kafka — it cannot make an external payment provider charge once. So billing derives an idempotency key from the order id and the payment attempt, sends it with the charge, and lets the provider collapse duplicates; Kafka then only needs at-least-once, which it already gives me. If billing also writes a result back into Kafka, I would wrap the read, the charge record and the offset commit in a transaction with read_committed downstream, so no partial state is visible. The warehouse loader batches and is idempotent by upsert on order id.

Retention is 35 days, so the 30-day replay is always inside the window; at this volume that is a real storage bill, which is where tiered storage earns its place by pushing older segments to object storage. Replay itself is just an offset reset on the warehouse group, or a brand-new group id starting from earliest. Consumer groups have completely independent offsets, so this does not touch billing or search — and that independence is the answer to the replay question: billing never sees the replayed events because it is a different group with its own committed position.

Durability is replication factor 3, acks=all, min.insync.replicas=2, so a write survives one broker loss and is rejected rather than silently single-copied if the ISR shrinks further. On failures: a record that cannot be processed goes to order-events-dlq with the exception attached after a bounded number of retries, never dropped and never retried forever — an infinite retry on one poison record stalls the whole partition behind it. When billing is down its lag simply grows while search and the warehouse carry on unaffected, and on restart it resumes from its last committed offset and works through the backlog; the only thing I must watch is that the outage stays well inside retention. Monitoring is lag per partition, not just the average, because the enterprise tenants mean a single partition can be badly behind while the mean looks healthy — max lag against average lag is the cheapest hot-partition detector there is. Slow processing gets max.poll.records lowered rather than max.poll.interval.ms raised, so a backlog does not turn into a rebalance loop.

The main tradeoff is transactions against downstream idempotency. Transactions give a clean story inside Kafka but add coordinator overhead, hold the Last Stable Offset back for read_committed readers, and still do not cover the payment provider. Idempotency keys are less elegant and put the burden on the consumer, but they work across the system boundary that actually matters here. I would take the idempotency key.