Exactly Once vs At Least Once: What Kafka's Guarantee Actually Covers

Every message broker advertises exactly-once semantics, but that guarantee doesn't mean what most developers assume. The Two Generals Problem proves that exactly-once delivery is mathematically impossible over any unreliable network — a lost ACK forces the producer to choose between duplication and data loss.

The achievable goal is exactly-once processing: your consumer might receive a message multiple times, but its effect only applies once. The key patterns are idempotent writes (upserts keyed on message ID), dedup tables (reject duplicates inside the same DB transaction), and atomic offset commits (store consumer position in your application database). Kafka's transactional API delivers true exactly-once, but only when both source and sink are Kafka topics — external databases and APIs fall outside the fence.

Design every consumer for at-least-once delivery. Assume duplicates will arrive. Make your handlers safe for it with a dedup check, and the distinction between delivery semantics stops mattering at the application layer.