Broker vs event stream

There are two ways to keep messages between a producer and a consumer. A message broker holds a message until a consumer acknowledges it, then removes it. An event stream appends events to a log and keeps them for a retention period, and each consumer remembers its own position in the log. A post office delivers the letter and is done with it. A ledger keeps every entry, and each reader keeps a finger on the line they reached.

Aspect	Message broker	Event stream
After consumption	Removed once acknowledged	Kept until retention expires, by time or size
Who tracks progress	The broker, with locks and acknowledgements	Each consumer group, with an offset
Replay	Not possible after acknowledgement	Rewind the offset and read again
Routing	Rich: exchanges, bindings, subscription filters	Topic and partition key, filtering done by the consumer
Per-message features	Built in: dead-lettering, message expiry	Mostly left to the application
Typical tools	Azure Service Bus, RabbitMQ queues	Apache Kafka, Azure Event Hubs, RabbitMQ Streams

Broker

A broker is the smart part of the system. It tracks each message, locks it while a consumer works on it, redelivers it if the consumer fails, and moves it to a dead-letter queue if it keeps failing. Azure Service Bus works this way: a consumer receives a message under a lock, and the message is removed when the consumer marks it complete. RabbitMQ queues likewise remove a message only after the consumer acknowledges it. A broker suits work that must be done once, such as a command.

Stream

A stream is closer to a log file than to a mailbox. Kafka states it directly: “Events are not deleted after consumption.” A consumer reads from any position, and a new consumer can start from the beginning. That makes replay possible: recovering after a bug, filling a new service with history, or reprocessing with new logic. Event Hubs describes the same model, in which the subscriber manages its cursor and can move forward and back in the stream. The consumer carries more responsibility, because the stream does not track individual messages for it, see partitions, offsets and consumer groupsPartitions, offsets and consumer groupsHow a stream splits a topic into lanes, marks each reader's place, and shares the work among a group.Opens in a new tab.

Choosing

Use a broker for work that one owner must do and finish. Use a stream for facts that several consumers read, where a late consumer may need the history. Most systems use both, a broker for commands and a stream for the event history.

The catch

The line is not clean. RabbitMQ has both queues and streams, and Kafka can behave like a queue, see one tool, both shapesOne tool, both shapesKafka and RabbitMQ can each act as a work queue or as publish-subscribe, depending on how consumers attach.Opens in a new tab. Replay also has a cost: a replayed event runs its side effects again, so consumers must tolerate seeing an event twice, see idempotent consumersIdempotent consumersA consumer that gives the same result whether a message arrives once or five times.Opens in a new tab. Retention is a setting, not forever.