One tool, both shapes

Queue, topic, broker and stream describe shapes of delivery. One product can act in more than one shape, and what decides the shape is how consumers are attached.

Kafka: the group id decides

Kafka’s consumer group is one abstraction that covers both shapes. In one group, each partition goes to one consumer, so the group shares the records like a work queue. Give a second application a different GroupId and it receives every record too, with its own offsets, which is publish-subscribe. See partitions, offsets and consumer groupsPartitions, offsets and consumer groupsHow a stream splits a topic into lanes, marks each reader's place, and shares the work among a group.Opens in a new tab.

var config = new ConsumerConfig{    BootstrapServers = "localhost:9092",    GroupId = "payments",    AutoOffsetReset = AutoOffsetReset.Earliest}; using var consumer = new ConsumerBuilder<string, string>(config).Build();consumer.Subscribe("order-placed"); while (!ct.IsCancellationRequested){    var result = consumer.Consume(ct);    Handle(result.Message.Key, result.Message.Value);}

Every instance of Payments uses GroupId = "payments" and the instances share the work. Notifications uses "notifications" and reads the same topic independently.

RabbitMQ: queues, exchanges and streams

A RabbitMQ producer publishes to an exchange, never straight to a queue. With several consumers on one queue, the queue is a work queue. A fanout exchange routes a message to every queue bound to it, so each subscriber has its own queue and gets a copy, which is publish-subscribe. See routing and topologyRouting and topologyHow a messaging system decides where each message goes: exchange types, subscription filters and partition keys.Opens in a new tab.

Classic and quorum queues remove a message once the consumer acknowledges it, so they cannot replay it. RabbitMQ also has streams, introduced in 3.9: an append-only log with non-destructive reads, where a consumer attaches at an offset, a timestamp, or the first or last record and can read the same data again. A stream is declared with the queue argument x-queue-type set to stream, and the stream documentation recommends its dedicated binary protocol for access to every stream feature. Quorum queues are the documented default choice for a replicated queue, and classic queue mirroring was removed in RabbitMQ 4.0.

The catch

The same shape does not mean the same behavior. A Kafka group acts like a queue, but there is no per-message acknowledgement or redelivery: progress is one offset per partition, and the number of consumers that can work is capped by the partition count. A RabbitMQ queue and a RabbitMQ stream look alike to a client but differ in what happens after a read. Choose by the behavior needed, see broker vs event streamBroker vs event streamA broker hands messages out and removes them once acknowledged, a stream keeps a log that consumers read at their own pace.Opens in a new tab.