Event versioning

2 min read
On trails:EDARELstep 9/9

An event published today may be read next month, by a consumer that is deployed on its own schedule. A change to an event’s shape has to be safe for readers that have not changed yet, so events change by addition wherever possible. The envelopeAnatomy of an eventAn envelope that says which event it is and when it happened, and a payload that carries the facts.Opens in a new tab carries a message type and a version for this purpose.

Changes that are safe

ChangeSafe for existing consumersNote
Add an optional fieldYesOld consumers ignore it. New consumers treat a missing value as null or a default
Add a new event typeYesConsumers that do not subscribe to it never see it
Remove a fieldOnly if nobody reads itA consumer that expects the field breaks or reads a default
Rename a fieldNoTo a reader it is a removal plus an addition
Change a type or unitNoFor example Total from a number to an object with a currency
Give a field a new meaningNoNothing fails, and the reader misreads it
Table 1: changing OrderPlaced

Tolerant readers

A tolerant reader takes what it needs from a message and ignores everything else. Martin Fowler describes it as being liberal in what you accept. In .NET, System.Text.Json already skips JSON properties that the target type does not have. Setting JsonUnmappedMemberHandling.Disallow (available from .NET 8) makes it throw instead, so leave it off for message contracts. A consumer that meets a version it does not recognise should dead-letter the message with a clear reason (see dead-letter queuesDead-letter queuesWhere a message goes when it cannot be processed, so the rest of the traffic keeps moving.Opens in a new tab) rather than guess at its meaning.

When the change must break

If the meaning or type has to change, publish a new type beside the old one and keep both for a while. Producers send OrderPlaced and OrderPlacedV2, consumers move to the new one one at a time, and the old one is retired when nobody consumes it any more. This is the same idea as running two versions of an HTTP API (see API versioningAPI versioningFour ways to let old clients keep working while the API changes.Opens in a new tab).

OrderPlaced.csC#
// Additive change: same type, one optional field. Messages sent before it arrive with CouponCode = null.public sealed record OrderPlaced(Guid OrderId, Guid CustomerId, decimal Total, string? CouponCode = null); // Breaking change: Total becomes a Money value. It is published as a new type beside OrderPlaced.public sealed record OrderPlacedV2(Guid OrderId, Guid CustomerId, Money Total, string? CouponCode); public sealed record Money(decimal Amount, string Currency);

Publishing both from the same transaction through an outboxOutbox and inboxWrite the event in the same transaction as the data, publish it from there, and deduplicate on the way in.Opens in a new tab keeps the two copies from drifting apart.

A schema registry

A schema registry stores each version of an event’s schema and checks a new version against the earlier ones before it is accepted. Confluent Schema Registry, for example, has compatibility modes: BACKWARD (the default, where consumers on the new schema can read old data), FORWARD, FULL and NONE, plus transitive variants that check against every earlier version. The mode decides the upgrade order. Under BACKWARD, upgrade all consumers before producers start writing the new schema. Under FORWARD, upgrade producers first. Under FULL, either side can go first. A check at publish time catches a breaking change in the build instead of in a consumer.

Sources