2026-05-15
You've just updated a user's email in Postgres and now need to publish a UserEmailChanged event to Kafka. The naive code looks fine:
db.commit()kafka.publish(event)What happens if the process crashes between those two lines? The database has the new email, but no event was ever sent. Downstream services — search indexes, notification systems, audit logs — silently drift out of sync. Worse, if you reverse the order, Kafka has an event for a change that was rolled back.
This is the dual-write problem: you cannot atomically write to two different systems without a distributed transaction, and distributed transactions are a performance and reliability nightmare.
The Outbox Pattern solves this by collapsing the two writes into one. Instead of publishing directly to the message broker, you insert the event into an outbox table in the same database transaction as your business write:
BEGINUPDATE users SET email = ... WHERE id = 42INSERT INTO outbox (aggregate_id, event_type, payload, created_at) VALUES (...)COMMITNow either both rows land or neither does — atomicity guaranteed by your existing database. A separate relay process (a background worker, or Debezium reading the WAL) polls or tails the outbox table, publishes each row to Kafka, and marks it as sent.
Real-world example: Shopify uses this pattern extensively. When you place an order, the order row and its associated events (OrderCreated, InventoryReserved, PaymentRequested) are written in one transaction. The relay handles fan-out to dozens of downstream services. If Kafka is down for 10 minutes, events queue up safely in Postgres and drain when it recovers.
Two things to get right:
DELETEing.The pattern's beauty is that it requires no new infrastructure — just a table and a worker. You trade a tiny bit of latency (events publish milliseconds after the commit, not during it) for bulletproof consistency between your database and your event stream.
