Distributed Transactions (Saga & Outbox)

A distributed transaction is one logical unit of work — “place an order” — that spans multiple services, each with its own database. In simple language — it’s trying to keep several separate databases in agreement when there’s no single transaction wrapping them all. And the honest truth interviewers want us to say is: we can’t, not cleanly, so we use patterns that give us eventual consistency instead.

Why ACID doesn’t stretch across services

In a single database, a transaction is ACID — all-or-nothing, isolated, durable. We BEGIN, do three writes, COMMIT, and either all three land or none do. Beautiful.

The moment “place an order” touches the Order service’s DB, the Payment service’s DB, and the Inventory service’s DB, that guarantee is gone. There’s no shared BEGIN/COMMIT across three separate databases owned by three separate services. Each can commit or fail independently. So how do we keep them consistent?

Two-phase commit (and why we avoid it)

The textbook answer is 2PC (two-phase commit): a coordinator asks every participant “can you commit?” (prepare phase), and if all say yes, tells everyone “commit” (commit phase).

It technically works, but we avoid it in microservices because:

• It’s blocking. Everyone holds locks while waiting for the coordinator’s decision. Slow and low-throughput.
• The coordinator is a single point of failure. If it dies mid-commit, participants are stuck holding locks, unsure what to do.
• It couples services tightly and most modern datastores/message brokers don’t support it well.

So in practice, we reach for the Saga pattern.

The Saga pattern

A saga breaks the big distributed transaction into a sequence of local transactions, one per service. Each service does its own little ACID transaction and then triggers the next step. No global lock, no coordinator holding everyone hostage.

The catch: there’s no automatic rollback across the whole thing. So for every step, we define a compensating action — an explicit “undo” that reverses it. If step 4 fails, we run the compensations for steps 3, 2, 1 in reverse. We don’t roll back; we apologize and reverse.

Think of it like a chain of “do X / to undo X do Y” pairs.

There are two ways to coordinate a saga:

• Choreography — no central boss. Each service listens for events and emits its own. Order emits OrderCreated → Payment reacts, emits PaymentDone → Inventory reacts. Decentralized and loosely coupled, but the flow is spread across services and hard to follow (“who does what?” gets fuzzy).
• Orchestration — one orchestrator service holds the workflow and tells each service what to do next, step by step. Easier to reason about and to add compensation logic, but the orchestrator is a component we have to build and maintain.

Rule of thumb: choreography for simple 2–3 step flows, orchestration once the workflow gets complex enough that you want it written down in one place.

The dual-write problem

Here’s a subtle bug that sagas run straight into. A service often needs to do two things atomically: update its database AND publish an event (“payment done”) for the next step. That’s a dual write — two separate systems, no shared transaction.

What if we commit to the DB but crash before publishing the event? The DB says “paid”, but no one downstream ever hears about it. The saga stalls. Consistency broken. We cannot make “write to DB” and “write to the message broker” atomic — they’re two different systems.

The transactional Outbox pattern

The Outbox pattern fixes the dual-write problem with one clever move: instead of publishing the event directly, we write the event into an outbox table in the same database, in the same local transaction as the business data.

Now it’s a single local ACID transaction — the order update and the event row commit together or not at all. No dual write.

-- ONE transaction: business write + event write commit atomically
BEGIN;
  UPDATE orders SET status = 'paid' WHERE id = 42;
  INSERT INTO outbox (topic, payload)
    VALUES ('order.paid', '{"orderId":42}');
COMMIT;

Then a separate relay process (a poller, or Change Data Capture tailing the DB log) reads unpublished rows from the outbox table and publishes them to the message broker, marking each as sent. If the relay crashes, it just re-reads on restart — so events are delivered at least once. That means downstream consumers must be idempotent (safe to process the same event twice), which pairs perfectly with the idempotency habits from retries.

Interview soundbite

ACID transactions can’t span services, and 2PC is avoided because it’s blocking with a single-point-of-failure coordinator. Instead we use a saga: a sequence of local transactions each with a compensating action for undo, coordinated by choreography (event-driven, decentralized) or orchestration (a central workflow service). To publish events reliably we use the transactional Outbox — write the event to an outbox table in the same local transaction as the data, then a relay ships it to the broker at-least-once — which sidesteps the dual-write problem.