CAP Theorem & Consistency Models

Once our data lives on multiple machines (replicas, shards), the network between them can break — and then we have to make a hard choice. The CAP theorem is the rule that describes that choice. In simple language — when the network splits, a distributed system can stay consistent or stay available, but not both. It has to give one up.

The three letters

• C — Consistency — every read sees the most recent write. All nodes agree on the current value. (This is linearizability, a stronger promise than the “C” in database ACID.)
• A — Availability — every request gets a non-error response, even if it’s not the freshest data. The system stays up.
• P — Partition tolerance — the system keeps working even when the network between nodes drops or delays messages.

”Pick 2” is a bit of a lie

The famous line is “pick 2 of 3”, but that framing misleads people. In any real distributed system, network partitions WILL happen — cables get cut, nodes get slow, packets drop. So P is not optional. We can’t choose to opt out of the network being unreliable.

That means the real choice is only between C and A, and only during a partition. When the network is healthy, we get all three. The moment it splits, we must pick:

• Refuse requests to avoid serving stale data → we kept C, gave up A → CP.
• Keep answering with possibly-stale data → we kept A, gave up C → AP.

CP vs AP with a concrete example

Picture two data-center nodes, and the link between them dies. A user writes balance = $100 to node 1, then reads from node 2 (which never got the update).

• A CP system makes node 2 refuse the read (or error) rather than return a stale $50. Correct, but that user is temporarily locked out. Good for banking, inventory, locks, config — anywhere wrong data is worse than no data.
• An AP system lets node 2 answer with the old $50 and reconciles once the link heals. Available, but briefly wrong. Good for social feeds, product catalogs, shopping carts, DNS — where being up matters more than being perfectly fresh.

Strong vs eventual consistency

CAP’s “C” is all-or-nothing, but real systems live on a spectrum. Two labels worth knowing:

• Strong consistency — after a write completes, every subsequent read (from anywhere) sees it. Simple to reason about, but needs coordination, so it’s slower and less available. This is the CP flavor.
• Eventual consistency — replicas may disagree for a moment, but with no new writes they all converge to the same value eventually. Fast and available, but a read might be stale. This is the AP flavor.

There are useful middle grounds too — read-your-writes (we always see our own latest changes) and monotonic reads (we never see time go backwards). These make eventual consistency feel far less broken to users.

PACELC — the footnote CAP forgets

CAP only talks about behavior during a partition. But partitions are rare — what about the 99.9% of the time the network is fine? PACELC extends it:

If Partition → choose A or C (the CAP part). Else (normal operation) → choose Latency or Consistency.

The insight: even with no partition, keeping replicas perfectly consistent costs latency (waiting for confirmations). So systems like Cassandra are PA/EL — available under partition, low-latency otherwise — while a strongly-consistent store is PC/EC. It captures the everyday trade-off CAP ignores.

Interview soundbite

CAP says that during a network partition a distributed system must choose consistency or availability — and since partitions are unavoidable, P is a given, so the real choice is CP vs AP. CP systems refuse rather than serve stale data (banking, locks, etcd); AP systems stay up and reconcile later (Cassandra, DynamoDB, DNS). Strong consistency means every read sees the latest write; eventual means replicas converge over time. PACELC adds the missing half: even without a partition, we still trade latency against consistency.