You have seen the triangle diagram: Consistency, Availability, Partition tolerance, “pick any two”. Then comes a table sorting databases into CP and AP, as if choosing a database were like choosing two toppings. It is a tidy picture and it is mostly wrong, in ways that lead to bad design decisions.
Myth one: you can choose to skip partition tolerance
A network partition means some nodes of your system cannot talk to others: a switch fails, a cable is cut, a cloud availability zone becomes unreachable, or a packet-loss storm makes messages effectively disappear. If your data lives on more than one machine, partitions will happen, and you do not get a vote on it. So “CA” for a distributed system is not a real option; it describes a system that is only safe if the network never fails, which is to say a single node.
The honest statement of the theorem, formalised by Gilbert and Lynch in 2002 following Eric Brewer’s earlier conjecture, is narrower: when a partition occurs, a system must choose between consistency and availability. When there is no partition, you can have both. The triangle makes it look like a permanent trade; it is a trade you make only on bad days.
Myth two: the words mean what they mean elsewhere
The “C” in CAP is not the “C” in ACID. In CAP, consistency means linearizability: every read sees the latest completed write, as if there were a single copy of the data. ACID consistency is about a transaction leaving the database in a state that satisfies its rules. Different things, same letter.
The “A” is also strict. Availability here means every non-failing node must return a non-error response to every request. A system that answers 98 per cent of requests with a timeout during an outage is not “available” in the CAP sense, and a system that is up 99.99 per cent of the time in your SLA is not making a CAP claim at all. The theorem talks about a specific, extreme guarantee.
A small scenario
Suppose a ticketing service keeps one seat inventory replicated in Mumbai and Hyderabad. One seat, 14C, remains. The link between the two cities breaks. A customer in Mumbai and another in Hyderabad both try to book 14C at nearly the same moment.
You have two honest choices. Choose consistency: the replica that cannot confirm with the other refuses the booking, or the minority side stops accepting writes. Nobody gets a double-booked seat, but some customers see an error. Choose availability: both sides accept the booking. Both customers get a confirmation, and when the link returns you have to reconcile, probably by cancelling one and apologising.
Neither is universally correct. For a seat or a bank balance, refusing is usually better. For a “like” counter or a product-view count, accepting both and merging later is fine, and no customer is harmed. The decision belongs to the business meaning of the data, not to a database brand.
Myth three: a database is permanently CP or AP
Classifying whole products is shaky. Many databases let you tune behaviour per query or per operation. In Cassandra, for instance, you pick a consistency level for each read and write; asking for a quorum leans towards consistency, asking for one replica leans towards availability. Documented behaviour also changes between versions and configurations, so a label from a blog post of several years ago may not describe what you are running. Read the vendor’s documentation for the specific mode you use, and test failure behaviour, rather than trusting a chart.
What matters more day to day: latency
Partitions are rare compared with ordinary operation, so the CAP choice describes an uncommon moment. Daniel Abadi pointed this out with the PACELC formulation: if there is a Partition, choose between Availability and Consistency; Else, when the system runs normally, choose between Latency and Consistency. Keeping replicas strictly in step requires waiting for acknowledgements across machines, maybe across cities. A write that must reach Mumbai and Hyderabad before returning cannot be faster than the round trip between them. So many systems relax consistency not only to survive partitions but simply to be quicker all the time. This everyday trade-off is the one your users feel.
Using the idea properly
When you evaluate a design, skip the triangle and ask questions instead.
- If two parts of the system cannot talk to each other, what does each part do with a write? Refuse it, or accept it?
- If writes are accepted on both sides, how are conflicts detected and resolved: last write wins by timestamp (which can silently drop data), merge rules, or a human?
- Which data in this product can tolerate being stale for a few seconds, and which cannot?
- What does the user see while the system is degraded?
Most real products end up with a mix: strongly consistent storage for orders and payments, relaxed caches and replicas for browsing and counts. That mix is a design choice you make field by field, and CAP is only the reason the choice exists.
Figures and names in the examples are illustrative.
Leave a Reply