Suppose you are testing a bill-splitting feature in a payments app. Amounts are stored in paise to avoid floating-point trouble. You write three example tests: ₹300 between 3 friends gives ₹100 each, ₹1,000 between 4 gives ₹250 each, ₹0 between 2 gives zero each. All pass. The feature ships, and within a day a user complains that splitting ₹100 three ways made one paisa disappear. Your examples were all neatly divisible. You chose them, and you chose them the way humans do: round numbers that are easy to check in your head.
Property-based testing removes that human bias from input selection. Instead of writing specific inputs and outputs, you describe a rule that must hold for all valid inputs, and a tool generates many inputs to try to break it.
From examples to properties
For the bill splitter, what is always true, whatever the amount and number of people?
- The shares add up exactly to the total.
- There is one share per person.
- No share is negative.
- The largest and smallest shares differ by at most one paisa.
None of these says what any particular share should be. They describe the shape of a correct answer. A property-based test states one or more of them and asks a generator for inputs: totals from 0 to, say, ten crore paise, and group sizes from 1 to 50. The tool then runs the check a hundred or more times with different values, deliberately including awkward ones such as 0, 1 and the maximum.
If the implementation simply divides and rounds down, the first property fails quickly, because most totals do not divide evenly. The tool reports a failing input.
Shrinking: the feature that makes it usable
A raw failing case might be a total of 73,481,229 paise split among 37 people. That is correct but unhelpful to debug. Good property-based tools then shrink the input: they try smaller and simpler values that still fail, and keep going until they cannot simplify further. For the rounding-down splitter, the shrunk case would be something like a total of 1 paisa between 2 people, which gives shares of 0 and 0 that add up to 0. That minimal example makes the bug obvious in seconds.
Shrinking is one of the main differences between property-based testing and simply throwing random data at code. Without it, failures are hard to understand; with it, the tool hands you the smallest reproduction it can find.
Where good properties come from
The hardest part is not the tool but thinking of properties. A few patterns cover a large share of real use.
Round trips. Encoding then decoding should return the original: serialise an object to JSON and read it back, compress and decompress, encrypt and decrypt, convert a date to a string and parse it. These properties catch surprising bugs with special characters, Unicode names, empty lists and time zones.
Invariants. Something about the output never changes: sorting keeps the same elements and the same count, and each element is no greater than the next; a transfer between two accounts leaves the combined balance unchanged.
Idempotence. Doing it twice equals doing it once: normalising a phone number, applying a discount code that should not stack, trimming whitespace.
A trusted oracle. Compare the new optimised implementation with a slow but obviously correct one, or a rewritten service with the legacy one, across generated inputs. This is extremely useful during migrations.
Hard to compute, easy to verify. Finding the cheapest delivery route might be complex, but checking that the returned route visits every stop once is simple.
Tools across languages
The approach began with QuickCheck for Haskell, described by Koen Claessen and John Hughes in 2000. Most mainstream languages now have mature libraries: Hypothesis for Python, jqwik for Java on the JUnit 5 platform, fast-check for JavaScript and TypeScript, and FsCheck for .NET. They share the same core ideas: generators for basic types that can be combined into generators for your domain objects, a configurable number of runs, shrinking, and some way to replay a failing case. Hypothesis, for instance, stores failing examples in a local database and tries them first on the next run, so a bug found once is not lost to randomness.
Limits and honest trade-offs
Property-based testing is not a replacement for example tests. Examples document specific business expectations that stakeholders can read (“a ₹300 bill for three is ₹100 each”). Properties catch the gaps between them. Most teams that use both keep a handful of named examples and add properties for the logic most exposed to unusual input.
Some practical cautions:
- Weak properties pass weak code. “The result is a list” is a property, but it will not catch much. Combine several properties so that only a correct implementation satisfies them all.
- Re-implementing the logic is a trap. If your property computes the expected answer with the same formula the code uses, both share the same bugs.
- Generators need care. Producing valid domain objects, such as a correctly formatted GSTIN or an order whose line items sum to its total, takes effort. Overly filtered generators that discard most random values slow the run and reduce coverage.
- Slow systems limit run counts. A property over a pure function can run thousands of times per second. A property that hits a database or UI cannot, so it is usually applied at the unit or service layer.
- Randomness and CI. Different seeds on each run can make a build fail intermittently when a rare case is found. That is the tool doing its job, but the team needs the failing seed or example in the log so it can be reproduced and fixed, not just rerun.
Property-based testing is closely related to fuzz testing, which also generates inputs automatically. The difference in emphasis is that fuzzing usually looks for crashes, hangs and security issues using a very large number of mostly invalid inputs, while property-based testing checks specific correctness rules on mostly valid ones. Both start from the same humility: the inputs you would pick by hand are not the ones that will break your code.
Written by the RP Strativa Team. The bill-splitting feature and figures are illustrative.
Leave a Reply