Skip to content
Software Testing · First Principles

Property-Based Testing: Generators and Invariants

The previous page, Example-Based vs Property-Based Testing, argued the why: hand-picked cases only cover the region you already imagined, while a generated space samples the whole ocean. That page left you convinced but empty-handed. This page fills your hands. We are going to write a property test from scratch — a generator that manufactures inputs, and a property function the framework runs across hundreds of those inputs — and then hand you a small kit of patterns so you can find good properties in your own code instead of staring at a function wondering what rule to assert.

Everything here serves the book’s throughline. A property is the cheapest honest statement of “correct” that scales: you write one rule, and a machine checks it against inputs you would never type. The confidence you earn is justified precisely because it did not come from twenty cases you liked — it survived a thousand the tool threw at it, aimed at the boundaries where software breaks.

The two moving parts

Section titled “The two moving parts”

Every property-based test, in every framework, is built from exactly two pieces:

  1. A generator — a description of what inputs to feed the code. Not one input: a recipe for producing arbitrarily many. “Any vector of integers.” “Any valid UTF-8 string.” “Any pair of a non-empty list and an index into it.” The framework draws from this recipe to build hundreds of concrete, random cases.
  2. A property — a boolean function of one generated input that must return true for every case. It is the rule. Not sort([3,1,2]) == [1,2,3] (that is an example — one input, one expected output), but ∀ xs: isSorted(sort(xs)) (a relationship that holds for all xs).

The framework’s job is the glue between them: draw an input from the generator, run it through the property, and if the property ever returns false, stop and report the input that broke it. Do that a few hundred times.

   ┌─────────────┐   draws     ┌──────────────────────┐
   │  generator  │ ─────────►  │  property(input)     │
   │ "any Vec<i>"│  input #1   │  → true / false      │
   └─────────────┘  input #2   └──────────────────────┘
                    input #3            │
                    ...  (×256)         │  first false?
                                        ▼
                             report the failing input
                             (then SHRINK it — next page)

That is the entire model. A framework like QuickCheck, Hypothesis, proptest, or fast-check is mostly a large, well-tuned library of generators plus that loop. What is left is genuinely hard and genuinely yours: choosing the property. The rest of this page is about that choice.

The mechanics, in one framework

Section titled “The mechanics, in one framework”

Here is the shape in Rust with proptest, testing a sort function. Read the proptest! macro as “for the following generated inputs, the body must hold every time”:

use proptest::prelude::*;


proptest! {
    // `any::<Vec<i64>>()` is the GENERATOR: any vector of i64s,
    // including empty, single-element, huge, all-equal, negatives.
    #[test]
    fn sort_output_is_ordered(mut xs in any::<Vec<i64>>()) {
        let original = xs.clone();
        xs.sort();


        // PROPERTY 1 (invariant): the result is non-decreasing.
        prop_assert!(xs.windows(2).all(|w| w[0] <= w[1]));


        // PROPERTY 2 (invariant): the result is a permutation of the
        // input — same multiset. Catches a "sort" that drops or
        // duplicates elements while still being ordered.
        let mut sorted_original = original;
        sorted_original.sort();
        prop_assert_eq!(xs, sorted_original);
    }
}

The same test in Python with Hypothesis reads almost identically — the decorator is the generator:

from hypothesis import given, strategies as st


@given(st.lists(st.integers()))   # the generator ("strategy")
def test_sort_is_ordered(xs):
    result = sorted(xs)
    assert all(a <= b for a, b in zip(result, result[1:]))   # invariant
    assert sorted(xs) == sorted(result)                       # permutation

Notice what you did not write: any specific list. You described the shape of the input and the rule of the output. The framework supplies the hundreds of lists — [], [0], [5, 5, 5], [i64::MIN, i64::MAX], a thousand-element shuffle — and checks the rule against each.

The test oracle problem, and how properties dodge it

Section titled “The test oracle problem, and how properties dodge it”

Here is the deep reason properties are powerful, and it has a name: the test oracle problem.

An oracle is any mechanism that tells you whether a program’s output is correct. In an example test, you are the oracle: you computed [1,2,3] in your head and wrote it as the expected value. That works only because you already know the right answer for that one input. But a generator hands you [8823, -4, 8823, 0, -4, 991, ...] — a list you have never seen. What is the “expected” sorted output? You would have to sort it by hand to know, which is absurd: you would be reimplementing the function to test the function.

This is the oracle problem: for a generated input, you usually cannot state the exact expected output. If property-based testing required exact expected outputs, it would be useless.

The escape is the whole idea: assert a relationship the output must satisfy, not the output itself. You do not need to know what sort returns for that gnarly list. You only need to know two things that are true of the answer whatever it is: it is ordered, and it is a permutation of the input. Both are checkable without knowing the answer in advance. You traded “I know the exact output” (impossible at scale) for “I know a rule the output obeys” (cheap, and true for all inputs). That trade is why properties scale where examples cannot.

   EXAMPLE TEST                    PROPERTY TEST
   oracle = you, in advance        oracle = a checkable relationship
   "the answer is [1,2,3]"         "the answer is ordered AND a
   needs the exact output           permutation of the input"
   → can't do it for random input  → works for ANY input, unknown answer

Four patterns for finding properties

Section titled “Four patterns for finding properties”

The hardest part of property testing is not the syntax — it is answering “what rule could I possibly assert?” These four patterns are the workhorses. When you are stuck, walk down this list and ask which one fits.

1. Round-trip: encode then decode

Section titled “1. Round-trip: encode then decode”

If a function has an inverse, applying both should return you to where you started. This is the single most productive pattern in practice.

   x ──encode──► e ──decode──► x        property: decode(encode(x)) == x

Serializers, parsers, compressors, URL encoders, base64, JSON, your own binary format — all round-trip. You do not need to know what the encoded bytes look like; you only assert that decoding them gives back the original.

@given(st.text())
def test_json_roundtrips(s):
    assert json.loads(json.dumps(s)) == s

This one property, over generated text, will find things your examples never will: it feeds strings with embedded quotes, backslashes, newlines, null bytes, emoji, right-to-left marks. If your encoder mishandles any of them, the round-trip breaks and hands you the exact character that did it.

2. Oracle: compare to a reference

Section titled “2. Oracle: compare to a reference”

When you do have a second, trusted implementation of the same behavior, use it as the oracle. Run both on the generated input; the outputs must match.

   input ──►  your fast/new version   ──► a
          ──►  a slow/obvious version  ──► b        property: a == b

Classic uses: a fast implementation checked against an obvious slow one; a rewrite checked against the old code it replaces; your parser checked against the standard library’s. The reference does not need to be efficient — only correct — so it can be the dumbest possible loop. This is the one pattern where you do get exact-output checking for free, because the reference computes the expected output for you.

3. Invariant: the output is always valid

Section titled “3. Invariant: the output is always valid”

Some rule holds of every valid output, regardless of input. sort’s output is always ordered. A balanced-tree insert always leaves a balanced tree. abs(x) is always >= 0. A shuffled deck still contains exactly the same 52 cards. You assert the invariant without ever computing the expected value.

proptest! {
    #[test]
    fn abs_is_never_negative(x in any::<i32>()) {
        // NOTE: this property FAILS for i32::MIN, whose abs overflows.
        // That failure is the point — see the worked example below.
        prop_assert!(x.wrapping_abs() >= 0 || x == i32::MIN);
    }
}

Invariants are the most general pattern and often the easiest to state: “the result is still a valid X.” If your type has a “well-formedness” check, running it on every output is an instant property.

4. Idempotence: doing it twice equals doing it once

Section titled “4. Idempotence: doing it twice equals doing it once”

Some operations should have no further effect once applied. f(f(x)) == f(x). Sorting an already-sorted list changes nothing. Normalizing an already-normalized path, trimming already-trimmed whitespace, deduplicating a set, abs(abs(x)) — all idempotent.

@given(st.text())
def test_normalize_is_idempotent(s):
    once = normalize(s)
    assert normalize(once) == once

Idempotence catches a sneaky class of bug: an operation that mostly works but leaves a tiny residue, so applying it again changes the result. If normalization is truly complete, the second pass is a no-op — and the generator will find the input where it is not.

Worked example: the property finds a bug the example misses

Section titled “Worked example: the property finds a bug the example misses”

Let us make the promise concrete. Suppose you write a function that splits a string into fixed-width chunks — say for formatting a key into groups of four, "ABCD1234WXYZ" → ["ABCD", "1234", "WXYZ"]. You test it the way most of us would:

def chunk(s, n):
    return [s[i:i+n] for i in range(0, len(s), n)]


# Example tests — all pass, all reasonable:
assert chunk("ABCD1234WXYZ", 4) == ["ABCD", "1234", "WXYZ"]
assert chunk("hello", 2)        == ["he", "ll", "o"]
assert chunk("", 4)             == []

Green. Ship it. Now instead, state a round-trip property: whatever you chunk, gluing the pieces back together must reproduce the original string. You never assert what the chunks are — only that they reassemble.

from hypothesis import given, strategies as st


@given(st.text(), st.integers())
def test_chunk_roundtrips(s, n):
    assert "".join(chunk(s, n)) == s

Run it and Hypothesis reports a failure almost immediately — with n = 0. When n is 0, range(0, len(s), 0) raises ValueError: range() arg 3 must not be zero, or with a naive implementation loops forever. Your three examples all used sensible widths (4, 2, 4); it never occurred to you to test chunk("hello", 0) because no human deliberately chunks into zero-width pieces. The generator has no such taste. It tried 0 because 0 is a boundary, and boundaries are exactly where it aims.

That is the shape of nearly every property win: an off-by-one, an empty input, a zero, a NaN, a lone surrogate in a UTF-8 string — a value in the input space that your examples skipped because you, reasonably, were thinking about the typical case. Widen the generator to st.text() over the full Unicode range and the same property will also flush out any place your chunker miscounts a multi-byte character as several characters. One rule; a whole family of edges surfaced.

The fix is not the lesson (constrain n to be positive, or reject n <= 0 explicitly). The lesson is that the property forced the question — “what should this do when n is 0?” — that your examples let you avoid. Properties are as much a specification tool as a bug-finder: the failing case is the design decision you never made.

Under the hood — how generators reach the edges on purpose

Section titled “Under the hood — how generators reach the edges on purpose”

A good generator is not uniformly random, and that distinction is why property testing works at all. If any::<i32>() drew truly uniformly from $2^{32}$ values, it would essentially never produce 0, 1, -1, i32::MIN, or i32::MAX — the exact values where integer code breaks. So generators are biased toward boundaries. QuickCheck, Hypothesis, and proptest all mix “special” values (zero, one, min, max, empty, the largest allowed size) into their random draws, and they grow complexity over a run: early cases are small and simple, later cases larger and weirder. Hypothesis goes further and remembers past failures in a local database, replaying them first on the next run so a fixed bug stays fixed. The randomness is real, but it is aimed — which is why a few hundred cases routinely find what twenty examples could not, and why “it passed 256 random cases” is stronger evidence than the raw number suggests.

When a property is lazy (and gives false comfort)

Section titled “When a property is lazy (and gives false comfort)”

Honesty, per this Part’s tone: a bad property is worse than no property, because it looks like coverage while asserting almost nothing. Two failure modes to watch:

  • The tautology. If you assert chunk(s, n) == chunk(s, n), or re-implement the function inside the test and compare, you are testing that the code equals itself. It passes a million cases and proves nothing. A property must state a rule independent of the implementation.
  • The over-constrained generator. If you generate only n in 1..=10 and only ASCII text, you have quietly excluded 0 and every multi-byte character — exactly the edges. The generator’s breadth is half the test; narrowing it to make the property pass is how you hide the bug you were trying to find.

A good property is one you cannot make pass by cheating and whose generator is as wide as the real input space. When those two things are true, a green run is genuinely earned confidence.

The tools, by ecosystem

Section titled “The tools, by ecosystem”

You do not need to build any of this; every major ecosystem has a mature library, and they share the generator-plus-property model above. As of mid-2026 the common choices are:

  • Haskell — QuickCheck. The original (2000). Every other tool on this list is a descendant of it.
  • Python — Hypothesis. Strategies as generators, a failure database, excellent shrinking. The de-facto standard.
  • Rust — proptest (and the QuickCheck-style quickcheck crate). proptest is macro-driven with strong shrinking; pick either.
  • JavaScript/TypeScript — fast-check. Integrates with Jest/Vitest/Mocha; arbitraries are its generators.
  • Most other languages have one too — Hedgehog (Haskell/F#/Scala), jqwik (Java), PropCheck (Elixir), Gopter (Go). The vocabulary — generator/arbitrary/strategy, property, shrink — carries across all of them.

Start by adding one round-trip or one invariant property to code you already have example tests for. You are not replacing your examples — the previous page argued they are complementary — you are adding a second oracle that searches the space your examples cannot.

The architect’s lens

Section titled “The architect’s lens”
  • Why does it exist? Because the space of inputs to any real function is astronomically larger than the handful a human will hand-write, and the interesting bugs hide in the region no one thought to type. Property testing exists to let a machine search that region against a rule you state once.
  • What problem does it solve? The test oracle problem: you usually cannot state the exact expected output for a randomly generated input. Properties sidestep it by asserting a relationship the output must obey — round-trip, invariant, idempotence, or agreement with a reference — none of which require knowing the answer in advance.
  • What are the trade-offs? You gain vast, compounding input coverage for one authored rule; you pay with harder-to-write assertions (finding a good property is real design work), non-deterministic-feeling failures until you learn to read them, and the risk that a lazy property or a narrow generator gives false comfort.
  • When should I avoid it? When the behavior is the specific examples — a lookup table, a handful of business rules from a spec, a regression pinned to one bug report — a plain example test is clearer and cheaper. Reach for a property only when a general rule over a space of inputs actually exists.
  • What breaks if I remove it? You lose the only cheap way to probe the edges you did not imagine. Your suite still passes, but its green now means “the cases I thought of work,” not “the rule holds” — a weaker, less justified confidence, and exactly the gap the Ariane 5 and Heartbleed classes of bug slip through.

Check your understanding

Section titled “Check your understanding”
  1. Name the two moving parts of every property test, and say in one sentence what the framework does with them.
  2. State the test oracle problem in your own words, and explain the specific trick properties use to avoid it. Why does that trick let them scale to inputs you have never seen?
  3. For each of these functions, name the property pattern (round-trip, oracle, invariant, idempotence) you would reach for first: (a) a JSON serializer, (b) a new fast matrix multiply replacing an old one, (c) abs(x), (d) whitespace-trimming a string.
  4. In the chunk worked example, the three example tests all passed but the round-trip property failed on n = 0. Explain why a human would skip that case and why the generator would not.
  5. Give one way a property can be “lazy” — passing a million cases while proving almost nothing — and say how you would fix it.
Show answers
  1. A generator (a recipe for producing arbitrarily many inputs of a given shape) and a property (a boolean rule of one input that must hold for every case). The framework repeatedly draws an input from the generator, runs the property on it, and reports the first input for which the property returns false.
  2. The oracle problem: for a randomly generated input you generally cannot state the exact expected output without reimplementing the function, so exact-output checking is impossible at scale. Properties dodge it by asserting a relationship the output must satisfy (ordered, a permutation, reversible, still valid) rather than the output’s exact value — and a relationship can be checked without knowing the answer in advance, which is exactly what lets it apply to inputs you have never seen.
  3. (a) round-trip — decode(encode(x)) == x. (b) oracle — compare the fast version against the trusted old one on the same input. (c) invariant — the result is always >= 0 (minding the MIN-overflow edge). (d) idempotence — trimming an already-trimmed string changes nothing, trim(trim(s)) == trim(s).
  4. A human picks sensible widths (4, 2) because chunking into zero-width pieces is meaningless to a person thinking about the typical case; 0 never comes to mind. The generator has no notion of “sensible” — it deliberately biases toward boundary values like 0, so it tries the zero-width case precisely because it is a boundary, which is where bugs cluster.
  5. A property is lazy if it is a tautology (e.g. comparing the function to itself, or re-implementing it inside the test) or runs against an over-constrained generator (only positive n, only ASCII) that excludes the edges. Fix it by asserting a rule independent of the implementation and widening the generator to match the real input space — including 0, empty inputs, and full Unicode.