Composing the Levels into a Strategy

The pyramid-vs-trophy page gave you a shape — a rough ratio of how many tests to keep at each level. A shape is not a strategy. It tells you you want “many unit tests and few E2E tests,” but it does not tell you which checks belong at which level for the feature in front of you right now. This page turns the five levels — unit, integration, system & E2E, and acceptance — into a decision procedure you can apply one check at a time.

The whole part has argued that confidence is layered: each level catches a class of bug the others structurally cannot. Composition is the skill of placing every check so the layers stack cleanly — no gap where a class of bug goes untested, and no wasteful overlap where the same bug is caught five times and reported five times. Get this right and your suite is a tower where each floor holds up the ones above. Get it wrong and you have either a tower with missing floors, or five floors all built on the same crack.

The composition rule

There is one rule underneath everything on this page:

Push each check to the lowest level that can honestly catch it. Let each higher level cover only what the levels below it cannot.

Two words are load-bearing. Lowest — because lower levels are faster, cheaper, more numerous, and point at a smaller suspect list when they fail. Honestly — because a low-level test only counts if it exercises the real thing that could break, not a mock that will happily agree with a wrong assumption. A unit test that stubs out the database cannot honestly tell you the SQL is valid; that check has to climb to integration, where a real database can reject it.

  a check to place  ─►  can a UNIT test honestly catch it?   ── yes ─►  put it at unit
                              │ no
                              ▼
                        can INTEGRATION honestly catch it?   ── yes ─►  put it at integration
                              │ no
                              ▼
                        can SYSTEM / E2E honestly catch it?   ── yes ─►  put it at system/E2E
                              │ no
                              ▼
                        is it about the RIGHT thing, not      ── yes ─►  put it at acceptance
                        the correct thing?

You walk down the list mentally but you place at the first level that can catch the check honestly. The result is a suite whose center of mass sits low — which is exactly the pyramid the previous page described, arrived at not by decree but by applying one rule to each check.

Why “lowest that can honestly catch it”

Consider a single bug: a discount calculation returns -5% when it should clamp to 0%. Where should the test that guards against it live?

• A unit test on the apply_discount() function can feed it the clamping edge case directly, in microseconds, with no database, no HTTP, no browser. If it fails, the suspect list is one function. This is the lowest level that can honestly catch it — the logic is fully present in that one function.
• An E2E test could also catch it: drive the checkout in a browser, add the item, read the total, assert it is not negative. But that test is thousands of times slower, can flake on timing, and when it fails the suspect list is the entire request path.

Both go green when the code is right and red when it is wrong. They have the same truth value and wildly different cost and precision. The rule says: pick the unit test. Save the E2E slot for something only E2E can prove — that the discount actually reaches the rendered total through real templating, real currency formatting, real rounding. That is a different check, and it belongs at E2E precisely because a unit test cannot honestly make it.

Avoiding redundant coverage

The failure mode on the other side of the rule is redundant coverage: many tests that all fail for the same root cause. This is not “belt and suspenders” safety — it is wasted signal.

Imagine the apply_discount() clamp bug again, but now it is also exercised, incidentally, by an integration test, three system tests, and two E2E journeys that all happen to run a discounted checkout. Introduce the bug and seven tests go red at once. That looks thorough. It is the opposite:

  one root cause  ─►  ✗ unit: apply_discount clamp
                       ✗ integration: order total endpoint
                       ✗ system: checkout flow
                       ✗ system: cart summary
                       ✗ system: receipt render
                       ✗ e2e: guest checkout
                       ✗ e2e: returning-user checkout

  seven red tests, one bug, ~4 minutes of slow suite — to tell you
  what the 3 ms unit test already said.

Redundant coverage costs you three things:

• Signal. Seven red lights do not mean seven problems. You now have to diagnose that they share a cause before you can act — the suite made your job harder, not easier. A good failure points at one thing.
• Speed. The slow-level tests ran (and re-ran on retry) to re-derive a fact the fastest test already delivered. Feedback that could have arrived in milliseconds arrived in minutes.
• Maintenance. When the discount requirement legitimately changes, you now edit seven tests instead of one, and each is a chance to get the update subtly wrong.

The fix is not “delete the higher tests” — it is to make each higher test assert something only that level can prove. The E2E checkout test should stop caring about the exact clamp arithmetic (that is unit’s job) and assert the thing E2E uniquely owns: that a real user, clicking real buttons, reaches a receipt page at all. When the clamp bug lands, only the unit test should go red. Every other level stays green because none of them was secretly re-testing the arithmetic.

By the numbers — redundant vs. layered coverage

Two suites, same feature, same real bug injected. The difference is entirely in where each check lives.

                          redundant suite        layered suite
  tests that go red         7                      1
  time to first red         ~180 s (waits on E2E)  ~3 ms (unit)
  suspect surface           whole request path     one function
  tests to edit on a
    legit requirement       7                      1

  same confidence. one suite tells you the answer in 3 ms and points
  at the function; the other tells you in 3 minutes and points at "somewhere."

The numbers are illustrative, not measured law — but the shape is the point: layered coverage is faster and more precise for the identical bug, because no test re-proves what a lower test already proved.

Matching the shape to feedback speed

The composition rule sorts checks by what they can catch. Feedback speed sorts them by when you can afford to run them. These two orderings line up almost perfectly, and that is not a coincidence — the levels that catch the smallest, most local bugs are also the fastest, so you can afford to run them constantly.

  fast, run constantly                              slow, run rarely
  ◄──────────────────────────────────────────────────────────────►
  lint · type-check · unit      integration      system · e2e      acceptance
       │                             │                 │                │
   every save / every            every push /     before merge /    before release /
   commit (seconds)              pre-push (tens    per PR (minutes)  per milestone
                                  of seconds)

The practical rule: fast levels gate every change; slow levels gate the transitions that matter. You run lint, type-checks, and the unit suite on every commit — even on every file save in a watch loop — because they finish before you have context-switched. You do not run the full E2E suite on every keystroke; you run it before a merge or a release, where its minutes buy you the confidence to promote code past a boundary that is expensive to walk back.

Where each level runs in the pipeline

This maps directly onto CI gates. Each gate is a place where slower, higher-confidence tests earn their cost because the thing on the other side of the gate is harder to undo.

   developer's machine        push to branch          open / update PR         merge to main → release
   ┌──────────────────┐      ┌──────────────────┐    ┌──────────────────┐     ┌──────────────────────┐
   │ lint             │      │ lint             │    │ everything left  │     │ everything left      │
   │ type-check       │  ►   │ type-check       │ ►  │ + integration    │  ►  │ + system / e2e       │
   │ unit (watch)     │      │ unit (full)      │    │   tests          │     │ + acceptance / smoke │
   └──────────────────┘      └──────────────────┘    └──────────────────┘     └──────────────────────┘
     seconds, every save       ~30 s, every push       minutes, per PR           minutes+, per release

The gating principle: the cost of the test should be paid where the cost of being wrong is incurred. A logic bug is cheap to fix on your machine, so guard it with a test that runs on your machine. Shipping a broken user journey to production is expensive, so guard that with E2E tests at the merge/release gate — the last cheap place to catch it before it becomes an incident. Pushing an E2E suite down to every-save would make saving unbearable; pulling unit tests up to release-only would let logic bugs marinate for hours. The alignment of speed and scope is what lets each gate hold the right line.

A worked example: one feature across all five levels

Take a concrete feature from the book’s kvlite key-value store: a SET key value command over the TCP server that must persist to the write-ahead log and survive a restart. Here is every check the feature deserves, each placed at the lowest level that can honestly catch it, and — crucially — why it is not one level lower.

  FEATURE: SET persists a key and survives restart

  UNIT         parse_command("SET k v") → Command::Set{k,v}
               why here:   pure string→struct logic, no I/O needed
               why not lower: nothing is lower

  UNIT         Db::set then Db::get returns the value (in-memory map)
               why here:   the map logic is fully present in one struct
               why not e2e: no socket or restart needed to prove the map works

  INTEGRATION  Db::open writes to a real WAL file; reopen re-reads it
               why here:   needs a REAL file — a mocked FS would lie about durability
               why not unit: durability across reopen is exactly what a mock can't honestly assert

  INTEGRATION  server round-trip: connect, "SET k v", "GET k" → v over a real socket
               (this is kvlite/tests/server.rs — bind :0, spawn server, real TcpStream)
               why here:   needs the real command loop + real TCP framing wired together
               why not unit: the wiring between parser, store, and socket is the point

  SYSTEM/E2E   boot the built binary, set a key, KILL the process, restart, GET the key
               why here:   only a real process restart proves the WAL replay path on startup
               why not integration: an in-process reopen skips real startup, arg parsing, signal handling

  ACCEPTANCE   "As an operator, keys I set survive a crash" — Given a running store,
               When I SET a key and the process crashes, Then after restart GET returns it
               why here:   this is the REQUIREMENT, phrased in the operator's words, not the code's
               why not system: system proves it works; acceptance proves it's the promised behavior

Read down that column and the composition rule is visible in every row. Each check sits at the first level that can make it honestly, and each higher level asserts something a lower one structurally could not:

• The parser and in-memory map are unit work — no I/O, so no reason to climb.
• Durability needs a real file, so it climbs to integration; a mock of the filesystem would cheerfully report success and teach you nothing about whether data actually lands on disk. (This is the kvlite/tests/server.rs and WAL-reopen family of tests.)
• Surviving a real crash needs a real process boundary — a kill and a fresh start — so it climbs to system/E2E, because an in-process reopen() never exercises the program’s actual startup path.
• And the promise “keys survive a crash” is a requirement, so it earns exactly one acceptance test, phrased in the operator’s language, that would still make sense to a non-programmer.

Now check the redundancy ledger: inject the WAL-not-flushed bug and only the durability integration test and the crash-survival system test go red — the two levels that touch real persistence. The parser unit test stays green (it never touched the file). The acceptance test is the one that would go red if we built the wrong promise, not if we built this promise incorrectly. No two rows fail for the same reason. That is a composed strategy: five levels, each earning its place, none re-proving another’s fact.

The architect’s lens

• Why does it exist? Because a shape (pyramid or trophy) tells you the ratio of tests but not which check goes where. Composition is the missing decision procedure that turns a ratio into a per-check placement rule.
• What problem does it solve? Two problems at once: gaps (a class of bug no level tests) and overlaps (many tests failing for one root cause). It gives you a single rule — lowest honest level — that closes gaps while eliminating wasteful redundancy.
• What are the trade-offs? It demands discipline per check: you must ask “can a lower level honestly catch this?” every time, and resist the comfort of “just add another E2E test.” Under-applied, you drift toward slow, redundant suites; over-applied as dogma, you might skip a genuinely valuable higher-level check that guards a real seam.
• When should I avoid it? Never as a principle, but relax it for a tiny script or a spike where any test at all is the win. And do keep deliberate overlap at true risk boundaries — a smoke test on a critical path may be worth its redundancy for the fast alarm it gives, as long as you chose it knowingly.
• What breaks if I remove it? You keep the levels but lose the strategy: tests accrete wherever they are easiest to write, the suite’s center of mass drifts upward into slow, flaky territory, one bug lights up seven tests, and real gaps hide behind the noise. You are paying for a tower and getting a pile.

Check your understanding

1. State the composition rule in one sentence, and explain what the word honestly forbids.
2. A discount-clamp logic bug can be caught by both a unit test and an E2E test that both go green when the code is right. If they have the same truth value, why does the rule still say to put it at unit?
3. Redundant coverage means “many tests fail for one root cause.” Name the three concrete costs that redundancy imposes, and the fix that is not “delete the higher tests.”
4. Why do feedback speed and scope line up so that fast levels gate every commit while slow levels gate merges and releases — and what would break if you ran E2E on every file save, or ran unit tests only at release?
5. In the kvlite worked example, the durability check climbs from unit to integration. What specifically makes a unit test unable to honestly catch a “data must survive a restart” bug?

Show answers

1. Push each check to the lowest level that can honestly catch it, and let higher levels cover only what lower ones cannot. Honestly forbids counting a low-level test that only passes because a mock stood in for the real thing that could break — e.g. a unit test that stubs the database cannot honestly claim the SQL is valid, so that check must climb to integration.
2. Because truth value is only one axis; cost and precision are the other. The unit test is thousands of times faster and, when red, points at one function instead of the whole request path. Same answer, far cheaper and more precise — so the E2E slot is saved for a check only E2E can make (the discount reaching the rendered total).
3. Costs: (1) signal — many red lights are one problem you must first diagnose as shared before acting; (2) speed — slow levels re-derive a fact the fastest test already gave you; (3) maintenance — a legitimate requirement change means editing many tests, each a chance for error. The fix is to make each higher test assert only what its level uniquely proves, so a given bug turns exactly one test red.
4. The levels that catch the smallest, most local bugs are also the fastest, so you can afford to run them constantly; the levels that prove whole-system behavior are slow, so you spend them at boundaries that are expensive to walk back (merge, release). Running E2E on every save would make editing unbearable and destroy the fast feedback loop; running unit tests only at release would let cheap logic bugs marinate for hours before anyone found out. The alignment of speed and scope is what lets each gate hold the right line.
5. Durability across a restart is precisely the thing a mock cannot assert. A unit test fakes out the filesystem, so it can only prove the code called the write API — a mocked FS will happily report success whether or not bytes ever reach disk. Proving the data actually persists and re-reads needs a real file (integration) — and surviving a real crash needs a real process restart (system/E2E).