Integration Tests — Where Units Meet

The previous page, Unit Tests, taught the fast, isolated, many end of the spectrum: each test exercises one unit with its collaborators replaced by fakes, so a failure points at exactly one place. That isolation is the source of their speed — and also their blind spot. A unit test can prove that your code does the right thing when everything around it behaves as you imagined. It cannot prove that everything around it actually behaves that way.

Integration tests fill that gap. They wire two or more real parts together — or wire your unit against a real collaborator like a Postgres database or an HTTP client — and check that the whole assembly works. They are slower, fewer, and more finicky than unit tests, and every one of those costs buys you something a unit test can never see: the truth about the seams.

The core value: bugs live in the seams

Draw any system as boxes and lines. The boxes are units — functions, classes, modules. The lines are the seams: the function calls, the SQL queries, the HTTP requests, the serialized messages that pass between boxes.

   ┌────────────┐   SQL    ┌────────────┐   HTTP   ┌────────────┐
   │  Handler   │ ───────► │ Repository │ ───────► │  Payments  │
   │  (unit)    │          │  (unit)    │          │  API (ext) │
   └────────────┘          └────────────┘          └────────────┘
        ▲                        ▲                       ▲
    unit tests             unit tests               unit tests
    cover here             cover here               cover here
        └───────── but nobody tested the LINES ─────────┘

Unit tests, by design, cut every line and replace the far end with a fake. That is exactly why a suite can be green while the system is broken: each box works in isolation, and no test ever crossed a line. The handler assumes the repository returns a User; the repository assumes the column is named user_id; the payments client assumes the API returns cents, not dollars. Each assumption is invisible inside a single box. It only becomes a bug when two boxes meet and disagree.

This is the first principle of this level: only a test that crosses a boundary can find a bug that lives on that boundary. Integration tests exist to cross boundaries on purpose.

What integration catches that unit tests miss

The seams fail in a small number of recurring ways. Knowing them tells you what to point an integration test at.

Serialization and encoding. Your object round-trips through JSON, a database row, or a byte stream and comes back subtly wrong: a timestamp loses its timezone, a Decimal becomes a lossy float, an empty list becomes null. In the repo’s real server test, this exact concern is checked on the wire — SET name ada lovelace must come back as VALUE ada lovelace, proving a value with spaces survives the wire and the line protocol did not mangle it.
Schema mismatches. Your code expects a column, table, or field that the real database does not have — or has under a different name or type. A fake repository will happily return whatever your test told it to; a real Postgres will reject the query.
Contract drift. An external API changed its response shape, added a required header, or started returning 429. Your hand-written fake still models last year’s contract, so your unit tests stay green while production breaks.
Misconfigured wiring. The dependency-injection container hands the wrong implementation to the wrong constructor; a connection string points at the wrong database; a middleware is registered in the wrong order. None of this is visible until the real objects are assembled and run.

Every item on that list is a disagreement between units, not a bug inside one. That is the whole reason this level has to exist.

Real vs faked collaborators

The central design decision of an integration test is: for the collaborator on the other side of the seam, do you use the real thing or a fake?

The honest rule is: use the real collaborator when the seam’s behavior is the thing under test. If the bug you are hunting is a schema mismatch or a SQL dialect quirk, a fake database cannot catch it by definition — the fake is your model of the database, and the bug is precisely where your model is wrong. Test against a real Postgres.

A fake is honest enough when the collaborator is (a) not the subject of the test, (b) slow, flaky, or costly to run for real, and (c) governed by a contract you trust and test elsewhere. A third-party payments API is the classic case: you do not want your CI suite charging real cards, so you fake it — but you must also run a separate, smaller contract test against the real API to catch drift, or your fake slowly becomes fiction.

   Collaborator            Real in integration tests?
   ─────────────────────   ──────────────────────────
   Your own database       YES — schema, SQL, tx behavior are the point
   Redis / cache           USUALLY — eviction & TTL bugs live here
   Filesystem / temp dir   YES — cheap, and encoding bugs are real
   Third-party paid API    NO in the main suite; contract-test separately
   Email / SMS provider    NO — fake it; assert the request was well-formed
   Wall-clock time         NO — inject a clock; time is not a collaborator you test

The trap to avoid is a fake so elaborate it re-implements the collaborator. At that point you are testing your fake, not your code, and you have paid all the cost of an integration test for none of the truth.

Integration tests are slower because real collaborators have real startup and I/O costs. Ballpark, order-of-magnitude figures (these vary widely by machine and setup):

  Operation                         Rough cost
  ────────────────────────────────  ────────────
  In-memory fake call               ~0.001 ms
  Start a Postgres container         500–3000 ms   (once per suite, if reused)
  One real INSERT + SELECT           1–10 ms
  One real HTTP round-trip (local)   1–20 ms

  A unit suite:   2000 tests × 0.05 ms   ≈   0.1 s
  An integ suite:  200 tests × 8 ms  + 2 s startup ≈  3.6 s

The integration suite is ~30x fewer tests yet still ~30x slower per test. That ratio — fewer but heavier — is why the levels form a pyramid, covered in The Pyramid vs the Trophy.

Why they cost more

Three forces make integration tests slower and more fragile than unit tests, and each one is a direct consequence of using real parts.

Process startup. A real database, message broker, or HTTP server must be running. Spinning one up — even a throwaway container — costs seconds. The repo’s real integration test avoids a separate process by spawning the server in-thread and binding to port 0 so the OS hands it a free port, then connecting a real client socket:

// From kvlite/tests/server.rs — a real TCP round-trip, no mocks.
let (db, _) = Db::open(&path).unwrap();
let listener = TcpListener::bind("127.0.0.1:0").unwrap(); // OS picks the port
let addr = listener.local_addr().unwrap();
thread::spawn(move || { let _ = server::serve_on(listener, db); });

let stream = TcpStream::connect(addr).unwrap();
// ... exchange real lines over the wire ...
assert_eq!(round_trip(&mut reader, &mut writer, "PING"), "PONG");

Even this lightweight approach pays for socket setup, a background thread, and a file-backed write-ahead log on disk — costs a pure unit test never incurs.

Real I/O. Disk writes, network round-trips, and TLS handshakes are orders of magnitude slower than a function call, and they can fail for real — a port is taken, a container is slow to accept connections, a disk is full. That is why integration tests need retries and generous-but-bounded timeouts where unit tests need neither.

Shared state and ordering. The moment two tests touch the same real database, they can see each other’s data. A test that assumes an empty table will fail if a prior test left a row behind — and worse, it will fail only in some orderings, making the failure look random. Unit tests dodge this because each one gets a fresh fake. Integration tests must earn their independence deliberately, which is the next section.

Keeping integration tests deterministic

An integration test that shares real state is only trustworthy if each test is independent (its result does not depend on which other tests ran) and deterministic (the same input always gives the same result). Two disciplines get you there.

Isolate the data each test sees

Give every test its own slice of the shared collaborator so tests cannot see each other. In order of preference:

A fresh namespace per test. A unique schema, database, key prefix, or — as in the real server test above — a temp file whose name embeds the process ID and an atomic counter, so parallel tests never collide:
```
std::env::temp_dir().join(format!(
    "kvlite-srv-{}-{}.wal",
    std::process::id(),
    N.fetch_add(1, Ordering::Relaxed)  // unique per test
))
```
A transaction rolled back at teardown. Run the test inside a database transaction and ROLLBACK instead of COMMIT. Fast and thorough, but it hides bugs that only appear across commits.
Truncate/clean between tests. Simplest to reason about, but forces serial execution and is the slowest option.

Make teardown unconditional

The classic flaky-suite bug: a test creates data, asserts, and cleans up — but the assertion fails, so cleanup never runs, poisoning every later test. Teardown must run whether the test passed or failed (a fixture, an after hook, a Drop guard, a defer). Notice the real test removes its WAL file at both the start and the end, so a crashed prior run cannot corrupt the next one:

let _ = std::fs::remove_file(&path); // before: don't inherit stale state
// ... run the test ...
let _ = std::fs::remove_file(&path); // after: don't leak state forward

Two more determinism rules earn their keep at this level: never depend on test order (if suite A must run before suite B, you have a hidden dependency, not two tests), and never let a real clock or random seed leak in — inject them, so “expires in 5 minutes” is a controlled input, not a race against the wall clock.

On 1 August 2012, a trading firm deployed new code to its servers but missed one of eight machines. That one server still ran old logic behind a reused configuration flag, so the fleet was internally inconsistent — the deployed units disagreed about what the flag meant. In roughly 45 minutes the firm lost about $440 million and was effectively finished. No single unit was “wrong” in isolation; the failure lived in the wiring between deployed components and their shared configuration — exactly the misconfigured-wiring seam that only a test assembling the real, fully-configured system can catch. Unit tests on each component would have stayed green. Hedge the exact numbers; keep the lesson: the seam between correctly-built parts is its own attack surface, and it needs its own tests.

The architect’s lens

Why does it exist? Because a suite of green unit tests can still describe a broken system: every box works alone, but no test ever crossed the lines between them. Integration tests cross those lines on purpose.
What problem does it solve? It catches the bugs that only exist between units — serialization, schema mismatches, contract drift, and misconfigured wiring — none of which a single isolated unit can reveal.
What are the trade-offs? Truth about the seams in exchange for speed and simplicity: process startup, real I/O, and shared state make these tests slower, flakier, and more order-sensitive, so you keep them fewer.
When should I avoid it? When a fake is honest enough — the collaborator is not the subject of the test, is costly or flaky to run for real, and is covered by a contract you test elsewhere. Don’t integration-test pure logic that a unit test already pins down.
What breaks if I remove it? The seams go untested. Your fakes drift from reality, a schema change or a changed API response ships green, and the failure surfaces in production instead of CI — as an outage, not a red test.

Check your understanding

A team has 100% unit-test coverage and every test is green, yet a schema change to the users table takes down production. Explain, using the boxes-and-lines model, how both facts can be true at once.
Give two concrete classes of bug that an integration test can catch but a unit test cannot by design, and say why the unit test is structurally blind to each.
You need to test code that calls a third-party payments API. Would you use the real API or a fake in your main CI suite? Justify the choice, and name the additional test you still owe to stay safe.
Two integration tests pass when run alone but one fails when they run together. What is the most likely root cause, and name two techniques that make each test independent.
Why must integration-test teardown run even when the test’s assertions fail? Describe the specific failure mode that appears if it doesn’t.

Show answers

The boxes (units) are each correct in isolation and their unit tests prove it, but unit tests replace every collaborator with a fake — they cut the lines. The schema mismatch lives on the line between the repository unit and the real database, and no unit test ever crossed that line, so nothing was ever red.
Any two of: serialization/encoding bugs (a fake returns your object unchanged, so a lossy round-trip through JSON or a DB row is never exercised); schema mismatches (a fake returns whatever you told it to, so a missing or renamed column is never hit); contract drift (a hand-written fake models the old contract, so a changed real API stays invisible); misconfigured wiring (fakes are injected by hand, so a wrong real binding is never assembled). In each case the fake is the assumption being tested, so it can never contradict itself.
Use a fake in the main suite: a real API is slow, flaky, and charging real cards in CI is unacceptable. But you still owe a separate, smaller contract test run against the real API (on a schedule or in a nightly job) to detect drift before your fake becomes fiction.
The most likely cause is shared state in a real collaborator — one test leaves data (or global config) the other test sees. Independence techniques (any two): a fresh namespace/schema/key-prefix per test; a transaction rolled back at teardown; truncating between tests; or embedding a unique id (PID + counter) in the resource name.
Because a failed assertion that skips cleanup leaves stale data behind, which poisons every later test that assumes a clean slate — turning one real failure into a cascade of misleading ones, often order-dependent and “random”-looking. Unconditional teardown (fixture/after hook/Drop/defer) keeps each test’s failure contained to itself.