Wire the Tests into CI

On the previous page we finished Snip’s test suite: hundreds of unit tests, dozens of integration tests against a real Postgres, a thin cap of end-to-end tests over real HTTP, and a coverage report with an 80% floor and critical paths at 100%. Right now all of that runs when you remember to run it, on your machine, with your database. That is a single point of failure wearing a hoodie.

This page removes the human. We take the exact same suites and hand them to a machine that runs them on every commit, against a real database, and refuses to let a red build merge. The tests stop being something you choose to run and become something the repository enforces. That shift — from one-time verification to continuous verification — is the whole payoff of CI, and it is the last piece that turns a pile of good tests into justified, durable confidence.

What “continuous integration” actually means here

Continuous integration (CI) is a plain idea: every time code changes, a server automatically rebuilds it and runs the checks, so integration problems surface within minutes of the commit that caused them instead of days later when someone finally runs the suite. For our purposes the definition is even narrower and more concrete.

   developer pushes commit / opens PR
            │
            ▼
   ┌─────────────────────────────────────────────┐
   │  CI runner (a fresh, clean machine)          │
   │    1. check out the code                     │
   │    2. install + cache dependencies           │
   │    3. run unit tests        (fast)           │
   │    4. run integration tests (real Postgres)  │
   │    5. run e2e tests         (slow, whole app)│
   │    6. measure coverage, enforce the floor    │
   └─────────────────────────────────────────────┘
            │
      pass ─┴─ fail
       │        │
       ▼        ▼
   merge      block the merge  ◄── the gate
   allowed        + show which test failed

Two properties make this worth doing. First, the runner is a fresh, clean machine every time — no leftover files, no “works on my laptop” state, no database someone hand-edited last Tuesday. If the tests pass there, they pass from zero. Second, the result is a required check: a red run is not a suggestion the reviewer can wave through, it is a locked door on the merge button.

Run the suites in order: fast first, slow last

The ordering rule falls straight out of the test pyramid. Run the cheapest, most-diagnostic tests first so that a broken commit fails in seconds, not after the pipeline has spent minutes booting a whole stack for e2e. A developer who typos a function signature should learn that from the unit stage almost immediately — not four minutes later at the end of the e2e stage.

   stage        cost      fails in     why this order
   ---------------------------------------------------------------
   unit         cheap     ~seconds     most bugs, fastest signal
   integration  medium    ~seconds     needs a real DB service
   e2e          expensive ~minutes     boots the whole app; run last
   coverage     cheap     ~seconds     computed from the runs above

Here is a minimal GitHub Actions workflow that does exactly this. The syntax is one tool’s; the shape — checkout, cache, install, then the suites in ascending cost — is universal and transfers to GitLab CI, CircleCI, or anything else.

name: ci
on:
  push:
  pull_request:          # run on every PR so the gate has something to gate

jobs:
  test:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v4

      - uses: actions/cache@v4        # cache deps — see "deterministic and fast" below
        with:
          path: ~/.cargo
          key: cargo-${{ hashFiles('Cargo.lock') }}

      - name: unit tests (fast, no I/O)
        run: cargo test --lib

      - name: integration tests (real Postgres)
        run: cargo test --test store
        env:
          DATABASE_URL: postgres://snip:snip@localhost:5432/snip_test

      - name: e2e tests (boots the whole app, slowest)
        run: cargo test --test e2e

      - name: coverage floor
        run: cargo llvm-cov --fail-under-lines 80

The build fails on any failure by construction: each run step is a shell command, a non-zero exit code stops the job, and any failing test makes the runner exit non-zero. There is no “continue on error” — a red test at any level turns the whole run red. That is the point. A pipeline that keeps going after a failure and reports green anyway is worse than no pipeline, because it manufactures false confidence.

Spin up the real database as a CI service

Our integration tests are only worth anything because they run against a genuine Postgres, not a fake (why that matters). So the CI machine needs a real Postgres too. The wrong way is to install and configure one by hand in a script; the right way is to declare it as a service container and let the runner start a throwaway database next to the job.

  test:
    runs-on: ubuntu-latest
    services:
      postgres:                       # a real Postgres, started fresh per run
        image: postgres:16
        env:
          POSTGRES_USER: snip
          POSTGRES_PASSWORD: snip
          POSTGRES_DB: snip_test
        ports:
          - 5432:5432
        options: >-
          --health-cmd "pg_isready -U snip"
          --health-interval 5s
          --health-timeout 3s
          --health-retries 5
    steps:
      - uses: actions/checkout@v4
      # ... the test steps from above; DATABASE_URL points at localhost:5432

Three details earn their keep. The container is a real postgres:16 image, so the tests exercise the same SQL engine, the same type coercions, and the same transaction semantics as production — the very bugs a mock would hide. It is fresh every run, so no test can depend on data another run left behind. And the health check matters more than it looks: without pg_isready, the test step can start before Postgres is accepting connections and fail with a confusing “connection refused” that has nothing to do with your code. The runner waits for the health check to go green before it starts your tests.

The recurring pattern behind a whole class of production incidents is not exotic: a system passes its tests against a substitute datastore, then behaves differently against the real one under real load. In-memory fakes silently ignore constraints, connection limits, lock contention, and timeout behavior that a genuine database enforces. A well-documented shape of outage is a service that tested green for months and then fell over the first time production traffic hit a database limit — a unique constraint, a connection ceiling, a lock wait — that the test double never had. The lesson is exactly why we run Postgres as a CI service: if the integration tests do not talk to the real engine, they are not testing the thing that breaks. CI is where you make “real database” cheap enough to do on every commit instead of never.

Publish coverage and make the floor a required check

We chose an 80% coverage floor with critical paths at 100% on the last page. A floor nobody enforces is a wish. CI turns it into a gate in two moves.

First, fail the job when coverage drops below the floor. That is the --fail-under-lines 80 step above: the coverage tool exits non-zero if the number falls under 80%, which turns the whole run red exactly like a failing test does. A pull request that adds a feature with no tests will drag the percentage down and be stopped, not merged and cleaned up “later.”

Second, publish the report so humans can read it. The number gates the merge; the report tells you which branches are missing so a reviewer can point at them.

      - name: coverage report
        run: cargo llvm-cov --html --output-dir coverage/
      - uses: actions/upload-artifact@v4
        with:
          name: coverage-report
          path: coverage/

The final, load-bearing step is outside the YAML entirely: in the repository settings, mark the test job as a required status check on the protected main branch. This is what makes the whole page real. Without it, CI is advisory — a red X next to a PR that a hurried human can still merge. With it, the platform itself refuses to merge a branch whose checks are not green.

   Branch protection on `main`:
     ✔ require the `test` check to pass before merging
     ✔ require the branch to be up to date first
     ✔ no merge button until the check is green

The gate and the report are a division of labor: the gate is the machine saying no, the report is the machine saying here is exactly where.

Make the pipeline deterministic and fast

A CI pipeline has two jobs beyond running the tests: it must give the same answer for the same code (deterministic), and it must give it quickly (fast), or developers will route around it. Four techniques carry most of the weight.

Cache dependencies

Downloading and compiling every dependency from scratch on every run is pure waste — the same inputs produce the same output. Cache them, keyed on the lockfile, and a run reuses the previous build’s dependencies whenever Cargo.lock (or package-lock.json, or poetry.lock) has not changed. This alone often turns a multi-minute cold build into a sub-minute warm one.

Run independent suites in parallel

The suites in our single test job run in sequence because the later ones depend on the earlier passing. But independent work should run at the same time. If unit tests need no database and a linter needs nothing at all, split them into separate jobs that run concurrently.

   sequential (one job):  unit ─► integ ─► e2e ─► cover        total = sum
   parallel (many jobs):  ┌ lint ─────────────────┐
                          ├ unit ─────────────────┤   total = the
                          └ integ ─► e2e ─► cover ─┘   longest branch

Wall-clock time drops to the length of the longest branch, not the sum of all of them. The constraint: only parallelize genuinely independent work, or a shared resource (one database, one port) turns parallelism into flakiness.

Seed randomness

Any test that uses randomness — a random short code, a shuffled input, a fuzz seed — must run from a fixed seed in CI. A test that generates a fresh random value each run is a test that passes 999 times and fails the 1000th for reasons no one can reproduce, because the failing input is already gone. Fix the seed and a failure is reproducible: the same run produces the same inputs, so a red build can be re-run locally and debugged instead of shrugged at.

   random each run:  passes... passes... FAILS (input lost) ─► "flaky, just re-run it"
   fixed seed:       same inputs every run ─► a failure reproduces ─► you can fix it

The deeper principle is that flaky tests are worse than no tests. A suite that fails intermittently for non-bug reasons trains the team to hit “re-run” on red — and a team that reflexively ignores red will eventually ignore the red that was a real bug. Determinism is not a nice-to-have; it is what keeps the gate trusted, and an untrusted gate is a gate people learn to climb over.

Representative wall-clock times for Snip’s pipeline, cold versus tuned:

  stage                 cold run     tuned run     technique
  ---------------------------------------------------------------------
  install deps          ~180 s       ~15 s         dependency cache
  unit tests            ~5 s         ~5 s          (already fast)
  integration tests     ~20 s        ~20 s         real Postgres service
  e2e tests             ~30 s        ~30 s         runs last, and in a
  lint (independent)    ~25 s        0 s           parallel branch, so it
                                                   overlaps the e2e branch
  ---------------------------------------------------------------------
  total wall time       ~260 s       ~70 s

Same tests, same assertions — roughly a 3.7x speed-up purely from caching deps and overlapping the independent lint job. The math matters because feedback time is behavioral: a ~70-second pipeline gets waited on; a ~4-minute one gets context-switched away from, and stale results get merged.

The payoff: confidence becomes continuous

Step back and see what changed. Before this page, every test we wrote was a one-time verification — it told you the code was correct at the moment you ran it, on the machine you ran it on. The instant someone else pushed a commit, that knowledge went stale, and nobody re-earned it until they remembered to.

CI makes verification continuous. Every commit, by every person, on a clean machine, against a real database, is re-checked against the entire risk map, the entire suite, and the coverage floor — automatically, before it can merge. Confidence is no longer a thing you establish once and watch decay; it is a property the repository maintains. A regression that would have slipped in unnoticed now turns a build red within a minute and blocks the merge that introduced it. That is the difference between “we tested it” (past tense, decaying) and “it is tested” (present tense, enforced) — and it is the entire reason the tests from the last five pages were worth writing.

The architect’s lens

Why does it exist? CI exists because human-run tests decay the instant someone else changes the code — it moves verification from a thing people remember to do to a thing the repository enforces on every commit automatically.
What problem does it solve? It removes “works on my laptop,” catches regressions within minutes of the commit that caused them, and makes green tests and a coverage floor a precondition for merging rather than a courtesy.
What are the trade-offs? You pay in pipeline maintenance and wait time, and a slow or flaky pipeline actively harms you by training the team to ignore or route around it — so caching, parallelism, and determinism are not optional polish.
When should I avoid it? Almost never for shared code; the honest caveats are throwaway spikes and one-person prototypes where the ceremony outweighs the payoff. Even then, the moment a second person or a deploy target appears, you want it.
What breaks if I remove it? Verification falls back on human memory: regressions merge unnoticed, the “real database” gets replaced by whatever is on someone’s laptop, the coverage floor becomes a wish, and confidence reverts to a one-time thing that decays with every commit.

Check your understanding

Why does the pipeline run unit tests before integration and e2e tests instead of in the reverse order, or all at once?
What does running Postgres as a service container buy you over installing and configuring a database by hand in the CI script? Why does the health check matter?
Marking the test job as a “required status check” is a settings change outside the YAML. Why is it the step that makes the whole page real?
Name two techniques that make the pipeline deterministic and fast, and explain the concrete failure each one prevents.
Restate the payoff in the book’s terms: how does CI change confidence from “one-time” to “continuous,” and why does that make the previous five pages of tests worth writing?

Show answers

Because feedback speed is the point of the pyramid: unit tests are the cheapest and most diagnostic, so running them first means a broken commit fails in seconds and names one function, instead of failing minutes later after the runner has booted a whole stack for e2e. Running them all at once (in one sequential job) would still make every developer wait for the slow stages before learning about a trivial unit-level break. Cheapest-and-most-diagnostic first is the ordering that minimizes time-to-signal.
A real service container runs a genuine Postgres image, so the integration tests exercise the same SQL engine, type coercions, transaction semantics, and constraint enforcement as production — the exact behaviors a mock or hand-rolled fake would hide — and it starts fresh every run, so no test can depend on leftover data. The health check matters because without it the test step may start before Postgres is accepting connections, producing a confusing “connection refused” unrelated to your code; the runner waits for the check to go green before running tests.
Everything else in the pipeline only reports a result — a red X next to a PR. The required-check setting is what makes the platform itself refuse to merge a branch whose checks are not green, turning the pipeline from advisory (a hurried human can still click merge) into an actual gate (there is no merge button until it passes). Without it, CI is a suggestion; with it, CI is enforcement.
Any two: caching dependencies (keyed on the lockfile) prevents wasteful re-downloading/re-compiling of unchanged deps, cutting a multi-minute cold build to a sub-minute warm one and keeping the pipeline fast enough that developers wait for it. Seeding randomness with a fixed seed prevents flaky, non-reproducible failures — a random-input test that fails once and loses the failing input — so a red build reproduces locally and gets fixed instead of shrugged off. (Running independent suites in parallel — cutting wall time to the longest branch instead of the sum — is also acceptable.)
Before CI, each test verified the code only at the moment and on the machine it was run, so the knowledge went stale the instant anyone else pushed a commit — confidence was one-time and decaying. CI re-runs the entire suite, against a real database, on a clean machine, on every commit before it can merge, so verification is continuous and the repository maintains confidence rather than losing it. That is what makes the earlier pages worth writing: their tests now protect every future commit automatically, instead of protecting only the moment someone remembered to run them.