Verification vs Validation — Building It Right vs Building the Right Thing

The previous page, The Cost of a Bug — Shift Left, argued that a defect gets more expensive the later you catch it, so we should push checking as early as possible. But “checking” is not one activity. There is a subtle fork hiding inside the innocent question is it correct?, and teams that never notice the fork ship polished, well-engineered software that solves the wrong problem.

The fork is this. “Correct” can mean the software does what the specification says — or it can mean the software does what the user actually needed. Those are not the same question, they use different evidence, and they fail in different ways. The first is verification. The second is validation.

This page is about telling them apart, because confusing them is one of the oldest and most expensive mistakes in the field. By the end you should be able to look at any bug report, any test, or any “done” claim and say which of the two questions it actually answers — and, just as importantly, which one it leaves wide open.

Two questions wearing one word

Barry Boehm’s classic one-liner is still the fastest way to lock the distinction in:

  Verification:  "Are we building the product RIGHT?"
                 Does the software conform to its specification?

  Validation:    "Are we building the RIGHT product?"
                 Does the software meet the user's real need?

Read them slowly. Verification looks inward, comparing the built thing against the written spec. Validation looks outward, comparing the built thing (or the spec itself) against reality — the user, the market, the actual job to be done.

Here is why they are genuinely different. A specification is a human artifact. Someone wrote it down, and that someone can be wrong. If the spec says “round tax to the nearest cent using banker’s rounding” and the law actually requires rounding half-up, then a program that perfectly matches the spec is perfectly, provably, verifiably wrong in production. Verification cannot catch that error, because verification’s yardstick — the spec — is the thing that is broken.

There is a deeper reason the two questions cannot collapse into one. Verification is, in principle, a closed problem: the spec is finite, written down, and comparable to the code by rules. Given enough effort you can decide whether the code matches it. Validation is an open problem: the “real need” is not fully written anywhere, it lives partly in users’ heads and partly in a market that is still moving, and it can change after you ship. You can never fully “finish” validation the way you can, in principle, finish verifying a fixed spec — you can only gather more evidence that you are still building the right thing. That asymmetry is why they need different tools, different cadences, and different people.

        did we build       did we build
        it right?          the right thing?
             │                    │
        ┌────▼─────┐         ┌────▼──────┐
        │  spec    │         │  user's   │
        │ conform? │         │  real need│
        └────┬─────┘         └────┬──────┘
             │                    │
        VERIFICATION          VALIDATION
       (against the spec)   (against reality)

Verification — did we build it right?

Verification answers: does the software meet its specification? Its reference point is a document — a requirement, an API contract, a design, an acceptance criterion someone agreed to. The question has a crisp shape: for every rule in the spec, does the code obey it?

Because the reference point is written down, verification is where the mechanical techniques live. You can automate most of it:

Unit and integration tests assert that a function or module produces the specified output for a given input.
Static analysis, type checking, and linters verify structural rules (“no null dereference”, “this field is non-empty”) without running the code.
Code reviews and inspections check the implementation against the design and the standard.
Contract tests verify that a service’s responses match the agreed schema.

A verification test in Rust looks like a direct statement of a spec clause — “an empty store returns nothing for any key”:

#[test]
fn get_on_empty_store_returns_none() {
    let store = KvStore::new();
    // Spec says: a key never written returns None, not an error.
    assert_eq!(store.get("absent"), None);
}

That test does not know or care whether returning None is the right product decision. Maybe users desperately want an error instead. The test only asks: does the code do what the spec said? That is verification’s entire remit — and its blind spot.

The tell is that every verification technique in that list takes the spec as a given and never questions it. That is a feature: it makes verification fast, automatable, and objective. It is also precisely why verification, run alone, will happily certify a product built on a mistaken premise — it was never in the business of checking the premise.

Validation — did we build the right thing?

Validation answers a harder question: does the software meet the user’s real need? Its reference point is not a document you can diff against; it is the world. And the world does not hand you an assertion you can put in assert_eq!. So validation leans on a different toolkit:

Demos and usability testing — watch a real user try to do a real task and see where they stall.
Acceptance testing — the customer, not the developer, confirms the feature solves their problem.
Beta releases, A/B tests, and telemetry — measure whether the feature actually gets used and actually helps.
User interviews and support tickets — the market telling you, after the fact, that the spec was wrong.

Notice that none of these compare code to spec. They compare the whole system — spec included — to the need. Validation is the only activity that can catch a wrong specification, because it is the only one that looks past the spec to the thing the spec was supposed to capture.

Validation evidence is also weaker and noisier than verification evidence, and it is worth being honest about why. A passing unit test is a crisp, repeatable boolean: the code did or did not match the rule. A usability session or an A/B test gives you a signal — this user struggled, that variant converted 3% better — that is entangled with sample size, context, and luck. You rarely get a clean pass/fail. That is not a flaw to be fixed; it is the nature of measuring against an open-ended reality. It means validation is a matter of accumulating confidence from many imperfect observations, not proving a single clean fact — which is exactly the “justified confidence” framing the rest of the book builds on.

  reality / user need
        ▲
        │  validation lives here (demo, acceptance, telemetry)
   ┌────┴────┐
   │  SPEC   │
   └────┬────┘
        │  verification lives here (tests, reviews, static analysis)
        ▼
     the code

Verification climbs from code up to the spec. Validation climbs from the spec up to reality. You need both rungs, or you fall.

Under the hood — why the boundary blurs in practice

The clean two-rung diagram is a teaching model, and reality is messier in one useful way: the “spec” is rarely a single frozen document. It is a stack of increasingly precise statements, and each level’s job is to verify against the level above it while validating that the level above was itself worth building.

  business goal   "reduce checkout abandonment"        ← validate against market
       │
  user story      "as a shopper, I can save my cart"   ← validate with users
       │
  acceptance      "cart persists 30 days across login" ← validate with customer
       │
  design/spec     "store cart_id in a 30-day cookie"   ← verify design vs acceptance
       │
  code            fn save_cart(...) { ... }            ← verify code vs design

Read top to bottom and each arrow is a verification step: does this level faithfully implement the one above it? Read the arrows as questions about whether the level above was correct, and each is a validation step. The same artifact — say, the acceptance criterion — is the spec for the code beneath it and a claim about reality to be validated from above. That is why the same activity can feel like verification to a developer and validation to a product owner: they are standing on different rungs looking in opposite directions.

The practical upshot: when you write a test, know which arrow it checks. A test that asserts “cart persists 30 days” verifies the code against the acceptance criterion. It does not validate that 30 days was the right number — only a user or the business can do that. Confusing the two is how teams end up with a beautifully tested cookie lifetime that nobody wanted.

The perfectly-built wrong product

The failure mode that makes this distinction worth a whole page: a system can pass every verification check and still fail validation. Every test green, every type checked, every review signed off — and the product is still wrong, because the spec it faithfully implements was the wrong spec.

NASA’s Mars Climate Orbiter was lost as it arrived at Mars. Widely reported investigations traced the root cause to a units mismatch: one piece of ground software produced a thrust-related value in imperial units (pound-force seconds) while the navigation software consuming it expected metric units (newton-seconds). Each module worked exactly as its own specification said — the producer correctly emitted imperial, the consumer correctly read metric. Every unit test either team could write against its own spec would pass. The failure lived in the gap between the two specs: no single component was wrong, so no amount of component-level verification could see it. Only validating the end-to-end system against reality — does the spacecraft actually end up where the physics says it should? — could have surfaced it. Perfect verification of each part, failed validation of the whole. (Details are simplified; the enduring lesson is that two individually spec-conformant components can still implement a mutually wrong spec.)

The reverse also happens, more quietly and far more often: a product nails the user’s need but violates its own spec in edge cases — a crash on a weird input, a rounding error, a race under load. That is a validation success with a verification failure. It ships, users love it, and then it falls over on the input nobody wrote a rule for.

Two kinds of bug, two kinds of fix

The reason this matters day to day: when a bug is reported, your first job is to classify which question it failed. The classification changes the fix.

  Bug report: "The invoice total is wrong."
        │
        ├─ Does the code match the spec?
        │      NO  ──► spec-conformance failure (verification bug)
        │              fix: change the CODE to obey the spec
        │
        └─ Does the code match the spec, but the
           spec is wrong?
               YES ──► wrong-spec failure (validation bug)
                       fix: change the SPEC (then the code)

A spec-conformance failure means the code disagrees with a correct spec. The fix is local and cheap: change the code, add a verification test that pins the correct behaviour, done.
A wrong-spec failure means the code faithfully implements a mistaken spec. Changing the code alone is a trap — you would be “fixing” it to match a spec that is itself wrong, or worse, hacking a special case that contradicts the written rule. The real fix is to correct the spec (and the shared understanding behind it), then re-verify the code against the corrected spec.

Miss this classification and you get the classic dysfunction: an engineer “fixes” a validation bug with a code patch, the tests stay green, the spec stays wrong, and the same misunderstanding resurfaces three tickets later in a new disguise.

A quick worked example. Suppose the invoice total on a $100 order with 8.25% tax reads $108.25 in the app, but the customer insists it should be $108.24.

If the spec says “round tax half-up to the nearest cent” and the code rounded correctly to $108.25, then the customer’s expectation disagrees with a spec everyone signed. Before touching code, you ask: is the spec right? If the jurisdiction genuinely requires round-half-down here, this is a wrong-spec failure — fix the spec, then the code follows.
If the spec says “round half-down” and the code produced $108.25 anyway, the code broke a correct rule. That is a spec-conformance failure — a one-line code fix plus a verification test that pins $108.24.

Same symptom, same number on screen, two completely different root causes and two completely different fixes. The only way to tell them apart is to hold the code and the spec up against each other, and then hold the spec up against reality.

Studies of large software projects across decades have consistently found that a large share of defects — often cited in the rough range of a third to a half — originate in requirements and specification, not in coding. These are validation defects: the code may be fine against the spec. A back-of-envelope illustration of why they hurt:

  Suppose 40% of defects are wrong-spec (validation) defects.
  Your unit-test suite has 95% line coverage and all tests pass.

  Coverage measures: does the code exercise the spec's paths?  → verification
  It says NOTHING about the other 40% of defects, because those
  live in the spec itself, which the tests are derived FROM.

  Green suite, 95% coverage, and up to 40% of your real defects
  are structurally invisible to it.

The point is not the exact percentage — it is that a perfect verification signal is silent about an entire class of failure. Treat any specific figure as illustrative, not a measured constant for your project.

Both are necessary — neither substitutes for the other

It is tempting to rank them. Don’t. They cover disjoint failure modes:

Verification without validation gives you a flawless implementation of the wrong idea — expensive, well-tested waste.
Validation without verification gives you a beloved feature that crashes on the edge case nobody validated — a good idea that falls apart in the details.

They also operate on different clocks. Verification is fast, repeatable, and mostly automatable, so it runs on every commit — a machine can rerun ten thousand assertions in seconds and give an unambiguous verdict. Validation is slower, needs humans and real usage, and runs at coarser milestones — a demo, a beta, a release — because you cannot summon a representative user or a month of real traffic on every push.

This clock difference has a management trap baked into it. Because verification is cheap and produces a satisfying green checkmark, it is easy to over-invest in it and quietly declare victory. Validation is expensive, ambiguous, and often delivers bad news, so it is the first thing dropped under deadline pressure. The teams that ship the wrong product are rarely the ones who couldn’t validate — they are the ones who mistook a full verification signal for permission to stop asking whether the thing was worth building. Shift Left applies to both: verify early with tests and reviews, and validate early with prototypes and demos, so you discover a wrong spec before you have built a mountain on top of it. This is exactly the engine behind the justified confidence this book keeps returning to: confidence that you built it right and that it was the right thing to build.

Where this sits in the rest of the book

This distinction quietly organizes almost everything that follows, so it is worth planting flags now:

The infinite input space is a verification problem — you cannot check the code against every point the spec covers. Validation has its own version: you cannot expose the product to every real user and situation either.
How much testing is enough must be answered twice, once per question. “Enough verification” (have we covered the risky spec paths?) and “enough validation” (are we confident this is the right thing?) are separate budgets with separate risk profiles.
The oracle problem — how do you know the right answer? — is really a question about where your oracle comes from. A verification oracle is the spec; a validation oracle is the user or the world. When you have no trustworthy oracle at all, you cannot cleanly do either.

Keep the two questions in your head as a lens you point at every later technique: does this help me build it right, help me build the right thing, or both? Most techniques are strong at one and silent on the other, and knowing which is half of using them well.

The architect’s lens

Why does this distinction exist? Because “is it correct?” secretly asks two questions with two different yardsticks — the written spec and the real need — and a single word hides the fact that they can disagree.
What problem does it solve? It stops teams from mistaking a green test suite for a successful product. It gives you a way to say which kind of correct you have evidence for, and which you still owe.
What are the trade-offs? Validation is expensive and slow — it needs real users, real usage, and human judgment — so teams under pressure quietly drop it and over-invest in cheap, automatable verification, optimizing the metric they can measure.
When should I lean hard on one? Early in a product’s life, or in a new domain, validation dominates: the biggest risk is building the wrong thing. In a mature, well-understood system with a trusted spec, verification dominates: the biggest risk is regressions against a spec you already trust.
What breaks if I remove it? Drop verification and you ship a product that is right in intent but broken in the details. Drop validation and you ship a product that is flawless in the details and wrong in intent — the perfectly-built wrong product, every test passing all the way down.

Check your understanding

State Boehm’s two questions and say which of verification and validation each one names.
Explain how a system can pass 100% of its verification tests and still be the wrong product. What is verification’s yardstick, and why can’t it catch that failure?
A colleague says “we have 95% code coverage, so quality is covered.” Using the verification/validation split, what class of defect is this claim silent about, and why?
A bug is filed: the app faithfully does what the requirements document says, but the requirements document contradicts the actual tax law. Is this a spec-conformance failure or a wrong-spec failure, and what is the correct fix?
Match the technique to the activity it primarily serves: unit tests, a usability demo with a real user, static type checking, an acceptance sign-off from the customer, a code review against the design.

Show answers

“Are we building the product right?” is verification (does the software conform to its specification?). “Are we building the right product?” is validation (does the software meet the user’s real need?).
Verification’s yardstick is the specification, not reality. If the spec itself is wrong, code that perfectly conforms to it will pass every verification test while doing the wrong thing. Verification can’t catch this because the broken thing (the spec) is exactly what it measures against — you need validation, which compares the whole system to the real need.
It is silent about wrong-spec / validation defects. Coverage measures how thoroughly the code exercises the spec’s paths — a verification signal. Tests are derived from the spec, so if the spec is wrong, high coverage tells you nothing about that entire class of defect.
A wrong-spec (validation) failure — the code matches the spec, but the spec is wrong. The fix is to correct the specification (and the shared understanding behind it) to match the law, then re-verify the code against the corrected spec. Patching only the code would either contradict the written spec or leave the wrong rule in place to resurface later.
Unit tests → verification. Usability demo with a real user → validation. Static type checking → verification. Acceptance sign-off from the customer → validation. Code review against the design → verification.