Acceptance Tests — Did We Build the Right Thing?

The system and end-to-end tests on the last page proved something powerful: the assembled product, driven like a real user through real I/O, does the job. Every branch fires, every service answers, the login flow works. And yet a product can pass all of that and still be a failure — because “does it work?” and “did we build the thing they needed?” are different questions, and the first four levels only ever ask the first one.

This page is about the level that asks the second one. It is a change in kind, not just in scope. Below acceptance, every test you write is graded against a spec you — the developer — inferred. Acceptance is graded against what the stakeholder actually wanted. That gap is where the most expensive bug in software lives, and it is the only bug no lower level can catch.

Verification versus validation

Two words that sound like synonyms and are not. Learn the split and half of this page is already yours.

• Verification — did we build it right? Does the software match the specification we were given (or wrote down)? This is what unit, integration, and system tests do. They compare the code to a spec.
• Validation — did we build the right thing? Does the software match the real need — the problem the user or business actually has? This compares the spec to reality.

        THE NEED                THE SPEC               THE CODE
   (what the user               (what the             (what got
    actually wants)         developer inferred)          built)
        │                          │                       │
        └──────── validation ──────┘                       │
                "right thing?"      └──── verification ─────┘
                                          "built it right?"

Notice where each arrow points. Verification lives entirely on the right — spec to code — and every level before this one is a verification level. It can be flawless and still leave the left arrow broken: a spec that never matched the need. Validation is the only activity that puts the left arrow under test, and acceptance testing is how you do it deliberately instead of discovering it in production.

A worked way to feel the difference: imagine the ticket said “sort results by date.” You build it, and every test is green — verification is perfect. But the user meant most recent first and you shipped oldest first. There is no unit test that fails, because the code does exactly what the spec said. Only a test written against the need — “as a user, I see my newest orders at the top” — catches it. That is validation.

Acceptance criteria: the executable definition of done

If validation is comparing the spec to the need, then somebody has to write the need down in a form you can check. That artifact is the acceptance criteria: a concrete, testable list of conditions that must hold for a feature to be considered done.

The critical property — and the thing that separates a real acceptance test from “an E2E test we happened to call acceptance” — is who owns it. Acceptance criteria are owned with the stakeholder, not inferred by the developer alone. The product owner, the customer, the domain expert: they define what “done” means, and the test encodes their definition. The developer’s job is to make the criteria pass, not to decide what they are.

The dominant way to make criteria executable is the Given-When-Then shape, drawn from behaviour-driven development (BDD):

Feature: Refund a paid order

  Scenario: A refund returns money and closes the order
    Given a customer has a paid order for $40
    When the support agent issues a full refund
    Then the customer is refunded $40
    And the order status becomes "refunded"
    And no further refund can be issued on that order

Read that aloud to a non-programmer and they can tell you whether it is right — that is the entire point. It is written in the language of the business, not the language of the code. Tools like Cucumber, SpecFlow, or Behave bind each Given/When/Then line to a step function so the plain-English scenario runs as a test. The prose is the spec; the binding is the plumbing. When the scenario is green, the definition of done — the stakeholder’s definition — is provably met.

This is what “executable specification” means: the document that says what to build and the test that proves it was built are the same artifact. It cannot drift out of date, because if it drifts, it fails.

Same tools as E2E, different audience

Here is a common confusion worth dissolving. An acceptance test and an end-to-end test often use the same machinery — both may drive the deployed product through a browser or an API, both may take seconds to run, both boot real dependencies. So what makes one “acceptance” and the other “E2E”?

The intent and the audience, not the tooling. An E2E test is written by and for engineers to answer “does the assembled system work through real I/O?” An acceptance test is written for the business to answer “is this the behaviour you asked for?” The same Playwright script can serve either role depending on who owns the assertion and what question it is evidence for.

             E2E TEST                    ACCEPTANCE TEST
 question    "does it work end          "did we build the
              to end?"                   thing you asked for?"
 owner       engineers                  stakeholder + engineers
 language    code / test DSL            business terms (Given-When-Then)
 evidence    "the system runs"          "the requirement is met"
 tooling     Playwright / API driver ── (often literally the same) ──►

The lesson: don’t classify a test by how it runs. Classify it by whose question it answers. If the assertion encodes a stakeholder’s definition of done, it is an acceptance test — even if it is one line away from an E2E test in the same file.

User acceptance testing: the human still in the loop

Not everything a stakeholder cares about can be pinned in a Then clause. “Does the new checkout feel trustworthy?” “Is this workflow how a real claims adjuster actually works?” “Would a first-time user find this?” These are validation questions with no mechanical oracle — no assertion can decide them, because the answer lives in a human’s judgement.

User acceptance testing (UAT) is the level where a real human — the customer, a domain expert, a representative user — exercises the system against their own need and signs off. It is often the final gate before release: the business confirming, by hand, “yes, this is the thing we asked for.” Automate everything you can, but UAT is where the human in the loop is irreplaceable, because the thing being validated is fitness for a real-world purpose that no one fully wrote down.

The trap is treating UAT as “one more round of QA bug-hunting.” It is not. QA verifies against the spec; UAT validates against the need. A UAT session that only finds crashes has been run wrong — those should have died three levels down. UAT exists to catch “this works, but it is not what I meant.”

What this level catches that the others structurally cannot

Every lower level shares one blind spot: they are all graded against a spec, and a spec can be wrong. Unit tests confirm the function matches its intended logic. Integration tests confirm two parts agree at their seam. System and E2E tests confirm the deployed product does what the tests expect. Every one of those can be green while the product solves the wrong problem — because the expectation itself came from a mis-read requirement.

Acceptance testing is the only level that puts the requirement under test. It catches:

• a correct implementation of the wrong requirement — the feature does exactly what the ticket said, and the ticket was wrong;
• a spec that drifted from the need between kickoff and delivery;
• behaviour that is technically compliant but useless in the real workflow it was meant to serve.

This is the most expensive class of bug in software, precisely because everything downstream passed. There is no red test, no stack trace, no failing assertion — just a shipped product that nobody wanted. The cost is not a hotfix; it is the entire build.

By the numbers — the cost of a wrong requirement

A requirement defect is cheapest to fix at the moment it is written and grows more expensive at every phase it survives. The classic (rough, much-cited, do not quote as law) shape of relative fix cost:

  found during        relative cost to fix
  requirements        1×
  design              ~3–6×
  coding              ~10×
  system test         ~15–40×
  in production       ~30–100×+

The lesson is not the exact multipliers — they vary wildly by study and context. It is the direction: a wrong requirement caught at acceptance/UAT, before release, is enormously cheaper than the same wrong requirement caught by an angry customer. That is the entire economic argument for owning acceptance criteria with the stakeholder up front, not inferring them and hoping.

It actually happened — the Ariane 5 Flight 501, June 1996

On its maiden flight in June 1996, the Ariane 5 rocket veered off course and self-destructed about 37 seconds after launch. The proximate cause was a software exception: a 64-bit floating-point value (horizontal velocity) was converted to a 16-bit integer, overflowed, and the guidance computer shut down.

The reused code was verified — it had flown correctly on Ariane 4. What failed was validation: the reused module’s specification assumed the flight profile of Ariane 4, and Ariane 5 flew faster. The code was built right; it was built for the wrong requirement. No amount of lower-level testing against the old spec could have caught a mismatch between the spec and the new rocket’s reality. (Specifics simplified; the lesson is the point.) Acceptance-level thinking — “is this the right behaviour for this system’s actual need?” — is exactly the gap that opened.

The architect’s lens

• Why does it exist? Because every test graded against a spec inherits that spec’s errors. Acceptance testing exists to put the requirement itself under test — to validate that the spec matched the need, not just that the code matched the spec.
• What problem does it solve? The single most expensive failure in software: a correct implementation of the wrong thing, where every lower level passes and the product is still useless. It closes the gap between “did we build it right?” and “did we build the right thing?”
• What are the trade-offs? Acceptance tests are slow, few, and expensive to keep, and they demand stakeholder time to define and (for UAT) to run. In exchange they buy a currency no other level offers — meaning, not just correctness. They are worth the most and cost the most per test.
• When should I avoid it? Never skip it entirely, but don’t push down into it what belongs higher: bugs, crashes, and contract mismatches should be caught by unit, integration, and E2E long before acceptance. Acceptance is not a bug net; it is a fitness check. And for internal tooling with no external stakeholder, a heavy UAT ceremony may be overkill.
• What breaks if I remove it? Your test suite can go fully green while you ship the wrong product. You lose the only signal that the requirement — not the code — was correct, and you discover it from the customer instead of from a test.

Check your understanding

1. State the difference between verification and validation in one sentence each, and say which arrow — spec-to-code or need-to-spec — each one puts under test.
2. Why are acceptance criteria described as an executable definition of done, and why does it matter that the stakeholder — not the developer alone — owns them?
3. Two tests use the identical Playwright script and run in the same time. What makes one an E2E test and the other an acceptance test?
4. UAT is often the last gate before release. What class of problem is it uniquely positioned to catch, and why can’t an automated assertion catch it?
5. A product passes every unit, integration, and E2E test and is still a failure. Explain how that is possible using the verification/validation distinction.

Show answers

1. Verification — did we build it right? — checks the code against the spec; it puts the spec-to-code arrow under test. Validation — did we build the right thing? — checks the spec against the real need; it puts the need-to-spec arrow under test. The first four levels are all verification; acceptance is validation.
2. Executable because the specification (what to build) and the test that proves it (what was built) are the same artifact — a Given-When-Then scenario that both describes and runs — so it cannot silently drift out of date; if it drifts, it fails. Stakeholder ownership matters because the whole point is validating against the need: if the developer defines “done” alone, the test just re-encodes the same inferred spec that might be wrong, and validates nothing.
3. Not the tooling — the intent and audience. It is an acceptance test if its assertion encodes a stakeholder’s definition of done and answers “is this the behaviour you asked for?”; it is an E2E test if it is engineers answering “does the assembled system work end to end?” The same script can be either depending on whose question it is evidence for.
4. It catches “this works, but it is not what I meant” — fitness for a real-world purpose that no one fully wrote down (does the workflow match how the user really works? does it feel right?). No automated assertion can catch it because there is no mechanical oracle: the answer lives in a human’s judgement of a need that was never fully specified.
5. Those three levels are all verification — they grade the code against a spec, and they can all pass while the spec itself is wrong. If the requirement was mis-read, the product can be a correct implementation of the wrong thing: green everywhere, and still not what the customer needed. Only validation (acceptance/UAT) puts the requirement under test and catches it.