Quality, Testing, and QA — Three Different Things

The previous page, Verification vs Validation, split the innocent question is it correct? into two questions with two yardsticks. This page performs a similar cut on three words that get used interchangeably in almost every team meeting: quality, testing, and QA. They sound like synonyms. They are not. They are three different things — a property, an activity, and a discipline — and conflating them produces some of the most predictable, most expensive dysfunctions in software.

Here is the confusion in its purest form. A manager says, “we need better quality, so let’s hire more QA to do more testing.” That single sentence collapses all three ideas into one, and every word of it can be true while the plan still fails — because more testing at the end cannot create quality that the process never built in. To see why, we have to pull the three words apart and look at each one on its own.

Three words, three categories

Start by noticing what kind of thing each word names. They do not even belong to the same category.

  QUALITY   is a PROPERTY   — of the product.   "How fit for purpose is it?"
  TESTING   is an ACTIVITY  — that measures.    "What is actually true right now?"
  QA        is a DISCIPLINE — that builds in.   "How do we make quality happen?"

A property is a state of the thing. An activity is something you do to learn about the thing. A discipline is a system of practices that causes the thing to come out well. You cannot substitute one for another any more than you can substitute a thermometer for a fever, or a fever for medicine. Keep this table in your head; every dysfunction below is a category error where one of these three got mistaken for another.

Quality is a property of the product

Quality is fitness for purpose — how well the product serves the real need of its real users and stakeholders. It is not a number you compute; it is a judgment made against a purpose, and different stakeholders judge it differently. A trader wants microsecond latency; an accountant wants provable correctness; a first-time user wants to not feel stupid. The “same” product can be high quality for one and low for another, because quality only means anything relative to whose purpose, for what job.

This is why quality cannot be defined purely inside the code. You can measure cyclomatic complexity, test coverage, and defect counts all day — those are internal signals — but none of them tells you whether the product is fit for purpose. Fitness lives at the boundary between the product and the world, and it is ultimately judged there: by users doing real work, by the business hitting its goals, by the operator who has to run it at 3 a.m.

        the world / real purpose
                 ▲
                 │  quality is judged HERE
        ┌────────┴────────┐
        │   the product   │
        └────────┬────────┘
                 │  internal signals (coverage, complexity)
                 ▼         live down here — necessary, not sufficient
              the code

Because quality is a property, you can only observe it, never inject it late. That single fact is the load-bearing idea of this whole page, and it is what the next two sections turn into a warning.

Testing is an information-gathering activity

Testing is the activity of gathering information about quality. You run the product against inputs, compare what it does to what it should do, and report the gap. That is the whole job: testing measures the property called quality. It is a thermometer, not a furnace.

The consequence people skip past: testing does not create quality — it reveals it. A test can tell you the product is broken; it cannot make the product not broken. When a thermometer reads 39°C, breaking the thermometer does not cure the patient, and no amount of extra thermometers lowers the temperature. Testing gives you information about the fitness of the product. What you do with that information — fix the code, change the spec, ship anyway — is a separate act.

Recall the book’s throughline: testing buys justified confidence. Confidence is knowledge about the product’s quality, not the quality itself. This is exactly why the infinite input space matters here: testing samples reality to estimate a property it cannot fully see, the same way a poll samples voters to estimate an opinion it cannot poll in full.

Here is testing doing its one job — measuring, and reporting the gap:

#[test]
fn withdraw_more_than_balance_is_rejected() {
    let mut acct = Account::with_balance(100);
    // This test does not make the code correct.
    // It reports whether the code already IS correct.
    let result = acct.withdraw(150);
    assert_eq!(result, Err(Error::InsufficientFunds));
    assert_eq!(acct.balance(), 100); // balance untouched on failure
}

If that assertion fails, the test has done its full job: it has told you the product is unfit on this input. It has not, and cannot, fix anything. The fix is a different activity, done by a different act, upstream of the measurement.

QA is the discipline that builds quality in

If testing only measures, where does quality actually come from? From everything upstream of the test: clear requirements, good design, code review, pairing, static analysis, a definition of done, sensible architecture, a culture that says “we don’t ship on red.” Quality Assurance is the whole process discipline that builds quality in — the assurance is in the process, not in a final inspection.

Note the word: assurance. It is a claim about the process being trustworthy, made before the fact, not a verdict rendered after. QA is preventive; testing is detective. QA asks “how do we make defects unlikely to be created?”; testing asks “were any defects created?” A team can test heavily and have no QA (they measure a bad process precisely), or have strong QA and light testing (a good process needs less measurement to trust).

   QA (build quality in) ─── upstream ───►  TESTING (measure it) ─── informs ──►  DECISION
   requirements · design                    run · compare · report            ship / fix / rethink
   review · standards · DoD
        ▲                                         │
        └──────────── feedback closes the loop ───┘
              (what testing reveals feeds back into the process)

Testing is a part of QA, not a synonym for it. This is why the job title “QA” causing everyone else to stop caring about quality is a category error with a bad smell: it renames a whole-team discipline into a single downstream role. Which brings us to the mistake that ties the page together.

You cannot test quality in

Put the three ideas in a line and a hard truth falls out. Quality is a property built by the process (QA) and merely measured by testing. Therefore:

A testing team downstream of a bad process does not create quality. It reports the damage.

This is the single most important sentence on the page. If the requirements were vague, the design was rushed, and no one reviewed the code, then by the time work reaches a testing gate the quality is already determined. The testers can measure it precisely — and measuring a disaster precisely does not make it less of a disaster. You cannot “test quality in” at the end any more than you can weigh yourself thinner. This is the shift-left argument wearing organizational clothes: quality is cheapest to build in early and impossible to add late.

The classic failure mode is the quality gate as a place to throw work over. Developers “finish,” lob it over a wall to a QA team, and treat quality as someone else’s job that happens after theirs. The gate then does one of two ugly things: it becomes a bottleneck that finds a mountain of defects too late and too expensively to fix, or it gets bypassed under deadline pressure because “QA is slow.” Either way the property was decided upstream; the gate only chose when you found out.

   THROW-IT-OVER-THE-WALL (anti-pattern)

   dev ──finish──►  │ WALL │  ──► "QA" measures ──► finds the damage TOO LATE
   (quality already fixed here)          (can only report, not repair, cheaply)

   BUILD-IT-IN (the fix)

   requirements ─► design ─► code+review ─► test ─► ship
        every step OWNS quality; testing is one measurement among many

Decades of studies (Boehm, IBM, NIST) put the cost of fixing a defect found in production at roughly 10–100× the cost of fixing the same defect at design or requirements time. A downstream testing gate is where you discover the expensive ones — not where they became expensive.

  Relative cost to fix one defect (illustrative, order-of-magnitude):

    requirements   1x
    design         ~5x
    coding         ~10x
    testing gate   ~15x   ← where "QA at the end" finds it
    production     ~30–100x

  Adding more testers at the gate finds defects faster.
  It does NOT move them left. Only the PROCESS (QA) does that.

Treat the exact multipliers as illustrative, not measured constants — the enduring point is that testing changes when you learn, while QA changes what gets created.

When the U.S. federal health-insurance exchange launched on 1 October 2013, it failed almost immediately under load, and widely reported post-mortems described a project where end-to-end integration testing happened only in the final days, requirements shifted late, and dozens of contractors delivered pieces that were never exercised together until the end. No amount of last-week testing could have created the quality that the fragmented, rushed process never built in — the late testing mostly revealed, at the worst possible moment, damage that was already baked in. The enduring lesson is not “they tested too little”; it is that testing at the end can only measure a property the process already determined. (Details are simplified; the point is the category error, not any one contractor.)

Everyone owns quality

The healthy stance is the exact inverse of the wall. Quality is everyone’s job; testing is a role and a skill, not a gate you throw work over. A developer choosing a clear name, a reviewer catching a missing edge case, a product owner sharpening a vague requirement, an operator adding an alert — each is building quality in. The person with the title “tester” contributes a specialized, valuable skill: designing incisive experiments that gather the most information about quality per unit of effort. That skill is real and hard. But it is a contribution to a shared property, not sole custody of it.

This reframes the manager’s sentence from the top. “We need better quality, so let’s do more testing at the end” is a category error: it tries to raise a property by doing more of the activity that only measures it, while ignoring the discipline that actually produces it. The corrected sentence is: “We need better quality, so let’s build it in earlier — clearer requirements, tighter review, faster feedback — and use testing to tell us how well we’re doing.”

The architect’s lens

Why does this distinction exist? Because three different kinds of thing — a property (quality), an activity (testing), and a discipline (QA) — got collapsed into interchangeable words, and the collapse hides where quality actually comes from.
What problem does it solve? It kills the fatal plan “add quality by testing more at the end.” Separating the three tells you to invest upstream in process, and to use testing as a measurement that informs decisions rather than a machine that manufactures quality.
What are the trade-offs? “Everyone owns quality” can decay into “no one owns quality” without a discipline (QA) that makes ownership concrete, and the specialist testing skill can get undervalued once you stop treating testers as the sole quality gate.
When should I be careful with it? Don’t swing so far that you disband the testing skill entirely — “shift left” does not mean “stop measuring.” You still need incisive, information-rich testing; you just stop pretending it creates the property it measures.
What breaks if I ignore it? You get the throw-it-over-the-wall organization: quality decided upstream by an unowned process, a downstream gate that only reports damage too late to fix cheaply, and a manager forever hiring more testers to fix a problem that lives in the process.

Check your understanding

Name the category each of the three words belongs to — quality, testing, QA — and give the one-line role of each.
Explain the thermometer analogy: why does adding more testing at the end not raise quality?
What is the difference between QA being preventive and testing being detective? Where does each sit relative to the moment a defect is created?
A team ships buggy software and the plan is “hire more testers.” Using the three-way distinction, state precisely why this plan misdiagnoses the problem, and what the corrected plan is.
“Everyone owns quality” risks becoming “no one owns quality.” What keeps that from happening, and what specialized contribution does a person with the tester role still make?

Show answers

Quality is a property of the product — how fit for purpose it is, judged by real users and stakeholders. Testing is an activity — gathering information about quality by running the product and comparing behavior to expectation. QA is a discipline — the whole process that builds quality in, upstream of any single test.
Testing measures quality; it does not create it. A thermometer reads the temperature but does not change it, and adding thermometers does not cool the patient. By the time work reaches a final testing gate, the quality is already determined by the process that produced it — testing can only report it, not repair it cheaply.
QA is preventive: it acts before a defect is created, making defects unlikely (clear requirements, review, design, standards). Testing is detective: it acts after, finding defects that were already created. QA sits upstream of defect creation; testing sits downstream, measuring the result.
It mistakes a property (quality) for something you can raise by doing more of the activity (testing) that only measures it, while ignoring the discipline (QA) that actually produces it. A downstream testing team can only report the damage a bad process already caused. The corrected plan: build quality in earlier — clearer requirements, tighter review, faster feedback — and use testing to tell you how well the process is doing.
A discipline (QA) makes ownership concrete — shared responsibility with defined practices (definition of done, review, standards) rather than a vague “everyone cares.” The tester still contributes a specialized, hard skill: designing incisive experiments that gather the most information about quality per unit of effort. It is a contribution to a shared property, not sole custody of it.