Overview — The Testing Landscape

The previous part sliced testing by scope — how much of the system a test puts under test, from one function to the whole product. That axis answered at what level you check. This part answers a different question that runs at a right angle to it: what kind of confidence does a change need at all?

Those are genuinely separate axes, and confusing them is a common mistake. “Unit test” is a level (one function, isolated). “Performance test” is a type (does it stay fast enough?). You can run a performance test at the unit level (micro-benchmark one function) or at the system level (load-test the whole service). You can run a functional test at any level too. Level tells you how wide you look; type tells you what you are looking for. This overview maps the “what you are looking for” axis so the rest of the part has a place to slot in.

The question this part answers

A test costs time to write, time to run, and time to maintain. You cannot run every kind of test on every change — nobody has that budget. So before you write a single assertion you have to answer: what could go wrong with this change, and which of those failures do I most need to rule out? That is a question about risk, and every type of testing is a tool for buying down one specific class of it.

Here is the mental model to carry through the whole part:

  a change ships some risk ──► pick the test type that buys down THAT risk

  "it computes the wrong answer"        ─► functional testing
  "my fix broke something else"         ─► regression / smoke / sanity
  "it's correct but too slow"           ─► performance testing
  "correct, but an attacker gets in"    ─► security testing
  "correct, but real users can't use it"─► accessibility / usability / compat
  "correct on the paths I imagined"     ─► exploratory testing (the ones I didn't)

You spend testing effort where the risk is highest, not evenly. A change to a payment calculation carries functional and regression risk; a change to a login form carries security and accessibility risk; a change to a database query under a hot endpoint carries performance risk. The skill this part builds is reading a change and naming its risks — then reaching for exactly the test types those risks warrant, and no more.

The primary split: functional vs non-functional

Every type of testing falls on one side of a single line.

Functional testing asks: does the software do the right thing? Given an input, does it produce the correct output? Does the button do what the button says? Does the API return the right JSON for the right request, and the right error for a bad one? Functional testing is about behaviour against a specification — a pure question of correctness, blind to how fast, how secure, or how pleasant the software is.

Non-functional testing asks: does the software do it well enough*?* The behaviour can be perfectly correct and the software can still fail its users — because it is too slow, falls over under load, leaks data to an attacker, is unusable with a screen reader, or breaks on last year’s browser. Non-functional testing is about the qualities of correct behaviour: speed, resilience, security, usability, accessibility, compatibility. These are often called the “-ilities.”

                         does it work?
                              │
            ┌─────────────────┴──────────────────┐
     FUNCTIONAL                              NON-FUNCTIONAL
  "does it do the right thing?"        "does it do it well enough?"
            │                                     │
   correct output for input          fast, resilient, secure, usable,
   correct behaviour vs spec         accessible, compatible ("-ilities")
            │                                     │
   a wrong answer                     a right answer delivered badly

The line matters because the two sides fail differently and are found differently. A functional bug shows up as a wrong result — a clear pass/fail you can assert on. A non-functional bug often passes every functional test: the answer is right, it just arrives 8 seconds late, or leaks a session token, or renders as invisible-grey-on-white for a colour-blind user. You cannot catch a non-functional failure with a functional test, because the functional test isn’t asking the question. This is the type-level echo of the levels-part lesson: each kind of test catches a class of failure the others structurally cannot.

Every type buys down a class of risk

The unifying idea of this part: a type of testing is a lens ground for one class of risk. You don’t run “all the types” any more than a doctor orders every possible scan — you run the ones that address the risks actually present. That reframes the whole catalogue below from “a list of things to learn” into “a menu you shop from, guided by the change in front of you.”

Two changes make this concrete:

• A typo fix in a help-text string carries almost no risk of any kind. Reading it and shipping it is a legitimate testing decision — the risk-adjusted cost of a performance test on a string constant is absurd.
• A change to how passwords are hashed carries functional risk (does login still work?), regression risk (did I break existing sessions?), security risk (is the new hash actually stronger?), and performance risk (is the new hash so slow it DoSes the login endpoint?). One small diff, four lenses warranted.

Same tool-belt, wildly different selections — because the risk profiles differ. Learning to read that profile is the closing skill of this part.

By the numbers — where the risk actually is

A useful gut-check before you pick types: what does a failure of each kind cost, and how likely is it on this change? Risk is roughly likelihood × cost. Rough industry-shaped figures (illustrative, not laws — do not quote as fact):

  type of failure        typical cost of one escaping to prod
  functional (wrong)     a bad result; refund, correction, lost trust
  regression             a working feature silently breaks; often worse — nobody expected it
  performance            checkout slows 1s → measurable drop in completed orders
  security               a breach; regulatory + reputational cost dwarfs the others
  accessibility          excluded users + legal exposure under ADA/EN 301 549-style law

  worked pick: change = "add a coupon field to checkout"
    functional  ─ high likelihood of a wrong-total bug        ► YES, write these
    regression  ─ touches the order-total path many use        ► YES, run the suite
    performance ─ one extra field, hot path                    ► maybe, if it hits the DB
    security    ─ user-supplied input into a total             ► YES, validate/injection
    accessibility ─ a new form field real users must operate   ► YES, label + keyboard

The point isn’t the numbers — it’s the habit of scoring each risk before spending on a test.

Roadmap for this part

Read these in order. Each page takes one type (or the decision that ties them together), teaches it from first principles, names the risk it buys down, and gives a one-line “reach for this when…” cue so you can pattern-match it against a real change.

#	Page	Risk it buys down	Reach for this when…
2	Functional Testing — Does It Do the Right Thing?	Wrong behaviour: incorrect output for a given input	…a change alters what the software computes, returns, or decides
3	Regression, Smoke & Sanity — Guarding What Already Works	A change silently breaks something that used to work	…you’re fixing, refactoring, or adding near existing behaviour
4	Performance Testing — Load, Stress, Spike & Soak	Correct but too slow, or collapses under load	…a change hits a hot path, adds I/O, or must survive traffic peaks
5	Security Testing Basics — Thinking Like an Attacker	Correct but exploitable: data or access an attacker can take	…a change touches input, auth, secrets, or trust boundaries
6	Accessibility, Usability & Compatibility — Testing for Real Users	Correct but unusable — for some users, devices, or browsers	…a change touches the UI, a workflow, or the range of supported clients
7	Exploratory Testing — Finding the Bugs Scripts Miss	Unknown unknowns: failures on paths no script imagined	…the change is novel, risky, or under-specified, and scripts can’t cover it
8	Choosing What a Change Needs — A Decision Guide	Wasted or missing effort: testing the wrong things for this change	…every change — this is the skill of picking from the menu above
—	Revision — Types of Testing	—	A prose recap of the whole part

The thread

Carry one idea through every page: a type of testing is not a box to tick — it is a lens ground for one class of risk, and you reach for the lens the change in front of you actually needs. The first six pages each hand you one lens and name the risk it clarifies. They are not ranked; there is no “best” type, just as there was no best level. A functional test and a security test are not competitors — they answer different questions, and a change can need both, one, or neither. Page 8 is where the whole part pays off: it turns “here are the lenses” into a decision — you read a diff (a bug fix, a new feature, a config bump), name its risks, and choose which types it warrants. Every page before it is teaching you to see one more kind of risk clearly, so that by page 8 you can look at any change and know what confidence it needs.

With the map in hand, start with the type every change touches first — the one that asks whether the software does the right thing at all: Functional Testing — Does It Do the Right Thing?.

Check your understanding

1. This part and the previous part (Levels) slice testing on two different axes. What does each axis answer, and why are they independent — i.e. why can one “type” of test run at several “levels”?
2. State the primary split of this part in one sentence, and give the defining question each side asks.
3. Why can a change pass every functional test and still be a serious failure in production? Give a concrete example of a non-functional failure that a functional test cannot catch.
4. The mental model says you spend testing effort “where the risk is highest, not evenly.” What two factors combine into the “risk” you’re scoring, and how do they tell you where to spend?
5. Preview the closing skill of the part: given a change like “add a coupon field to checkout,” what is the process for deciding which test types it warrants?

Show answers

1. Levels answer at what scope you test — how much of the system runs, from one function to the whole product. Types answer what kind of confidence the change needs — correctness, speed, security, usability, and so on. They’re independent because scope and question are orthogonal: a performance test (a type) can run as a micro-benchmark (unit level) or a full load test (system level); a functional test can run at any level too. Level = how wide you look; type = what you look for.
2. Functional vs non-functional. Functional asks does it do the right thing? (correct output/behaviour for a given input, against the spec). Non-functional asks does it do it well enough? (the qualities of correct behaviour — fast, resilient, secure, usable, accessible, compatible: the “-ilities”).
3. Because functional tests only ask “is the answer correct?” — they’re blind to the qualities of that correct answer. Example: a checkout that returns the exact right total but takes 8 seconds, or leaks a session token, or renders as invisible grey-on-white for a colour-blind user. Every functional assertion passes; the software still fails its users. The functional test can’t catch it because it isn’t asking that question.
4. Risk is roughly likelihood × cost — how likely a failure of that kind is on this change, times what it costs if it escapes to production. You score each class of risk and spend on the test types where that product is highest, rather than running every type on every change.
5. Read the change and name its risks, then map each risk to the type that buys it down: a coupon field alters the order-total computation (functional risk → functional tests), touches a path many features share (regression risk → run the suite), takes user-supplied input into a money calculation (security risk → validation/injection tests), and adds a form field real users must operate (accessibility risk → labels + keyboard). Score each, then reach for exactly the lenses those risks warrant — the subject of page 8, Choosing What a Change Needs.