Exploratory Testing — Systematic Paranoia
Every earlier page in this part handed you a checklist. The adversarial mindset taught you to assume the code is guilty. Happy path vs sad path, zero, one, many, empty, null, and boundaries, extremes and overflow, and ordering and duplicates each gave you a category of input to attack. Those categories are gold. But they share a blind spot: you can only put on a checklist the bugs you already thought of.
This page is about the other half of the job — hunting the bugs no checklist predicted. That skill is called exploratory testing, and it is the most misunderstood technique in the craft. Done badly, it looks like a person clicking around at random. Done well, it is the opposite of random: it is systematic paranoia, a disciplined loop of learning, designing, and executing tests all at once, with notes, heuristics, and a running list of assumptions you are actively trying to break.
What exploratory testing actually is
Section titled “What exploratory testing actually is”The classic definition, from the testers who named it, is worth memorizing:
Exploratory testing is simultaneous learning, test design, and test execution.
Scripted testing separates those three in time: someone writes the test cases first, then someone (or a machine) runs them later. That separation is a strength when you already know what to check — it is repeatable and automatable. But it is a weakness when you don’t know yet, because a script can only ask the questions its author already had.
Exploratory testing collapses the three into one tight loop:
┌──────────────────────────────────────────┐ │ │ ▼ │ observe the system ──► form a question/hunch │ ▲ │ │ │ ▼ │ read the result ◄── design & run a quick test ──┘ │ └──► every surprise feeds the next questionYou do a thing, you watch what happens, the result teaches you something, and that something tells you the next thing to try. The knowledge you gain in second three changes the test you design in second four. A script cannot do that; a curious human can.
The crucial word is guided, not random. Random clicking generates motion, not information. Exploratory testing is steered — by assumptions you want to falsify and by heuristics that tell you where to look next. The rest of this page is those two steering mechanisms.
Assumptions as a map: list your beliefs, then attack them
Section titled “Assumptions as a map: list your beliefs, then attack them”The single most productive exploratory habit is embarrassingly simple: write down what you believe is true about the system, then go try to prove each belief wrong.
Every system rests on a pile of unstated assumptions — the developer’s, the spec’s, and your own. Each one is a place where reality might disagree with the code. So make them stated. Before you touch a login form, jot the beliefs it implies:
Belief (what I assume is true) Attack (how I'd falsify it) ──────────────────────────────── ────────────────────────────── "email must be unique" register the same email twice "password is hashed, never stored raw" register, then read the DB row "session expires after 30 min" log in, wait, act at minute 31 "lockout after 5 bad attempts" try 6 — then a 7th; does it reset? "you can't log in to a deleted account" delete mid-session, then act "the form validates before submitting" disable JS, POST directlyNotice what this does. It converts a vague feeling (“I should test login”) into a list of concrete, falsifiable experiments. Each row is a hypothesis and its refutation attempt — the same move a scientist makes. A belief you cannot falsify is confirmed for now; a belief that shatters is a bug you would never have found by clicking around.
This is where the earlier checklist pages plug in. Each assumption is a candidate for the categories you already know: “email must be unique” is a duplicates question; “password field” is an empty/null/boundary and extremes question. The assumption list tells you where to point the paranoia; the checklists tell you what to throw once you are pointed.
Heuristics: how to explore systematically, not aimlessly
Section titled “Heuristics: how to explore systematically, not aimlessly”An assumption list gets you started, but you will not think of every assumption cold. Heuristics are the fix: reusable prompts that jog you toward areas you would otherwise skip. Three are worth carrying everywhere.
Tours — walk the product with one lens at a time
Section titled “Tours — walk the product with one lens at a time”A tour is a themed pass through the whole system, wearing a single pair of glasses. Instead of testing everything at once (and therefore nothing well), you make several focused passes:
Tour The one question you ask on this pass ──── ───────────────────────────────────── Money tour every place value is created, moved, or destroyed — correct? Data tour every field: what's the longest/weirdest value it accepts? Interruption start an action, then kill it — refresh, back, close, drop net Configuration toggle every setting; do combinations conflict? Landmark tour hit the "famous" features first, the way a tourist wouldOne lens at a time keeps you thorough. The interruption tour alone finds a startling number of bugs, because almost no developer tests what happens when a user closes the tab mid-transaction.
CRUD — the lifecycle every entity has
Section titled “CRUD — the lifecycle every entity has”For any entity the system stores, walk its full lifecycle: Create, Read, Update, Delete. The bugs cluster at the seams between these, not inside them:
- Create then Create — duplicate handling (the ordering-and-duplicates attack).
- Read a Deleted — does a stale link, cache, or second tab still show it?
- Update a Deleted — save an edit to a record someone else just removed.
- Delete then Read — is it gone, or merely hidden while the row lingers?
CRUD is a heuristic because it is exhaustive by construction: you cannot forget the “update after delete” case if you always walk all four operations against all four prior states.
SFDIPOT — the “San Francisco Depot” checklist
Section titled “SFDIPOT — the “San Francisco Depot” checklist”The broadest exploration heuristic, from James Bach’s Heuristic Test Strategy Model, is remembered by the mnemonic SFDIPOT (“San Francisco Depot”). It is a prompt list for dimensions of the product to probe:
S — Structure what it's made of: files, code, hardware F — Function what it does: features, calculations, error handling D — Data what it processes: input, output, big/small, defaults I — Interfaces how it connects: APIs, UI, other systems P — Platform what it depends on: OS, browser, network, time zone O — Operations how it's used: real users, environments, scenarios T — Time when things happen: order, concurrency, timeouts, datesYou do not test all seven every time. You use the list as a checklist against forgetting: run your eye down it and ask “have I explored the Platform dimension of this feature? the Time dimension?” The T alone reconnects you to the whole ordering and duplicates page — proof that these heuristics and your input checklists are the same paranoia viewed from different angles.
From hunch to failing test: the payoff move
Section titled “From hunch to failing test: the payoff move”Finding a bug by exploring is only half a win. A bug you found once by hand, with a vague memory of “I did something with a long name and it broke,” will be lost by next week. The defining discipline of a good exploratory tester is turning every discovered bug into a minimal, reproducible, automated regression test before moving on.
The pipeline has three stages, and each throws away something:
1. Reproduce Can I make it fail again, on purpose, from a clean state? If not, it isn't a bug report yet — it's a rumor.
2. Minimize Strip every step that isn't required for the failure. Remove inputs one at a time; keep only what's load-bearing.
3. Automate Encode the minimal repro as a test that runs in CI forever, so this exact bug can never silently return.Reproduce turns a hunch into a fact. If you cannot get it to fail again, you have not understood it — and an intermittent failure is often a concurrency or ordering bug wearing a disguise, which is a clue, not a dead end.
Minimize is where most of the value hides. The bug you found might have taken twelve steps; usually two of them matter. Delete one input, rerun; if it still fails, that input was noise. What remains after minimizing is the diagnosis — the smallest thing that triggers the failure points straight at the cause.
Automate is what makes it permanent. A found-and-fixed bug that has no test is a bug that will come back. The automated regression test is a promise to your future self: this specific failure is now guarded forever.
Here is the move in miniature. Suppose that while exploring kvlite, you notice that inserting the same key twice does something you did not expect. You do not file “insert seems weird.” You minimize it to two calls and pin the exact contract as an assertion:
// Minimal repro promoted to a permanent regression test.// The exploratory hunch was "does the second insert win?"// The test pins the answer so it can never silently change.let store = Store::new();store.set("hits".to_string(), 0).unwrap();store.set("hits".to_string(), 5).unwrap(); // same key, twice
// The bug we found: we ASSUMED the first write survived.// It doesn't — last write wins. Pin the real behavior:assert_eq!(store.get(&"hits".to_string()).unwrap(), Some(5));Two lines of setup, one assertion. Every incidental step from the original discovery is gone. Whatever the correct behavior turns out to be, the test now encodes it, and the next person who changes set will be told instantly if they break it. That is a hunch converted into justified, permanent confidence — the whole thesis of this book in six lines.
Paranoia with a method: balancing the two styles
Section titled “Paranoia with a method: balancing the two styles”So which wins — the categorized checklist of the earlier pages, or open exploration? Neither. They are two legs of the same walk, and a tester who drops either falls over.
Checklist / scripted Exploration ──────────────────── ─────────── finds the bugs you predicted finds the bugs you didn't repeatable, automatable adaptive, one-shot by nature cheap to re-run in CI expensive human attention can't discover new categories discovers new categories to add proves known things still work questions whether you know enough
▼ the loop that unites them ▼
explore → discover a NEW failure category → write the minimal regression test for it → that test JOINS the checklist, forever → checklist frees your attention to explore furtherThe relationship is a flywheel. Exploration discovers categories; you distil each into an automated check; the check joins the permanent suite; and because the machine now guards the known cases, your scarce human attention is freed to go find the next unknown one. The checklist is your memory of every trap you have already met. Exploration is how you meet new ones. Systematic paranoia is holding both at once — never trusting the code, but never poking at it aimlessly either.
The architect’s lens
Section titled “The architect’s lens”- Why does it exist? Because scripted tests can only check what their author already imagined, and the bugs that hurt most are the ones nobody imagined. Exploratory testing exists to discover the unknown unknowns that no pre-written case would ever ask about.
- What problem does it solve? It closes the blind spot of the checklist: it finds new categories of failure — surprising states, hidden assumptions, weird interactions — and feeds them back so they can become permanent automated checks.
- What are the trade-offs? It depends on skilled human attention, so it is expensive and not directly repeatable; a session is only as good as the tester running it, and its output is notes rather than a green/red number. It complements automation, it does not replace it.
- When should I avoid it? When the behavior is already well understood and stable — there, a cheap automated regression check gives more confidence per dollar. Reserve exploration for the new, the changed, the risky, and the code you don’t yet trust.
- What breaks if I remove it? Your test suite ossifies: it perfectly guards the bugs you already know while remaining permanently blind to every new one. Confidence becomes an illusion — high coverage of a shrinking, outdated map of reality, with the live bugs all in the unmapped territory.
Check your understanding
Section titled “Check your understanding”- Exploratory testing is defined as three activities happening at once. Name them, and explain why collapsing them into one loop finds bugs a script cannot.
- Describe the “assumptions as a map” technique in two steps, and explain how it connects to the categorized checklists from earlier pages.
- Pick one heuristic — tours, CRUD, or SFDIPOT — and explain how it makes exploration systematic rather than aimless.
- You find a bug while exploring. Walk through the three stages that turn it into a permanent regression test, and say what each stage discards.
- Exploration and scripted checklists are described as a flywheel, not rivals. Explain the loop that connects them and why removing either one is harmful.
Show answers
- Simultaneous learning, test design, and test execution. Collapsing them into one loop means each result immediately reshapes the next test you design — the knowledge you gain from running one experiment changes what you try next. A script fixes its questions in advance, so it can only ask what its author already thought of; the loop lets a tester ask questions that only became visible after the previous step surprised them.
- Step one: write down every belief you hold about the system (“email must be unique,” “session expires in 30 minutes”). Step two: for each belief, design and run an experiment to falsify it. It connects to the checklists because each assumption points you to a category — a uniqueness belief is a duplicates test, a field-length belief is an extremes/boundary test — so the assumption list tells you where to aim and the checklists tell you what to throw.
- Example (tours): a tour is a themed pass through the whole product wearing one lens at a time — a money tour, a data tour, an interruption tour. Testing one dimension at a time keeps you thorough and prevents the aimlessness of trying to check everything at once; the interruption tour, for instance, systematically finds “user killed the action midway” bugs that ad-hoc clicking would miss. (CRUD: walk every entity through Create/Read/Update/Delete and probe the seams; SFDIPOT: run your eye down seven product dimensions so you can’t forget the Platform or Time angle.)
- Reproduce — get it to fail again on purpose from a clean state; this discards the “rumor” status and turns a hunch into a fact (a failure you can’t reproduce isn’t a bug report yet). Minimize — strip every step and input not required for the failure; this discards all incidental noise, leaving the smallest trigger, which is the diagnosis. Automate — encode that minimal repro as a CI test; this discards reliance on human memory, making the guard permanent so the exact bug can never silently return.
- Exploration discovers a new failure category; you distil it into a minimal automated regression test; that test joins the permanent suite; and because the machine now guards all known cases, your human attention is freed to explore for the next unknown one. Remove exploration and the suite never learns new categories — it ossifies, guarding old bugs while blind to new ones. Remove the checklist and you have no memory: known bugs recur, and every session re-discovers the same traps instead of hunting new ground.