AI-Assisted Testing — Generation and Triage (as of 2024)

The last three pages handed the machine a search. Fuzzing let it search the input space; property-based testing let it search within a rule you wrote; chaos engineering let it search the failure space of a running system. In every case the human supplied the thing that made the search mean something — the invariant, the property, the steady-state hypothesis. The machine explored; you decided what “correct” was.

This page is about a newer helper that seems, at first glance, to break that pattern: a large language model that will happily write the test for you — inputs, assertions, fixtures, and all. It is genuinely useful, and it is genuinely dangerous, for the exact same reason. This is a fast-moving area, so everything below is dated and hedged on purpose: it describes what these tools do as of 2024, not a permanent state of the art.

Two live use cases, honestly scoped

Strip away the marketing and, as of 2024, AI adds real leverage to a tester’s day in two concrete places. They are worth separating, because one of them is far safer than the other.

   USE CASE 1 — GENERATION                USE CASE 2 — FLAKINESS TRIAGE
   "write me a test for this"             "which of these failures matter?"

   code / spec ─► LLM ─► draft test       thousands of ─► clustering ─► "these 40
                        (inputs,             historical       + ranking    reruns are
                         assertions,         test runs                     one root
                         fixtures)                                         cause"

   RISK: the assertion is a GUESS          RISK: low — it is re-reading
   at the correct answer.                  data you already trust.
   Must be human-verified.                 A human still acts on it.

Keep that asymmetry in your head for the whole page. Generation asks the model to invent what correct looks like — the one thing it cannot actually know. Triage asks it to find patterns in results you already have — a data problem it is genuinely good at.

The reason the asymmetry matters so much is that the two use cases fail in completely different directions. When triage is wrong, a human notices — a cluster looks off, a “real regression” turns out to be flaky, and you shrug and move on having lost a few minutes. When generation is wrong, nobody notices, because a wrong generated test is green, and green is exactly the signal we trained ourselves to trust. One failure mode is loud and cheap; the other is silent and expensive. A good tester spends their skepticism where the silent failures live.

Use case 1 — AI-assisted test generation

Point a 2024-era coding assistant at a function, a class, or a written spec and it will draft tests: plausible inputs, matching assertions, and the setup and fixtures to make them run. For well-trodden code — a validation routine, a parser, a CRUD handler — the drafts are often close to what an experienced tester would write, and they are produced in seconds instead of minutes.

The value is real but specific. The model is very good at the mechanical parts of a test: enumerating the empty/null/boundary cases you were taught to reach for, remembering the fixture boilerplate, covering the shapes of input a human would get bored halfway through. It is a tireless drafter of the cases you already know you should write.

It helps to name the three things generation drafts, because they carry very different risk:

   WHAT THE MODEL DRAFTS      HOW MUCH TO TRUST IT
   ─────────────────────      ────────────────────
   inputs / cases       ─►    high — it enumerates shapes tirelessly
   fixtures / setup     ─►    high — this is boilerplate it has seen a million times
   assertions           ─►    LOW  — this is a GUESS at the correct answer

The inputs and the fixtures are the parts a human finds tedious and the model finds easy, and there the two of you are well matched. The assertion is the part a human finds hard — because it requires knowing what correct means — and there the model only looks helpful. Read that third row every time.

Here is the shape of what you get. Ask a model to test a divide function and it will confidently produce something like this:

# LLM-drafted — looks complete, but read every assertion.
def test_divide_basic():
    assert divide(10, 2) == 5          # fine
    assert divide(9, 3) == 3           # fine

def test_divide_by_zero():
    with pytest.raises(ZeroDivisionError):
        divide(1, 0)                   # is this actually the spec? or a guess?

def test_divide_rounding():
    assert divide(7, 2) == 3           # ← integer division ASSUMED. Bug or intent?

Every line reads like a competent test. But the last assertion has silently decided that divide(7, 2) should be 3 — integer division — when the spec might have wanted 3.5. The model did not know; it pattern-matched a common convention and wrote it down as fact. That is not a bug in the tool. It is the tool doing exactly what it does, meeting the one problem it cannot solve.

Use case 2 — flakiness triage

The second use case points AI at a problem it is genuinely well suited to. A large test suite that runs many times a day produces a firehose of results, and some fraction of failures are flaky — intermittent, non-deterministic, unrelated to the change under test. Sorting real regressions from flaky noise by hand is soul-destroying and slow.

This is a data problem, and pattern-finding over historical runs is exactly where AI adds leverage with low correctness risk. Given the history of which tests failed, when, on which machines, after which changes, a model can:

Cluster hundreds of red runs into a handful of underlying causes (“these 40 failures are all the same timeout”).
Root-cause by correlating a failure with the commit, environment, or time-of-day it tends to accompany.
Prioritise — surface the failures most likely to be real regressions and quiet the ones that reliably self-heal on rerun.

None of these steps needs an oracle. Clustering asks “which of these failures resemble each other?”, root-causing asks “what did these failures co-occur with?”, and prioritising asks “which failures have historically turned out to be real?” — all questions about data you already have, none about what the correct answer should have been. That is the whole reason triage sits on the safe side of the line: it never has to invent correctness, only to organise evidence a human then judges.

Notice why this is safe in a way generation is not: the model never claims to know the correct output of anything. It re-reads results you already trust and finds structure in them. If it mis-clusters, you lose a little time; you do not merge a wrong test. A human still decides what to do with the cluster. The blast radius of a mistake is small.

Imagine a suite of 3,000 tests running 30 times a day. On a rough, illustrative day:

   90,000 test executions/day
        │
        ├─ 89,100 pass                        (99.0%)
        └─    900 fail                         (1.0%)
                │
                ├─ ~120 real regressions       (the ones that matter)
                └─ ~780 flaky / environmental  (noise a human must sift)

   Hand-triage of 900 failures at ~1 min each ≈ 15 hours of human attention.
   Cluster the 900 into ~25 root-cause groups ─► a human reviews 25, not 900.

The model did not decide any test was correct — it grouped failures you already had. The 15 hours saved is real leverage; the risk of a bad cluster is a little wasted time, never a shipped bug. That risk profile is why triage is the safer of the two use cases. The figures are illustrative; the ratio — a lot of noise, a little signal, buried together — is the durable part.

The oracle problem does not go away

Everything on this page funnels into one hard limit, and it is the same limit that has haunted testing since the tester’s mindset part: the oracle problem. A test needs two things — an input, and a way to know the correct output for that input. The second thing is the oracle. It is the hard half, and it always was.

   A TEST                     WHO CAN SUPPLY IT?
   ──────                     ──────────────────
   an input       ─────────►  an LLM, trivially (it will generate thousands)
   the CORRECT    ─────────►  NOT an LLM. Only the spec, the domain,
   output for it              the user, or a human who understands intent.

An LLM can generate an input effortlessly. It cannot know the correct output, because the correct output is defined by intent — the spec, the domain, the user’s actual need — and the model has none of those. It has a statistical sense of what output usually looks like for code that looks like this. That is a plausibility engine, not an oracle.

This produces the single most dangerous failure mode of AI-assisted generation: an AI-written assertion can encode a bug as “expected.” If the code under test is already wrong, the model reads the wrong behaviour, decides it looks normal, and writes an assertion that locks the bug in. The test passes. It is green. And it is now actively defending the defect — the next engineer who fixes the bug will see this test go red and “fix” the test back to broken. A generated test that agrees with buggy code is worse than no test, because it wears the costume of verification while doing the opposite.

The rule that falls out of this is not negotiable: a human must own the oracle. The model may draft the input and even a proposed assertion, but a person who understands the intended behaviour has to verify — for every assertion — that “expected” really is what the spec expects. There is no tool, in 2024, that removes this step. There is only the temptation to skip it.

Under the hood — why a plausibility engine is not an oracle

It is worth being precise about why the model gets the input right and the oracle wrong, because the reason is structural, not a temporary weakness that a bigger model patches.

A 2024-era LLM predicts likely text given prior text. When it drafts a test, it is answering “what does a test for code that looks like this usually look like?” — and tests for similar-looking code really are abundant in its training, so the inputs and structure it produces are excellent. But the assertion asks a different question: “what output is correct here?” Correctness is not a property of what the code looks like; it is a property of what the code was meant to do, which lives in the spec, the ticket, the conversation, and the user’s head — none of which the model can see from the code alone.

   QUESTION THE MODEL IS GOOD AT        QUESTION THE ORACLE REQUIRES
   ─────────────────────────────        ────────────────────────────
   "what does a test for code that      "what is the CORRECT result,
    looks like this usually contain?"    given what this code is FOR?"
        │                                     │
   answerable from patterns             answerable only from INTENT
   in similar code  ✅                   (spec / domain / user)  ❌ not in the code

So the model confidently fills the assertion with the output that is most common for code of that shape — divide(7, 2) == 3 because integer division is common — regardless of whether this division was meant to be exact. A larger, better model narrows the gap on average but never closes it, because the missing information was never in the input. This is the same lesson the oracle problem taught before AI existed: knowing the answer is a separate act from generating the question, and only intent can supply it.

Generated tests can inflate coverage without adding confidence

There is a second trap, and it connects directly back to the coverage and metrics part’s central warning. Generated tests are extremely good at moving the coverage number. Point a model at an untested module and coverage can jump twenty points in an afternoon. That number is seductive, and it can be almost entirely hollow.

Recall the mechanism from Goodhart’s law: coverage records that a line executed, never that an assertion checked what the line produced. AI generation makes the cheap, assertion-thin test trivially easy to mass-produce. You can flood a suite with tests that run every branch and verify almost nothing meaningful — or worse, verify the wrong thing, as above. The dashboard glows. The confidence it is supposed to stand for did not move.

   BEFORE                         AFTER an afternoon of AI generation
   coverage:   48%                coverage:   84%   ← looks like a win
   real oracles: ~40 checks       real oracles: ~41 checks (one added by hand)
   confidence: baseline           confidence: ~baseline — the number lied

The figures are illustrative, but the shape is the trap: coverage was never the goal, and a tool that makes coverage cheap to inflate makes that lesson more urgent, not less. If you let generated tests be scored by the line count they light up, you have simply industrialised the exact gaming the coverage part warned you against.

There is a compounding danger worth naming. Before AI, an assertion-free or wrong-assertion test cost effort to write, and that effort was a small natural brake on how many hollow tests entered a suite — a bored human only writes so many. Generation removes the brake. It is now as cheap to produce a thousand plausible-but-unverified tests as ten, so a suite can bloat from “small and understood” to “large and unread” in a single afternoon, and every one of those tests will run in CI forever, slowing the pipeline and, worse, lending the appearance of a thoroughly tested codebase. A big green suite that no human has read is not evidence of quality; it is a liability wearing the badge of one. The size of a suite was never the point — the number of verified oracles in it always was.

Realistic framing — what these tools are, in 2024

A few things are worth stating plainly, and stating with dates, because this is the part most likely to be out of date by the time you read it:

Capabilities move fast. Any specific claim about what a model “can” or “can’t” do is a snapshot. As of 2024 the picture is “strong drafter, weak oracle.” That balance may shift; the structure of the argument — you still need an oracle a human trusts — is far more durable than any capability claim.
Results are non-deterministic. Ask the same model for tests twice and you may get different tests, different assertions, different assumptions. This is fine for a draft you review and unacceptable for anything you merge unread.
Augmentation, not replacement. As of 2024 these tools augment a tester’s judgment — they draft the tedious parts and surface patterns in noise — they do not replace the judgment itself. The scarce skill was never typing tests; it was knowing what correct means and where the risk lives. AI makes the typing cheap and leaves the scarce skill exactly where it was.

Put the whole picture on one line: AI moved the cost of writing a test toward zero, and did nothing to the cost of knowing what the test should assert. Testing was never bottlenecked on typing. It was bottlenecked on judgment — on deciding what “works” means for this system and where the dangerous edges are. A tool that accelerates the cheap half and leaves the expensive half untouched is genuinely useful, but only if you keep spending on the expensive half. The failure mode is spending the saved time on more generated tests instead of on verifying the assertions you already have — trading a small, honest suite you understand for a large, plausible one you don’t.

There is a well-known and recurring pattern, long predating AI, of characterisation tests — tests written to capture whatever the code currently does, without checking whether that behaviour is correct. They are legitimately useful for pinning down legacy code before a refactor. But teams have repeatedly been burned when a characterisation test, written against already-broken behaviour, silently became the “spec”: a later correctness fix turned the test red, someone reverted the fix to make the suite green, and the bug shipped defended by its own test. AI generation industrialises this hazard — it produces assertion-against-current-behaviour at scale, in seconds, with no signal about whether the current behaviour was ever right. The specific incidents vary and are easy to misattribute, so treat any single retelling as illustrative; the mechanism is solid. The lesson is blunt and dateless: a green test written against unverified behaviour is not verification — it is a photograph of a bug with a frame around it.

Practical guardrails

You do not need to fear these tools; you need to use them like the drafts they are. As of 2024, a small set of guardrails keeps the leverage and drops the danger:

Treat AI output as a draft to review, never as a result. The model’s job ends at “here is a plausible test.” Yours begins there. Read every assertion and ask: is this what the spec wants, or what the model guessed the code does?
Keep a human accountable for the oracle. Name the person who verified that “expected” is correct. If nobody can say who checked the assertions against intent, the oracle is unowned, and an unowned oracle is no oracle.
Never merge a green test nobody read. “It passes” is not a reason to merge; it is the default state of a generated test, whether or not it verifies anything. Green means nothing until a human has confirmed what is green.
Score generated tests by oracle, not by coverage. When you accept an AI-drafted test, the question is “does this assertion catch a real fault?” — not “how many lines did it light up?” Refuse to let generation move a coverage gate.
Lean hard on triage, lightly on generation. Flakiness triage is where the risk/reward is best: it finds structure in data you already trust, and a mistake costs a little time. Generation earns its keep only under review. Weight your trust accordingly.

A concrete review loop for a generated test

Guardrails are only as good as the habit that enforces them. Here is the minimal loop to run on every AI-drafted test before it enters the suite — it is short on purpose, because a review step people actually perform beats a thorough one they skip:

   FOR EACH GENERATED TEST
   ───────────────────────
   1. Read the ASSERTION first, not the setup.   ← the oracle lives here
   2. Ask: is "expected" from the SPEC, or from
      the code's CURRENT behaviour?               ← if the latter, STOP
   3. Mutate the code under test by hand and
      confirm the test goes RED.                  ← proves it checks something
   4. Delete assertion-free / tautological cases. ← they only inflate coverage
   5. Name yourself as the human who owns it.     ← unowned oracle = no oracle

Step 3 is the cheapest high-value move on the list, and it is worth dwelling on: a test that stays green when you deliberately break the code it covers is not testing that code — it is decoration. Breaking the implementation by hand and watching the assertion fail is a five-second proof that the oracle is real. It is the same discipline as mutation testing applied one test at a time, and it is exactly the check a generated test most often fails, because a plausibility engine writes assertions that describe code far more reliably than assertions that constrain it.

The architect’s lens

Why does it exist? Because the two most tedious parts of a tester’s day — drafting the obvious cases and sorting real failures from flaky noise — are, respectively, a text-generation problem and a pattern-finding problem, and 2024-era models are good at both. It exists to take the typing and the sorting off a human’s plate.
What problem does it solve? It compresses the mechanical labour of testing: enumerating boundary cases, writing fixture boilerplate, and clustering thousands of historical runs into a few root causes. It does not solve — and cannot solve — the problem of knowing what “correct” is.
What are the trade-offs? You trade speed for a new, subtle risk: an AI-written assertion can encode a bug as “expected,” and generated tests can inflate coverage while adding no confidence. Every second saved on drafting must be partly re-spent on human verification, or the saving is an illusion.
When should I avoid it? Avoid unreviewed generation anywhere the correct answer is subtle, safety-critical, or genuinely unknown to the model — precisely the code where an oracle matters most. Never wire generated tests to a coverage gate, and never let “it’s green” substitute for “a human checked the oracle.”
What breaks if I remove it? Very little that is load-bearing. You lose a fast drafter and a good flaky-test triager, and you go back to writing boilerplate by hand and grinding through failure logs — slower, but with the oracle already where it belongs: in a human’s head. Removing the tool costs convenience; removing the human oracle costs correctness.

Next we push into territory where the oracle problem is not a footnote but the whole subject: testing ML systems, where there is no fixed correct answer to check against at all.

Check your understanding

This page separates two 2024 use cases and calls one far safer than the other. Which is which, and what single property makes the difference?
State the oracle problem in your own words, then explain precisely why an LLM can supply half of a test but not the other half.
Describe the “encode a bug as expected” failure mode step by step, and explain why the resulting green test is worse than having no test at all.
AI generation can lift a coverage number by twenty points in an afternoon. Tie this back to Goodhart’s law: why is that jump often hollow, and what should you score a generated test by instead?
Give three of the practical guardrails and explain the one principle they share.

Show answers

Test generation is the riskier one; flakiness triage is the safer one. The difference is the oracle: generation asks the model to invent what correct looks like (the assertion), which it cannot actually know, whereas triage only finds patterns in results you already trust — a wrong cluster costs a little time, a wrong assertion can lock in a bug. Triage never claims to know a correct output; generation does.
The oracle problem: a test needs both an input and a trustworthy way to know the correct output for that input (the oracle), and the oracle is the hard half. An LLM supplies inputs trivially — it will generate thousands — but the correct output is defined by intent (spec, domain, user need), which the model does not have. It has only a statistical sense of what output usually looks like, which is plausibility, not an oracle.
If the code under test is already wrong, the model reads that wrong behaviour, judges it normal-looking, and writes an assertion that says the wrong output is “expected.” The test passes and goes green. It is worse than no test because it now defends the bug: the next engineer who correctly fixes the code sees this test fail and is tempted to “fix” the test back to matching the broken behaviour — so the wrong test actively resists correction while wearing the costume of verification.
Coverage records only that a line executed, never that an assertion checked the result, and AI makes assertion-thin (or wrong-assertion) tests trivially cheap to mass-produce — so the number can rise twenty points while real, fault-catching oracles barely move. That is Goodhart’s law: the rewarded proxy (coverage) decouples from the goal (confidence). Score a generated test by whether its assertion catches a real fault — its oracle — not by how many lines it lights up, and never let generation move a coverage gate.
Any three of: treat AI output as a draft to review, never a result; keep a named human accountable for the oracle; never merge a green test nobody read; score generated tests by oracle rather than coverage; lean hard on triage and lightly on generation. The shared principle: a human must own the oracle — the tool may draft inputs and even assertions, but only a person who understands the intended behaviour can verify that “expected” is actually correct, and no 2024 tool removes that step.