File the Bug You Found

Every page of this Part has produced an artifact. We mapped the risks, designed the cases, wrote unit and integration tests, added end-to-end tests and a coverage goal, and wired it all into CI. Somewhere in that work, honest testing did the thing honest testing always eventually does: it turned something red.

This page is about what happens next. A failing test is not the finish line — it is a lead. The finish line is a fix that ships and a regression test that guarantees the defect can never silently come back. Between those two points sits one small, unglamorous, load-bearing artifact: the bug report. This page writes one, properly, for a real defect Snip’s boundary tests surface.

The defect the boundary tests found

Snip has a feature we glossed over in the overview: short codes can expire. A code created with a time-to-live is meant to stop resolving once it lapses. The risk register (from Map the Risks) has an entry for it:

  RISK-07  Expired code still resolves
           Likelihood: medium   Impact: high (dead/stale links, possible
           redirect to a URL the creator wanted to retire)

The test-design page turned that risk into a boundary case. Expiry is a boundary — a moment in time with a before and an after — so we test both sides and the instant itself:

  t < expiry     GET /:code  ->  302  (still valid)
  t = expiry     GET /:code  ->  404  (boundary — the instant it lapses)
  t > expiry     GET /:code  ->  404  (expired)

We wrote the case as a black-box end-to-end test that creates a code with a 1-second TTL, waits past it, and asserts a 404. Here it is, and here is what happened when CI ran it:

def test_expired_code_returns_404(client):
    # Arrange: create a code that lives for 1 second.
    r = client.post("/shorten", json={"url": "https://example.com/a",
                                       "ttl_seconds": 1})
    code = r.json()["code"]

    # Act: let it expire, then resolve it.
    time.sleep(1.2)
    r = client.get(f"/{code}", allow_redirects=False)

    # Assert: an expired code must not resolve.
    assert r.status_code == 404

  FAILED tests/e2e/test_expiry.py::test_expired_code_returns_404
  assert 302 == 404
   +  where 302 = <Response>.status_code

The test expected 404. Snip returned 302 — a redirect to the expired URL. The expiry check exists on the write path but not on the read path: GET /:code looks the code up and redirects without ever comparing now to the stored expiry. This is exactly RISK-07, made concrete. A vague worry (“do expired codes really stop working?”) is now a reproducible fact.

That is the whole point of the last six pages. We did not stumble on this bug in production three months from now, from a confused user’s report. We designed a case for the boundary we were already worried about, and the case caught the gap before a single user did.

Anatomy of a bug report that gets fixed

A bug report is a hand-off. Whoever fixes this — a teammate, or you in two weeks having forgotten everything — needs to reproduce the defect on the first try, understand its severity without guessing, and know when it is fixed. A good report is engineered for exactly that. It has seven parts.

  1. Title        one line, specific: symptom + where
  2. Environment  version/commit, OS, config that reproduces it
  3. Steps        exact, numbered, copy-pasteable — no "just"
  4. Expected     what the spec/contract says should happen
  5. Actual       what actually happened, verbatim
  6. Severity/    how bad, and how soon — tied to real impact
     Priority
  7. Evidence     the failing test + logs that prove it

The two failure modes of bug reports are both about the reader. A report that is too vague (“expiry seems broken”) forces the fixer to re-derive everything you already knew, and half the time they cannot reproduce it and close it as “works for me.” A report that is too much narrative (“so I was testing and I noticed…”) buries the three lines that matter. Aim for the middle: terse, exact, evidence-first.

The filled-in report

Here is RISK-07 written up. Read it as a template you can reuse for any defect.

  Title:   GET on an expired code returns 302 (redirects) instead of 404

  Environment:
    Component:  Snip API
    Version:    commit 4f9c1ab (main), 2026-07-09 build
    Runtime:    Python 3.12, container image snip:ci-4f9c1ab
    Config:     default; store=postgres, code TTL honored on write

  Steps to reproduce:
    1. POST /shorten  {"url": "https://example.com/a", "ttl_seconds": 1}
       -> 200, {"code": "a3Xk9", ...}
    2. Wait > 1 second (e.g. sleep 1.2s) so the code has expired.
    3. GET /a3Xk9  with redirects disabled.

  Expected:
    HTTP 404 Not Found. An expired code is equivalent to an unknown
    code; it must not resolve. (Spec: overview, GET /:code contract.)

  Actual:
    HTTP 302 Found, Location: https://example.com/a
    The expired code still redirects to the original URL.

  Severity: High     Priority: High
    Root cause (suspected): expiry is checked on the write path but
    not on the read path in GET /:code; the lookup ignores the stored
    expiry timestamp.

  Evidence:
    Failing test: tests/e2e/test_expiry.py::test_expired_code_returns_404
      assert 302 == 404
    Risk register: RISK-07 (Expired code still resolves).

Notice what is not in it: no “I think”, no story, no blame. Every line is a fact the fixer can act on. The title alone — symptom (returns 302 instead of 404) plus location (GET on an expired code) — tells a triager most of what they need before they read a word further.

Two details in that report earn their keep and are worth calling out. The environment pins the exact commit (4f9c1ab) and image, so “I can’t reproduce it” becomes a real signal rather than a version mismatch — if it reproduces on that commit and not on yours, the difference between the two commits is the bug. And the suspected root cause is offered as a hypothesis, not a verdict: it points the fixer at the read path without pretending to have already done their debugging. A report that guesses the root cause and is wrong wastes more time than one that stays silent, so hedge it honestly with “suspected.”

Under the hood — why the failing test is the reproduction

The most valuable line in that report is the one pointing at a test. In a report without one, “steps to reproduce” is a set of instructions a human follows by hand, imperfectly. With an automated failing test, the reproduction is executable: the fixer runs one command and watches it go red, with zero ambiguity about environment or timing.

  Bug report WITHOUT a test        Bug report WITH a test
  ─────────────────────────        ──────────────────────
  human re-reads steps             `pytest -k expired_code`
  sets up state by hand            state set up in code
  "works for me" is possible       red is red for everyone
  fix verified by eyeballing       fix verified: red -> green
  can silently regress later       CI blocks the regression forever

This is why we attach the minimal reproducing case, not a screenshot of the whole suite. The report should hand over the smallest program that fails. That minimality is a gift to the fixer (less to understand) and to the codebase (a tight, fast, permanent guard).

Severity and priority, assigned honestly

Two words that get used interchangeably and should not be:

Severity is how bad the defect is when it happens — its impact on the user or the system. It is a property of the bug.
Priority is how soon it should be fixed relative to everything else — its position in the queue. It is a property of the schedule.

They come apart constantly. A typo on the “About” page is low severity and, before a big launch, might briefly be high priority. A rare crash that corrupts data is high severity but, if it needs a once-in-a-year input, sometimes lower priority than a bug hitting every user today. Assigning them honestly means resisting the two temptations: inflating everything to “critical” (which trains everyone to ignore your severities) and deflating your own bugs to avoid the awkwardness of blocking a release.

For RISK-07, tie the numbers back to the risk register instead of guessing. The register already did the impact analysis: an expired code that still resolves means dead or retired links keep working — including links a creator deliberately expired to stop sending traffic to a URL (a takedown, a rotated secret in a query string, a promotion that ended). That is a High-severity, High-priority defect, and the report says so with the reasoning attached, not as an opinion.

The industry’s oldest, most-repeated finding is that a defect gets more expensive the later it is caught. The exact multipliers vary by study and are often over-cited, so treat them as orders of magnitude, not physics — but the direction is not in dispute.

  Where the expiry bug is caught      Rough relative cost to resolve
  ────────────────────────────────    ──────────────────────────────
  In test design (as a risk)          ~1x   (write one case)
  In CI, on the pull request          ~1x   (fix before merge; here)
  In staging / QA                     several x
  In production, user-reported        many x + trust + incident time

Our expiry bug was caught on the pull request, at the cheapest possible point. The whole apparatus of this Part — risk register, designed boundary case, CI gate — exists to move defects leftward on that table. The bug report is what keeps a leftward-caught defect from drifting rightward again.

Close the loop: risk → test → bug → regression guard

Now watch the full loop close. This one defect touched every artifact the Part produced, in order:

  RISK-07 (Map the Risks)
     "an expired code might still resolve"        <- a worry
        │
        ▼
  Boundary case (Design the Test Cases)
     t<expiry:302 · t=expiry:404 · t>expiry:404   <- a plan
        │
        ▼
  Executable test (E2E page) + CI (Wire into CI)
     test_expired_code_returns_404  ->  RED       <- evidence
        │
        ▼
  Bug report (this page)
     title · env · steps · expected/actual ·
     severity · the failing test as proof         <- an actionable hand-off
        │
        ▼
  Fix + the same test  ->  GREEN
     and it stays in the suite forever            <- a regression guard

The last box is the one people forget. Once the fix lands and the test goes green, you do not delete the test. It stays in the suite as a permanent tripwire. The next engineer who “optimizes” the read path and drops the expiry check will not ship it, because test_expired_code_returns_404 will turn red in CI and block the merge. The bug can never silently return. It can only return loudly, at the cheapest possible moment, to the person who caused it.

That is the difference between fixing a bug and closing a bug. A fix makes the symptom go away today. A regression test makes the symptom’s return impossible to miss forever. A bug report that carries its own failing test is what connects the two: it is simultaneously the evidence that the bug is real, the instructions to reproduce it, and — after the fix — the guard that keeps it dead.

Look back at the overview’s throughline: what could break, and how do we know it doesn’t? RISK-07 was a “what could break.” The failing test was proof that, at that moment, it did break. The fix plus the retained regression test is the finished “how we know it doesn’t” — not a feeling, but an executable check that runs on every future change. The loop is closed.

Knight Capital’s 2012 trading disaster (introduced in this Part’s overview) hinged on a code path that had been dormant and was later reactivated by a configuration edge case. There was no automated guard asserting that the old behavior stayed dead. The lesson maps directly onto this page: it is not enough to fix or retire a behavior once. Without a retained test that fails the instant the old behavior comes back, a future change — a deploy that misses a server, a flag flipped by accident — can silently resurrect it. A regression test is the difference between “we fixed that” and “that can never quietly return.” Snip’s expiry test is small; the principle is the one that companies have lost their lives to ignoring.

The architect’s lens

Filing a defect as a report-plus-regression-test is a technique, not paperwork. Here is why it exists.

Why does it exist? Because a fix without a written, reproducible report and a retained test is a private, temporary act — it lives in one person’s head and lasts until the next refactor. The report-plus-test makes the defect and its resolution durable, shared, and self-verifying.
What problem does it solve? It closes the gap between finding a bug and keeping it fixed: unambiguous reproduction for the fixer, an honest severity for the scheduler, and a permanent tripwire so the same defect cannot silently return.
What are the trade-offs? It costs time up front — writing exact steps, minimizing the repro, adding a test — and every retained regression test is a small permanent maintenance cost in the suite. You pay that so the far larger cost of re-finding and re-debugging the defect never recurs.
When should I avoid it? Almost never for a real defect. The one anti-pattern is ceremony for its own sake: a throwaway prototype you are about to delete does not need a formal report, and a flaky non-deterministic “failure” should be diagnosed before it is filed as a bug.
What breaks if I remove it? Fixes stop being verifiable (“works for me”), severities become opinions instead of impact-backed decisions, and defects regress silently — the exact failure mode behind the most expensive outages in the history of software.

Check your understanding

The boundary test for expiry checks three points in time. What are they, and why is testing the exact instant of expiry (t = expiry) worth a separate case?
Name the seven parts of the bug report, and explain why the title should contain both a symptom and a location.
Distinguish severity from priority. Give one example where a bug has high severity but low priority, and one where it has low severity but high priority.
Why is attaching the minimal failing test more valuable than attaching a screenshot or a paragraph of prose steps? Give two distinct reasons.
After the fix makes the test go green, why do you keep the test in the suite instead of deleting it? Tie your answer to the phrase “the bug can never silently return.”

Show answers

The three points are before expiry (t < expiry, expect 302), at the instant of expiry (t = expiry, expect 404), and after expiry (t > expiry, expect 404). The instant itself is a classic off-by-one boundary: an implementation using now < expiry versus now <= expiry behaves differently at exactly that moment, so it needs its own case to pin down which side of the boundary the code lands on.
Title, environment, steps to reproduce, expected, actual, severity/priority, and evidence (the failing test/logs). The title carries both a symptom (returns 302 instead of 404) and a location (GET on an expired code) so a triager can understand and rank the bug from the title alone, before reading the body — the title is what people search, sort, and skim.
Severity is how bad the defect is when it occurs (impact); priority is how soon it should be fixed relative to other work (schedule). High severity / low priority: a data-corrupting crash that requires a once-a-year input combination. Low severity / high priority: a typo on the landing page the day before a major launch.
First, the failing test is an executable reproduction — the fixer runs one command and sees red with no ambiguity about environment or timing, eliminating “works for me.” Second, once the fix lands, that same test becomes a permanent regression guard in CI, so the defect cannot come back silently — neither of which a screenshot or prose steps can do.
Because the retained test is a permanent tripwire: any future change that reintroduces the defect turns the test red in CI and blocks the merge. That converts the bug from something that could quietly reappear in production into something that can only reappear loudly, at the cheapest moment, in front of the person who caused it — which is exactly what “the bug can never silently return” means.