Null, Optional, and the Absence of a Value

We have attacked numbers, strings, collections, and dates. Every one of those was an edge inside a value that was there. This page is about the value that isn’t — the field that came back null, the key the JSON never sent, the string that is present but empty, the object that exists but holds nothing. Absence is the most common edge on this whole shelf, because absence hides in every other type. A number can be missing. A string can be missing. A collection can be missing. A date can be missing. “Nothing” is not one case; it is a case that rides along inside every field you named on the previous pages.

The trap is that “nothing” looks like one thing and is actually five things that are not interchangeable, and that most languages let you dereference “nothing” as if it were a value — which is where the single most expensive class of bug in software history lives. This page teaches you to name the flavors of absence, to make the compiler force you to handle the empty case, and to test the absence path of every boundary on purpose instead of discovering it in production.

The five flavors of “nothing”

Start by refusing to say “empty” or “no value” loosely. There are at least five distinct states a slot in your program can be in, and code that treats them as one is code with a latent bug:

  STATE            EXAMPLE                MEANS
  ──────────────────────────────────────────────────────────────────
  null / None      user.middle_name=null  present, explicitly "no value"
  missing key      {"name":"Ada"}         the field was never sent at all
  empty string     user.bio = ""          present, a value, of length zero
  empty collection user.roles = []        present, a container, holding nothing
  zero / default   user.credits = 0       a real value that looks like absence

These are not synonyms. Consider a user profile field phone:

• null — “we asked, the user has no phone.” A deliberate, recorded absence.
• missing key — the API version that produced this record didn’t have a phone field. Absence of the question, not the answer.
• "" — “the phone field exists and someone saved it blank.” Often a validation failure upstream.
• 0 / default — a numeric field defaulted to 0 is a value; a caller who reads if (credits) as “has credits” treats a legitimate zero balance as absence and skips the user.

A single conflation causes real bugs. “If discount is falsy, apply no discount” silently drops a genuine 0% code and a genuine 0-dollar order and an empty string that should have been rejected — three different intents collapsed into one if. The first discipline of this page: decide, per field, which of these five states are legal, and what each one means. Only then can you test them.

Present-but-null is different from absent

The distinction that trips even careful engineers is null versus missing. In JSON they look almost the same, but they carry opposite information:

  {"name": "Ada", "phone": null}   → phone was considered, answer is "none"
  {"name": "Ada"}                  → phone was never part of this payload

A PATCH request makes this life-or-death. {"phone": null} means “erase the phone.” {} (phone absent) means “leave the phone untouched.” Merge them and you either wipe fields the user never mentioned or ignore deletions they explicitly asked for. Any code that deserializes a partial update must distinguish “key present, value null” from “key absent” — and both must be test cases.

The billion-dollar mistake

In 2009, Tony Hoare — who introduced the null reference into ALGOL W in 1965 — publicly called it his “billion-dollar mistake.” His reasoning: it was so easy to add that he couldn’t resist, and it has since caused, by his estimate, on the order of a billion dollars in bugs, crashes, and vulnerabilities across the decades. The mechanism is simple and universal.

In most languages, a variable of type String can hold either a string or null, and the type system does not tell them apart. So user.name.length type-checks even when user.name is null — and at runtime you get a NullPointerException, a TypeError: Cannot read property 'length' of null, a segfault, or a nil panic. The type lied: it said “String,” it meant “String, or a landmine.”

  Declared type:   String name
  Actual values:   "Ada"  |  "Grace"  |  null   ← the type didn't warn you
                                         │
                            name.length  ─┘  ► crash at runtime

This is the null-dereference class of bug, and it hides specifically behind optional fields and nullable columns — the exact places you’d expect absence but the type didn’t force you to check for it. A database column declared NULLABLE, a JSON field that’s “sometimes there,” a lookup that “usually finds a match” — each is a nullable value wearing a non-nullable type, waiting for the one row, one payload, one miss where it’s actually empty.

It actually happened — a null in the wrong field

Null-dereference and “unexpected absence” failures are a genre, not a single incident, and they span from crashes to serious outages. The most instructive public example is the 2021 Facebook outage: during a routine maintenance change, a command’s effect was to withdraw the routes advertising the network to the internet, and downstream systems — including internal tooling and DNS — behaved as though the whole system had simply vanished, which cascaded because recovery paths assumed things would be present that were now absent. The enduring pattern, across many reported outages over the years, is the same: a value that was “always there” turns out to be absent exactly once, and the code on the happy path never modeled the empty case, so a missing thing becomes a crash or a cascade instead of a handled condition.

The testing lesson is independent of the specifics: the absence path is a real path through your code, and if you never send null/missing/empty through a boundary in a test, production will send it for you — usually at the worst time. Treat the incident as illustrative; the rule is firm.

Model absence explicitly: Option / Maybe / Result

The fix for the billion-dollar mistake is not “remember to check for null everywhere” — humans forget, and a forgotten check is invisible. The fix is to make absence a value in the type so the compiler refuses to let you ignore it. Languages call this Option/Maybe (Rust, OCaml, Haskell, Swift’s Optional, Kotlin’s T?), and for the “success or failure” flavor, Result/Either.

The idea: instead of String secretly meaning “String or null,” you write Option<String>, which is explicitly one of two shapes — Some(value) or None — and you cannot read the value without first opening the box. The empty case stops being a landmine you might step on and becomes a branch the compiler makes you write.

This is exactly what the repo’s own kvlite store does at its boundary. A GET on a missing key does not crash and does not return a magic empty string — it returns a distinct absence signal, NIL, that the caller must handle:

  > GET name
  NIL                         ← key absent: a real, named "nothing"
  > SET name ada lovelace
  OK
  > GET name
  VALUE ada lovelace          ← key present: the value, clearly tagged
  > DEL name
  DELETED
  > DEL name
  NOT_FOUND                   ← delete of an already-absent key: also named

Internally that’s a get returning something like Option<Value>, and the wire protocol makes the two cases un-confusable: NIL for absent, VALUE ... for present. Deleting a key that isn’t there isn’t a crash and isn’t silent success — it’s its own answer, NOT_FOUND. Absence is modeled, named, and therefore testable.

// The compiler will not let you forget the empty case.
match db.get("name") {
    Some(value) => println!("VALUE {value}"),
    None => println!("NIL"),          // ← must be written; can't be skipped
}

Compare the two worlds:

  NULLABLE (implicit)              OPTION (explicit)
  ────────────────────            ─────────────────────────
  fn get() -> String              fn get() -> Option<String>
  caller may forget the check     caller MUST match Some/None
  crash is possible at runtime     "forgot the empty case" = compile error
  absence is invisible in the type absence is right there in the type

You can get most of this discipline even in a nullable language: turn on the strictness that surfaces it (TypeScript strictNullChecks, Kotlin’s nullable types, C#‘s nullable reference types, Java’s Optional and @Nullable/static analysis). The goal is the same — move “did you handle nothing?” from hope to the compiler said so.

Under the hood — how “nothing” is actually stored

The five flavors of absence aren’t just a modeling convenience; they exist because different layers represent “nothing” with different bit patterns, and the boundaries between those layers are where confusion leaks in.

  LAYER          "nothing" is represented as
  ────────────────────────────────────────────────────────────
  C / pointers   NULL = the address 0x0  (dereference → segfault)
  SQL            NULL = a three-valued marker (NULL != NULL!)
  JSON           the literal null  — OR  the key simply absent
  Rust/Haskell   Option::None / Nothing — a real, taggable value
  JS             two of them: null AND undefined (missing ≈ undefined)

Two of these deserve special caution at test time. SQL NULL is not equal to itself: WHERE phone = NULL matches nothing — you must write WHERE phone IS NULL, and aggregates like COUNT(phone) silently skip nulls. A query that “loses” rows or miscounts is often a null-comparison bug, and it needs a test row whose column is actually NULL. JavaScript has two absences, null and undefined: a missing object key reads as undefined, an explicitly cleared field is often null, and == blurs them while === does not — so “is this absent?” has two different answers depending on which operator you reached for. When a value crosses one of these layer boundaries — SQL row to JSON response, JSON to a typed struct — the deserializer is exactly where “present-but-null,” “missing,” and “empty” must be pinned down, and exactly where a test should feed each one across the seam.

Partial data: present-but-empty is its own case

Explicit optionals fix the top-level “is there a value,” but real payloads are deeper, and absence hides at every level. A response can be partially there, and each degree of partialness is a separate test case:

  {"user": {"name": "Ada", "roles": ["admin"]}}   fully present
  {"user": {"name": "Ada", "roles": []}}          object present, list EMPTY
  {"user": {"name": "Ada", "roles": null}}        field present, value NULL
  {"user": {"name": "Ada"}}                        key MISSING
  {"user": null}                                   parent present, NULL
  {"user": {}}                                     parent present, EMPTY object
  {}                                               everything MISSING
  (no body at all)                                 response absent entirely

Code that does for role in user.roles is correct for the first line, harmless on the empty list, and crashes on null and on the missing key. These are four distinct behaviors from four inputs that a lazy reader calls “no roles.” An empty collection is safe to iterate and produces zero passes; a null collection is a null-dereference; a missing key is a KeyError; a null parent means you never even reach .roles. If your test suite only ever feeds the fully-present shape, none of these are exercised.

The principle from the collections page — test zero, one, many — extends here: for every nested field, the “zero” case forks into empty, null, and missing, and they do not behave the same. Enumerate them deliberately.

By the numbers — one field, its absence states

Field: user.roles  (expected: a list of role strings)

  INPUT              your loop `for r in roles`     correct behavior?
  ─────────────────────────────────────────────────────────────────
  ["admin","ops"]    iterates twice                 yes
  []                 iterates zero times            yes (no roles)
  null               TypeError / NPE / panic        NO — must default to []
  key missing        KeyError / undefined           NO — must default to []
  "admin" (a str!)   iterates CHARACTERS            NO — wrong type entirely

3 of 5 inputs that a human calls "empty roles" take a DIFFERENT path.
A test that only feeds the first row proves almost nothing about the
other four — which are exactly the ones that ship broken.

Test the absence path on purpose

The technique that ties this page together: pass null / None / missing / empty into every boundary deliberately, and assert a defined, safe behavior — not a crash. The absence path is a real branch through your code; the only question is whether you run it in a test or a user runs it in production.

Make it mechanical. For every input a function or endpoint accepts, ask: can this be null? missing? empty? zero? For each “yes,” write a case:

def display_name(user):
    # Intent: full name, or a safe fallback — never a crash.
    if user is None:                 # the whole object absent
        return "Unknown"
    name = user.get("name")          # key may be missing → None
    if not name:                     # None OR "" both fall here — decide!
        return "Unnamed"
    return name

def test_absence_paths():
    assert display_name(None)            == "Unknown"   # object absent
    assert display_name({})              == "Unnamed"   # key missing
    assert display_name({"name": None})  == "Unnamed"   # present but null
    assert display_name({"name": ""})    == "Unnamed"   # present but empty
    assert display_name({"name": "Ada"}) == "Ada"       # the happy path

Notice the comment on if not name: in Python that single test catches both None and "" — which is convenient here but is exactly the conflation that bites elsewhere (it also catches 0 and []). The discipline is to make the collapsing deliberate and tested, not accidental. Where the flavors must differ (the PATCH example: null erases, missing preserves), write name is None versus "name" not in payload explicitly, and test each.

A useful checklist to run against any boundary:

  For each field the boundary accepts:
    ☐ send it null / None          → assert defined behavior, no crash
    ☐ omit the key entirely        → assert defined behavior, no crash
    ☐ send it as "" or []          → assert this is distinct from null if it matters
    ☐ send it as 0 / false         → assert it's treated as a VALUE, not absence
    ☐ send the parent as null/{}    → assert you never dereference through it

The reward for this tedium is that the empty case stops being the input you forgot and becomes the input you chose — and your code returns "Unknown" instead of a stack trace.

The architect’s lens

• Why does it exist? Because a slot in a program can hold “nothing,” and “nothing” is not one thing — it is null, missing, empty, zero, and defaulted, five states that carry different meanings and take different code paths. Naming them exists so you can reason about, and test, each one.
• What problem does it solve? It kills the null-dereference class of bug — Hoare’s billion-dollar mistake — by forcing the empty case to be handled, and it prevents silent data corruption from conflating “erase this field” (null) with “don’t touch it” (missing).
• What are the trade-offs? Explicit optionality (Option/Maybe, strict null checks) adds ceremony: every access grows a match/if, and every partial payload multiplies your test cases. That verbosity is the cost of turning “forgot to check for nothing” from a runtime crash into a compile error.
• When should I avoid it? Never avoid modeling absence — but avoid inventing it: don’t make a field nullable/optional when the domain says it’s always present, because a spurious Option forces callers to handle a None that can’t occur and hides which absences are real. Model the absences that exist; forbid the ones that don’t.
• What breaks if I remove it? You get code that crashes on the first null a user sends, PATCH requests that wipe fields nobody edited, a 0 balance treated as “no balance,” and a green test suite that only ever ran the fully-present happy path — until production hands it the empty one.

Check your understanding

1. List the five flavors of “nothing” from this page and give a one-line example where treating two of them as interchangeable causes a real bug.
2. In a JSON PATCH payload, why must {"phone": null} and {} (phone absent) be handled differently, and what goes wrong if you merge them?
3. What is Hoare’s “billion-dollar mistake,” and where specifically does the null-dereference bug tend to hide (name the two kinds of fields)?
4. How does an Option/Maybe type prevent the null-dereference bug that a plain nullable String allows? Use the kvlite GET → NIL / VALUE behavior in your answer.
5. For a field roles expected to be a list, name at least three distinct “empty-ish” inputs that a human might call “no roles,” and explain why a for role in roles loop does not behave the same on all of them.

Show answers

1. null/None (present, explicitly no value), missing key (the field was never sent), empty string "" (present, length zero), empty collection [] (present, holds nothing), and zero/default (a real value that looks like absence). Example: treating 0 as absence — if (credits) applyBonus() — silently skips a user whose balance is a legitimate 0, conflating “zero credits” with “no credits field.”
2. {"phone": null} means “erase the phone”; {} with phone absent means “leave the phone untouched.” If you merge them — treating missing as null — you wipe fields the user never mentioned; if you treat null as missing, you ignore deletions the user explicitly requested. Partial-update code must distinguish “key present, value null” from “key absent,” and test both.
3. It’s the null reference Hoare added in 1965, which he later estimated has caused on the order of a billion dollars in crashes and vulnerabilities, because a nullable type (String) lets you dereference null as if it were a value and crash at runtime. It hides specifically behind optional fields (“sometimes there” JSON keys) and nullable database columns — the places you’d expect absence but the type didn’t force a check.
4. A plain String can secretly be null, so name.length type-checks and then crashes at runtime; Option<String> is explicitly Some(value) or None, and you cannot read the value without opening the box, so the compiler forces you to write the empty branch. kvlite does this at its boundary: a missing key returns the distinct, named NIL (its None) rather than a crash or a magic empty string, and a present key returns VALUE ... (its Some) — the two cases are un-confusable, and the caller must handle both.
5. Distinct inputs include: [] (present, empty list), null (present, null value), the key missing entirely, and even "admin" (a string mistaken for a list). for role in roles iterates zero times on [] (safe), but crashes on null (null-dereference) and on the missing key (KeyError/undefined), and on a bare string it iterates over characters — four different behaviors from inputs a lazy reader lumps together as “no roles,” which is why each is its own test case.