Accessibility, Usability & Compatibility — Testing for Real Users

The last three pages checked things a machine can measure: is the answer correct, is it fast enough, can an attacker break in. This page turns to a question none of those asks: when a real human — with their own eyes, hands, assistive tech, browser, and phone — sits down in front of this software, does it actually work for them?

Three kinds of testing live under that question. Accessibility asks can everyone use it, including people with disabilities. Usability asks is it actually usable, even by someone with no impairment and no manual. Compatibility asks does it work across the matrix of browsers, operating systems, and screen sizes users actually run. They are grouped together because they share one dangerous property: they all fail silently for the one person least likely to notice — the developer whose setup already works.

Why these three fail silently

A functional bug is loud. A crash logs a stack trace; a wrong number fails an assertion. The three human-facing concerns are quiet, and each is quiet for the same structural reason: the developer is not the affected user.

   the developer tests on:        the user actually has:
   ──────────────────────         ──────────────────────
   a mouse and 20/20 vision   →   a screen reader, or a tremor, or low vision
   the app they built weekly  →   never seen it before, under time pressure
   latest Chrome on a 27" Mac →   Safari on a three-year-old Android phone

Nothing in that left column ever triggers the failure in the right column. The code runs. The tests pass. “Works on my machine” is true — and useless, because the machine is not representative. This is why these concerns are easy to skip (they never bite you in development) and expensive to ignore (they bite every user who isn’t you, and you find out from an angry review, a lawsuit, or a support queue instead of a red test). The whole discipline here is deliberately reproducing conditions you don’t personally experience.

Accessibility testing — can everyone use it

Accessibility (often abbreviated a11y) is the check that people with disabilities — visual, motor, auditory, cognitive — can perceive, operate, and understand the software. It is not a niche courtesy: it is a legal requirement in many jurisdictions, and the techniques that help a blind user usually help everyone (captions help in a noisy room; keyboard support helps power users).

The reference standard is WCAG — the Web Content Accessibility Guidelines. Its success criteria are organized under four principles, remembered as POUR:

   P  Perceivable   — can the user sense the content?  (alt text, captions, contrast)
   O  Operable      — can they operate it?             (keyboard, no time-traps)
   U  Understandable — is it clear and predictable?    (labels, error messages)
   R  Robust        — does it work with assistive tech? (valid, named semantics)

WCAG criteria come at three conformance levels — A (must), AA (should, and the level most laws target), and AAA (aspirational). “We meet WCAG 2.1 AA” is the sentence most compliance requirements reduce to, as of mid-2026.

What you actually check

Four concrete checks catch most real failures:

• Keyboard navigation. Unplug the mouse. Can you reach every interactive element with Tab, activate it with Enter/Space, and see where focus is? A custom <div onClick> that isn’t focusable is invisible to a keyboard user — the single most common a11y bug.
• Screen reader. Turn on VoiceOver (macOS), NVDA (Windows), or TalkBack (Android) and navigate blind. Does each button announce a meaningful name, or does the reader say “button, button, button”? An icon-only button needs an accessible name; an image needs alt text.
• Color contrast. Text must contrast enough with its background — WCAG AA requires a ratio of at least 4.5:1 for normal text, 3:1 for large text. Light-grey-on-white “looks clean” and is unreadable for millions of users.
• Semantics and labels. Every input needs a programmatic label; headings must nest in order; state changes (an error appearing) must be announced, not just shown.

Automated vs manual — and why you need both

An automated scanner (axe, Lighthouse, pa11y, and similar tools, as of 2026) parses the rendered page and flags machine-detectable violations: a missing alt, an input with no label, a contrast ratio below threshold. This is fast, cheap, and belongs in CI.

But automated tools catch only part of the problem — commonly cited industry figures put it around a third to a half of WCAG issues, and you should treat that as a rough ceiling, not a target. The rest is inherently human.

   AUTOMATED catches:                MANUAL catches:
   ─────────────────                ───────────────
   missing alt attribute       →    alt text that says "image123.png"
   input without a label       →    a label that's technically present but wrong
   contrast below 4.5:1        →    a focus order that jumps around illogically
   invalid ARIA role           →    a screen-reader flow that's technically valid
                                     but incomprehensible

A machine can tell you alt="" is missing. Only a person can tell you alt="chart" is useless on a chart that conveys three quarters of revenue. Automated checks are the floor; a keyboard-and-screen-reader walkthrough by a human is the real test.

It actually happened — the Domino’s accessibility case, 2019

A blind customer sued Domino’s Pizza in the United States after being unable to complete an order on its website and app using screen-reader software. In 2019 the Supreme Court let a lower-court ruling stand, effectively holding that accessibility law applied to the company’s digital storefront. The lesson for testing is blunt: a site can be fully functional — every endpoint works, every order processes — and still be unusable, and legally non-compliant, for an entire class of users, because nobody ever ran it with a screen reader. Accessibility is not a feature you add later; it is a way you test from the start. (Details summarized from public reporting.)

Usability testing — is it actually usable

Accessibility asks can a person use it. Usability asks will they — easily, without frustration, without a manual. And here is the sharpest distinction on this page: usability testing is not a pass/fail assertion. There is no assert_eq! for “this menu is confusing.” It is a qualitative, human-in-the-loop method: you put a real person in front of the software, give them a task, and watch.

   functional test:  give INPUT → assert OUTPUT == EXPECTED   (binary, automatable)
   usability test:   give a TASK → observe a HUMAN attempt it (graded, human-run)

   "Order a large pizza for pickup."
      → Where do they hesitate? What do they click that isn't the answer?
      → Where do they give up? What do they say out loud?

The core technique is moderated task observation: recruit five or so people from the real audience, give each a realistic goal, ask them to think aloud, and record where they stumble — without helping them. The friction they hit is data no specification could have contained, because a spec encodes what the team intended, and usability testing measures what a stranger experiences. The gap between those two is exactly the bug.

By the numbers — why five users

A widely cited usability finding (Nielsen, popularized around 2000 and still the industry rule of thumb) models the probability of catching a given problem with n users as:

   problems_found(n) = 1 − (1 − L)^n      with L ≈ 0.31 (per-user detection rate)

   n = 1   →  31% of problems surfaced
   n = 3   →  67%
   n = 5   →  84%
   n = 15  →  99%

Because the same big problems recur across users, the first five testers surface the large majority of issues, and each additional tester finds less. The practical takeaway: run small, frequent studies — five users, fix, retest — rather than one giant study. It is a heuristic, not a law; the exact detection rate varies by product.

Usability sits apart from every other kind of testing in this Part precisely because it can’t be reduced to an automated check. You can measure proxies — task completion rate, time-on-task, error count, a System Usability Scale score — but the richest findings come from watching a face fall when a stranger can’t find the button you were sure was obvious.

Compatibility testing — does it work everywhere users are

Compatibility (also cross-browser / cross-device testing) verifies that the same behavior holds across the matrix of environments your users actually run: browsers (Chrome, Safari, Firefox, Edge), rendering engines, operating systems, screen sizes, and versions. The functional test that passes on your laptop’s Chrome says nothing about Safari on an iPhone with a notch, or an Android tablet in landscape, or a corporate machine still on an old browser.

                Chrome   Safari   Firefox   Samsung Internet
   Windows 11    ✓        —        ✓         —
   macOS         ✓        ✓        ✓         —
   iOS           ✓*       ✓        ✓*        —          (*all iOS uses Safari's engine)
   Android       ✓        —        ✓         ✓

Each cell is a distinct environment where CSS can render differently, a JavaScript API may be missing, a date can parse in another format, or a touch target can be too small. The failure mode is a layout or behavior difference, not a crash — a button pushed off-screen, a form that won’t submit on one engine, a video codec one platform can’t play.

Why you cannot test the whole matrix

Multiply four browsers by three OSes by many versions by many screen sizes and you get thousands of combinations. Testing all of them is impossible and, more importantly, wasteful — almost nobody uses most of those cells. The professional move is to bound the matrix with real analytics, not with a wish list.

   DON'T: "support every browser" → infinite, and mostly serving no one
   DO:    pull your own analytics  → a ranked, representative support set

   from real traffic (illustrative):
     Chrome  62% │ Safari 22% │ Edge 8% │ Firefox 5% │ other 3%
     mobile 58% │ desktop 40% │ tablet 2%
   → test set: latest + 1 of the top ~4 browsers, on the 2–3 most common
     screen sizes, weighted by who actually visits — covers ~95% of sessions

Pick a representative support set — the smallest list of environments that covers the overwhelming majority of your real sessions — and test that thoroughly, rather than testing everything badly. Then draw an explicit line: environments below your threshold get “should basically work” rather than a guarantee, and you say so publicly. This is the same bounding instinct as every other testing decision in this book: you cannot test infinity, so you spend your finite budget where the users actually are.

Under the hood — how the matrix gets run

You do not own three hundred physical devices. In practice compatibility testing runs one of three ways, in increasing fidelity:

1. Emulators / responsive mode. Your browser’s dev tools fake a screen size and touch. Cheap and instant, but they use your rendering engine, so they catch layout issues and miss engine-specific bugs entirely.
2. Real-device cloud farms. Services (BrowserStack, Sauce Labs, LambdaTest, and similar, as of 2026) rent actual devices and browsers over the network. You run your end-to-end suite against Safari-on-a-real-iPhone without owning one.
3. A small physical device lab. A handful of the most important real phones on a desk for the final human look. Nothing beats holding the actual device your top-traffic users hold.

The automatable part — “does this page render and this flow complete on browser X” — folds neatly into the same end-to-end suite from earlier pages; you just parameterize it across your support set and run it on a device farm in CI.

The architect’s lens

• Why does it exist? Because software is correct for a person in a context, and the developer’s person-and-context is never representative. These three kinds of testing exist to check the software under conditions the author does not personally experience — a disability, a fresh pair of confused eyes, a different browser on a cheaper phone.
• What problem does it solve? They close the gap between “passes every functional test on my machine” and “actually works for the humans who use it.” Accessibility closes the who can use it gap, usability the can they use it easily gap, compatibility the does it work in their environment gap.
• What are the trade-offs? They are slower, partly manual, and can’t be fully reduced to a green check — usability especially needs real humans and judgment. Automated a11y and cross-browser runs help, but they are a floor, not a proof, and the manual remainder costs real time.
• When should I avoid it? Never skip them for anything a real, varied audience touches. You can scope them down for an internal tool with one known browser and no disabled users in scope, or a throwaway prototype — but “we assumed nobody would need it” is exactly the assumption that produces the incidents above.
• What breaks if I remove it? The software silently excludes people (accessibility), quietly frustrates and loses them (usability), or plainly breaks for a slice of them (compatibility) — and because it all fails on their machine and not yours, you learn about it from lost users, bad reviews, and lawsuits instead of from a test.

Check your understanding

1. All three kinds of testing on this page share one reason they are easy to skip and expensive to ignore. What is that shared reason, and how does “works on my machine” make it worse?
2. What does WCAG’s POUR stand for, and why can an automated accessibility scanner never be enough on its own? Give one example of a bug a machine misses but a human catches.
3. Usability testing is described as fundamentally different from a functional test. What is the difference, and why can’t “this screen is confusing” be written as an assert?
4. You are told to “support every browser.” Explain why that instruction is wrong, and describe how you would instead bound the compatibility matrix.
5. Name one concrete check for each of accessibility, usability, and compatibility, and say who or what performs it (a machine, a human, or both).

Show answers

1. The shared reason is that the developer is not the affected user — the failure occurs under conditions (a disability, a first-time user’s confusion, a different browser/device) that the author doesn’t experience, so it never surfaces during development. “Works on my machine” is literally true and therefore misleading: the machine is unrepresentative, so a passing local run gives false confidence while real users hit the bug.
2. Perceivable, Operable, Understandable, Robust — the four WCAG principles. A scanner catches only machine-detectable violations (missing alt, unlabeled input, contrast below 4.5:1) — commonly a third to a half of issues — and cannot judge meaning. Example: it can flag a missing alt, but not that alt="chart" on a revenue chart is useless, or that a technically valid focus order is illogical to navigate.
3. A functional test is a binary, automatable assertion: given INPUT, OUTPUT == EXPECTED. Usability testing is qualitative and human-in-the-loop: you give a real person a task and observe where they hesitate, err, or give up. “Confusing” is a graded human experience, not a boolean about output, so there’s no expected value to compare against — you can measure proxies (task completion, time-on-task) but the finding comes from watching a human, not from an assertion.
4. It’s wrong because the matrix (browsers × OSes × versions × screen sizes) is effectively infinite, impossible to test fully, and mostly made of cells no real user occupies — so you’d test everything badly and still miss the cases that matter. Instead, pull your own analytics, rank environments by real traffic, and pick the smallest representative support set that covers the large majority of sessions; test that thoroughly and explicitly declare weaker support below your threshold.
5. Examples: Accessibility — keyboard-only navigation of every control (human) or a contrast/alt scan (machine); both. Usability — moderated think-aloud task observation with ~5 real users (human). Compatibility — running the end-to-end flow on Safari-on-real-iPhone via a device farm (machine, parameterized across the support set), with a final look on a physical device (human).