A scan report lands on your desk. Somewhere near the top is a number — say 312 violations, though the particular number is beside the point — and that number is doing the heavy lifting. Three hundred violations sounds like a site nobody maintained, a defendant who never cared, a case worth settling high.
Almost always, the number means less than it seems to. Nobody lied. The tool worked as designed. But automated testing measures something narrower than “violations,” and WCAG itself is built in a way most reports quietly ignore. Three things inflate the count. All three can be checked.
Not all of WCAG is a requirement
People talk about WCAG as one rulebook. The W3C sees it differently, and says so in the standard: part of the document is normative, and most of the material surrounding it is not.
The normative core is small. It consists of the success criteria — numbered, testable statements like 1.1.1 Non-text Content or 2.4.7 Focus Visible — plus the conformance requirements and the glossary definitions the criteria depend on. Meeting the success criteria is what conformance means. Nothing else counts toward it.
Everything the W3C publishes around that core — the Understanding documents, the Techniques — is informative. Here the W3C is blunt:
Techniques are informative — that means they are not required. The basis for determining conformance to WCAG 2.2 are the success criteria from the WCAG 2.2 standard — not the techniques.
It even warns readers against requiring them. Then comes the sentence that matters most, again straight from the W3C: content that does not implement a sufficient technique does not necessarily fail the success criterion. A criterion can be satisfied in ways nobody documented.
One exception. The W3C also publishes documented Failures, and those run the other way: match a documented Failure and the criterion is unsatisfied, full stop. So a finding can rest on two very different footings. “Did not use a recommended technique” proves nothing by itself. “Matches a documented failure condition” proves a great deal.
| Part of WCAG | Type | Testable on its own? |
|---|---|---|
| Principles (Perceivable, Operable, Understandable, Robust) | Normative | No — organizing categories only |
| Guidelines (13 total) | Normative | No — general goals under each principle |
| Success criteria (e.g. 1.1.1, 2.4.7) | Normative | Yes — this is what conformance actually means |
| Techniques (Sufficient/Advisory) | Informative | No — one way to meet a criterion, not the only way |
| Understanding documents | Informative | No — explains intent, adds no requirement |
| Documented Failures | Informative label, but functions as binding | Yes — a documented Failure means a criterion is unsatisfied |
Why does any of this matter for a scan report? Because rule engines live almost entirely in the informative layer. No engine can decide whether “the purpose of each link can be determined from the link text” — that takes a human reading the page. What an engine checks is a stand-in: is the link text on a list of known-vague strings, does the image carry an alt attribute, does the element have a particular ARIA role. Good stand-ins, mostly. Still stand-ins. A report that piles proxy failures and true criterion failures into one column labeled “violations” has merged two claims that are not the same claim.
So a finding backed only by an unimplemented technique, with nothing showing the criterion itself went unsatisfied, has not established nonconformance. Fix it anyway, perhaps. Count it as a violation, no.
One defect, counted many times
The second problem is plain arithmetic. Scanners crawl, and they report per page, per instance. For a developer that is exactly right — you want every location listed. Drop the same output into a legal document and it misleads.
Websites are built from shared parts. One navigation menu, one header, one cookie banner, one product-card template: written once, rendered everywhere. Give that one component a defect and a hundred-page crawl returns a hundred violations. One defect. One fix. A hundred findings gone at once.
Which is why the headline number tracks neither the size of the problem nor the cost of fixing it. Those 312 findings might boil down to eight distinct defects, six of them sitting in shared components. Eight is not 312, and the gap matters — for the remediation budget, and for how the plaintiff gets to characterize the site.
Count defects, not findings, and keep the instance tally alongside. Two findings are one defect when they share a root cause and a single correction clears both. Making that call requires looking at the site, not the report. It is not hard. It is routinely the single largest reduction on the table.
Some findings do not reproduce
The third problem is one the vendors themselves flag. Automated tools cannot weigh context, and the better ones admit it in their own documentation and interfaces.
Separate two cases. First: findings the tool refuses to decide. Axe labels them “incomplete” or “needs review”; WAVE calls them “alerts.” They are open questions addressed to a human. Roll them into a violation count and a question has been turned into an answer the vendor never gave.
Second: findings stated affirmatively that fall apart under testing. The “missing” alternative text on an image that turns out to be decorative and properly hidden from assistive technology. A contrast failure computed against a background the tool guessed wrong. An “unlabeled” form field labeled through a mechanism the tool did not follow. The rule fired correctly on what it could see. The conclusion is still wrong.
None of that condemns the tools, and accuracy about their reach cuts both ways. Deque — publisher of the axe engine behind much of the industry — reports that automated testing covered 57 percent of the accessibility issues found across some two thousand audits, a figure it offers against the lower numbers usually quoted, built by counting issues rather than success criteria. Automation finds plenty.
What automation never answers: how many distinct defects exist, which findings survive inspection, whether any criterion actually went unsatisfied. A scan was not built to answer those. The trouble starts when output meant to direct a developer's attention gets recycled as a measure of how far a legal violation extends.
What manual testing establishes
Testing the allegation list means taking each finding as alleged and working it on the live site — keyboard, screen reader, recorded environment, reproduction steps. Every alleged item lands in one of four places:
- It reproduces, and the normative text of a success criterion goes unsatisfied. A genuine violation, reported as one.
- It reproduces as a real barrier that no success criterion addresses. Flagged for fixing, reported separately as advisory — not a violation.
- It duplicates another item on the list: one defect, with an instance count.
- It does not hold up, and the write-up says exactly why — rule misapplied, context the tool could not weigh, condition it could not detect.
The output is a matrix covering everything alleged, written to be handed across the table. Nothing gets softened. Real barriers are called real barriers — and finding them early usually serves the defendant, since remediation costs less than litigation, and a documented, completed remediation with a policy preventing recurrence has supported mootness arguments in web accessibility cases.
The point is not a smaller number. The point is an examined one.
A note on the standard itself
One more thing, stated carefully, because it gets blurred from both directions.
Nothing in the ADA's text requires WCAG conformance of a private business, and as of this writing no Title III regulation adopts it. The Department of Justice did adopt WCAG 2.1 Level AA by rule in 2024 — but that rule reaches state and local government entities under Title II. 89 Fed. Reg. 31320 (Apr. 24, 2024), codified at 28 C.F.R. § 35.200. Courts have still treated WCAG 2.x Level AA as a workable benchmark again and again, and it appears routinely in settlements and consent decrees. For a private business, the statutory question stays what it always was: effective access.
That cuts in both directions technically, too. A conformance failure is evidence of an access problem, not proof — a criterion can go unsatisfied in a way no user ever encounters. And a real barrier can exist with no conformance failure attached, because WCAG does not reach everything; cognitive accessibility is the usual gap. An analysis that reports only one side of that is half an analysis.
What this is not
This is technical analysis, not legal analysis. Whether a given barrier denies access under the ADA, whether an alternative was reasonably achievable, what any of it is worth — those are questions for counsel. An examiner who answers them has stepped outside their competence, and an opposing party will say so.
The technical scope is narrow on purpose, and the narrowness is the defense: here is what the site does, here is the environment it was tested in, here are the steps to reproduce it, here is the requirement that is or is not satisfied. Anyone can repeat it and land in the same place.