Tabpath Research · 18 September 2026 · Method
Our keyboard pass reported 31 controls with no accessible name. An independent second pass confirmed two. The other 29 were controls that read perfectly well.
Automated accessibility tools have two failure modes and they point in opposite directions. They over-report the things they check, and they do not check a large part of what actually fails. We can put numbers on both, because we built the second pass before we shipped anything from the first.
Our first pass walks a page by keyboard, records every tab stop, and asks two questions at each one. Does this control have an accessible name? And does anything visible happen when it receives focus? Across two storefronts it produced 31 findings of the first kind and 16 of the second.
The second pass re-opens the live page and answers both questions again without reading the first pass's conclusions. For names it uses a fuller algorithm — one that descends into a nested <img alt> and into <title> inside an <svg> — and then cross-checks every surviving finding against axe-core's own name rules. Those rules are an independent oracle: if axe says a control has a name and we said it did not, we are wrong, and the finding is dropped. For focus indicators it re-photographs each flagged control with a much more generous frame, because an indicator drawn outside the first pass's window would have been recorded as absent.
Anything the two passes disagreed about was dropped rather than averaged. This is what was left.
6.5% survived. Both survivors are independently corroborated by axe-core.
81.3% survived. No automated rule tests this criterion, so none of these appear in any scan report.
Every one of the dropped findings was a control that reads perfectly well to a screen reader. The name was carried by alt text on an image inside the link, or by the title element inside an SVG icon. A check that reads an element's text content and stops there never sees either — it finds an empty string and reports a failure.
That is not a broken tool. It is a plausible one, and that is the problem. It is the shape of a great deal of low-cost scanning output, and the shape of a great deal of the evidence attached to accessibility demand letters. In January 2025 the Federal Trade Commission ordered an automated-remediation vendor to pay $1 million over deceptive claims that its AI product could make websites compliant with accessibility guidelines. Over-reporting is not a hypothetical failure mode in this field. It is the one the regulator has already acted on.
The second bar is the mirror image. Focus visibility — WCAG 2.4.7, Level AA — has no automated rule in any scanner we are aware of, because deciding whether an indicator is visible means looking at rendered pixels rather than reading CSS. We photograph each control while it holds focus and again one Tab later, and compare the two frames. Thirteen real failures across two ordinary storefronts, none of which would appear in any report those merchants could run for themselves.
| Storefront | Finding | Criterion | Produced | Confirmed |
|---|---|---|---|---|
| A — firearms storage, Shopify | No accessible name | 4.1.2 (A) | 7 | 0 |
| A | No visible focus indicator — six product "Quick View" buttons | 2.4.7 (AA) | 7 | 6 |
| B — sports nutrition, Shopify | No accessible name — two links | 4.1.2 (A) | 24 | 2 |
| B | No visible focus indicator — six 7×7px buttons and one link | 2.4.7 (AA) | 9 | 7 |
Both storefronts passed reflow at 320px with no horizontal overflow, both expose a skip link, and neither showed a focus-order inversion. A third storefront in the same batch returned no failures on these criteria — but only three controls were ever reachable behind its promotional modal, so that is a statement about what we could test, not about the site.
On that third storefront the first pass reported sixty consecutive tab stops without ever reaching the page. Left alone that goes out as a WCAG 2.1.2 keyboard trap — Level A, on a home page, about the most serious thing you can find.
It was wrong. The stops were cycling through three controls inside a promotional modal, and one of them was a button labelled "Dismiss this popup". A modal that holds focus is correct behaviour, and one offering a keyboard-reachable dismiss satisfies 2.1.2. Our test had pressed Tab and it had pressed Escape. It had never pressed Enter on the dismiss button. It now activates the control before drawing any conclusion, and the result is recorded as a pass.
This is our third published correction. Two of them went out before we caught them: a contrast census inflated by elements measured mid-animation, and a set of remediation figures we had estimated and then presented as measured. This one never shipped, because the second pass existed before the first pass had anything to say. That is the only difference between the three, and it is the whole argument for building the second pass first.
If you commission accessibility work — or if you are holding a scan report someone has attached to a demand letter — the question worth asking is not how many issues were found. It is what was done to try to prove them wrong, and what got thrown away when it worked.