Tabpath field measurement · 31 August 2026 · second edition
Fifty storefronts run an accessibility overlay. We confirmed each widget was running, gave it fifteen seconds, and measured what was left.
Every instance count in the first publication of this edition was too high, and the method statement below did not describe the run that produced them. We stated that axe was run with the five WCAG tags. It was actually run with those plus Deque's best-practice ruleset, and the combined total was then reported as WCAG failures. region, heading-order and landmark-unique are Deque house recommendations, not WCAG requirements.
Third correction, 17 September 2026: that sentence below — “every figure on this page has been rebuilt” — was itself not quite true. One survived: the median comparison in the limitations list still carried best-practice rules until today. It read 22 with an overlay against 17 without; WCAG-only it is 15 against 10. We found it by having the whole page re-derived from the raw scans by a reviewer whose only job was to break it, which is now how anything gets published here. The same pass corrected fifteen smaller things, most of them sentences rather than numbers: every table cell, every total and the +143 reconciled exactly.
Every figure on this page has been rebuilt from WCAG 2.0/2.1/2.2 A and AA rules only. What changed: the 15-second total falls from 1,819 to 1,353; the short-settle total from 1,676 to 1,210; and one store — relieffactor.com — does reach zero WCAG failures, so our earlier “not one of the fifty was clean” was wrong. The count of stores unchanged by the longer settle rises from 30 to 34.
What did not change is the finding. The net effect of giving every overlay 12.5 extra seconds of settle is +143 instances either way — identical under both rule sets. Removing the best-practice rules moved the totals and left the conclusion exactly where it was.
An accessibility overlay is a script a store adds to its site, typically for $490 to $3,500 a year, that promises WCAG conformance without touching the underlying code. This is a measurement of what a visitor actually receives.
The first edition of this study covered ten stores. This one covers fifty, drawn from a census of 313 storefronts on the public client rosters of fifteen e-commerce agencies. Scaling the sample corrected one of our own headline numbers and confirmed the other.
Two methodological problems have to be solved before any of this means anything, and most published claims about overlays solve neither.
Overlay vendors inject their remediation asynchronously, after their script boots. Audit two seconds after load and you have measured the page before the product you are paying for has run — which proves nothing except impatience. So every page here was measured twice, at 2.5 seconds and at 15 seconds.
A script tag is not a running product. It can be a lapsed subscription, a conditional load, or a leftover include. So each site was checked for the vendor's rendered DOM — [data-acsb], #userwayAccessibilityIcon, [id^="ae_"], #INDmenu-btn — before any claim was made about it.
That check removed seven sites. See below; it is a finding in itself.
Fifty storefronts with a confirmed-rendering overlay widget, measured at both settle windows. “Clean” counts stores reaching zero machine-detectable failures.
| Vendor | Stores | At 2.5 s | At 15 s | Δ | Critical | Serious | Clean | Got worse |
|---|---|---|---|---|---|---|---|---|
| accessiBe | 31 | 752 | 786 | +34 | 188 | 594 | 1 | 8 |
| UserWay | 14 | 323 | 325 | +2 | 145 | 177 | 0 | 2 |
| AudioEye | 3 | 100 | 100 | 0 | 74 | 26 | 0 | 0 |
| More than one vendor | 2 | 35 | 142 | +107 | 3 | 139 | 0 | 1 |
| Total | 50 | 1,210 | 1,353 | +143 | 410 | 936 | 1 | 11 |
That last figure is the one to sit with. Across fifty storefronts and four vendors, the best result anyone achieved in fifteen seconds was removing five failing instances from happyhead.com, which still had fifteen left. Thirty-four stores finished on the same failing-instance count, and twenty-eight of those were unchanged rule for rule — on hukitchen.com a different rule failed at the same total. Critical-severity instances moved from 411 to 410; serious rose from 792 to 936.
The group’s +107 is one store regressing by 112 against the other improving by 5. The 112 came from insideweather.com, running both accessiBe and UserWay, where lazy-loaded content arrived during the longer window. That is content appearing late, not overlays causing damage — but it is worth noting that two overlays did not help it either.
Two stores render markers for more than one vendor and are grouped together above. One of them is a detection limitation rather than a real double-install: #INDmenu-btn is the trigger id for EqualWeb and also appears in accessiBe's rendered DOM, so our verifier cannot separate them from outside. We would rather group those two than assign them to a vendor on a guess.
Twelve of the fifty stores have no skip-to-content link among the first five in-page anchors, judged by link text. That figure is a keyboard probe rather than an axe rule, so it is unaffected by the ruleset correction above — but the probe is narrow, and a skip link placed later in the source or labelled only by aria-label would be recorded as absent.
Of 57 storefronts whose pages referenced an overlay vendor's script, seven rendered no widget at all after fifteen seconds — no trigger, no button, no vendor DOM. Whether those are lapsed subscriptions, misconfigurations, or conditional loads, we cannot say from outside.
We excluded all seven from every efficacy figure on this page, because it would be dishonest to blame a product for not working when the product was not running. But it carries a warning for anyone else measuring this: 12% of the sites carrying a vendor script render no widget at all, which makes the script-tag prevalence figure 14% higher than the real one. Prevalence across the 294 valid sites is 19.4% by script tag, and 17.0% by rendered widget.
Withdrawn pending re-measurement. We previously said 190 failing instances across 32 of the 50 stores sit inside the overlay's own injected markup, and noted that almost all were region failures. That is the problem: region is a Deque best-practice rule, not a WCAG requirement, so almost all of that 190 was never a WCAG failure. We have not yet re-measured how many WCAG failures the widgets' own markup accounts for, and we would rather publish nothing here than replace one unverified number with another. The observation below is a specific, checkable defect and stands on its own.
One store's accessiBe trigger is an <li role="button"> wrapping another interactive element — nested-interactive, a control assistive technology cannot reliably operate. That is the button a screen-reader user is meant to press to switch the accessibility features on.
Every limitation below cuts against the finding. We would rather publish them than have someone else discover them.
Load the page in Chromium at 1366×900. Wait 15 seconds after domcontentloaded. Then walk the whole page — scroll in steps of 0.8 viewport-heights, dwelling 350ms at each, back to the top, and wait about four seconds more — so every scroll-triggered reveal has finished. Skipping that walk is not a minor deviation: it is worth a 20x difference in reported contrast failures on an animated theme, which we found the hard way and wrote up at Phantom Contrast. Only then run axe-core 4.13 with tags wcag2a, wcag2aa, wcag21a, wcag21aa, wcag22aa — and nothing else.
That last clause is the one we got wrong. Adding best-practice to the tag list is a single word in a config array and it silently inflated every total on this page by roughly a third, because region fires on almost every site — 1,926 instances across 223 of the 294, and on three sites well over a hundred each. If you are reproducing this, print your tag list next to your total. A number is only as good as the ruleset nobody checked.
Before quoting a number, do the two things that make it defensible. Count the vendor's rendered nodes to prove the widget is live. Then check whether each failing node carries the vendor's own attributes — conflating the store's defects with the widget's overstates the case.
And discard any page that is not the page: status ≥ 400, a bot-challenge title or challenge DOM, or a body with no interactive elements. Nineteen of our 313 were error pages, interstitials or unreachable. Counting those as merchant defects is the easiest way to publish something false.
Tabpath fixes accessibility defects in live e-commerce storefronts. We built the scanner to find work, and this was the first thing it found that we did not expect: a sixth of the stores we looked at were already running a product sold to solve the problem, and the problem was still there fifteen seconds after load, with the vendor's own button sitting on the page.
If you sell or install overlays, we will publish a correction on any figure here you can show us is wrong. We have now corrected two of our own — a prevalence figure in the first edition, and every instance count in this one. Both corrections made our case weaker and are on the page rather than in a changelog.