Most accessibility testing happens on one version of a site: the one the team builds and looks at every day, usually in English and usually from the office. But a site with a dozen markets is really a dozen sites. Translations change text length and wrap, local teams add their own banners and forms, different components appear in different countries, and the version a visitor in Berlin or Tokyo gets may never have been tested at all.
We wanted to know how much that matters in practice. So we took 23 large multi-market websites, loaded their US, German, French and Japanese versions from inside each of those countries, and ran the same automated accessibility audit on all of them. This guide shares what we found, the code we used, and how to build market-aware testing into your own process.
Key takeaways
- The version you test is not the version everyone gets. Five of the 20 sites with stable results had at least one type of accessibility failure in a localized version that never appeared on their US version.
- The differences were concrete: unlabelled birthday fields on a sign-up form, a search box without a label, images without text alternatives, unnamed buttons, and text that failed contrast.
- Pages change between visits, so measure noise first. Three of 23 sites showed different results on two loads of the same US page minutes apart.
- Automated rules catch only part of the problem. Treat them as a market-by-market regression net, and keep manual testing with assistive technology.
- In the EU, the European Accessibility Act has applied since 28 June 2025 to many consumer services, including e-commerce, which makes every market version part of the compliance surface.
Why one version is not enough
Several things change between markets that can affect accessibility:
- Text. German words are long, Japanese text uses different fonts and line breaking, and translated labels can be lost, duplicated or left empty.
- Components. Consent banners, cookie walls, age gates, currency and country selectors, local payment widgets and regional promotions often appear only in some markets.
- Content. Local marketing teams publish their own images and campaigns, sometimes outside the main design system, and alternative text is the first thing to go missing.
- Delivery. Some sites serve a different build, domain or content management instance per region.
What those versions look like depends on where the visitor is. A test that loads the German page from a US office may get redirected, see a different consent flow, or see the US variant of a regional component. To test what users in Germany get, the test has to run as a visitor in Germany.
How we tested
We picked large websites that publish US English, German, French and Japanese versions of their homepage, taking the URLs from each site’s own hreflang annotations. On 4 October 2026 we loaded each version in Chromium through a Shifter exit in the matching country, with the browser’s language and time zone set to match: New York for the US, Berlin, Paris and Tokyo. We waited five seconds for client-side rendering and consent banners, then ran axe-core 4.13 with its WCAG 2 Level A and AA rules.
Two of the 25 sites we started with returned a block page in every market and were excluded, which left 23. To measure noise, we audited every US page twice. Three of the 23 sites gave different results between those two US runs, typically because of rotating banners or content loaded at random, so we excluded those three when comparing markets.
What we found
| Measure | US, first run | US, second run | Germany | France | Japan |
|---|---|---|---|---|---|
| Violation instances across 23 sites | 158 | 165 | 174 | 168 | 161 |
| Sites with no detected violations | 6 | 5 | 3 | 3 | 5 |
| Average types of violation per page | 1.6 | 1.7 | 1.8 | 1.8 | 1.7 |
The localized versions did slightly worse on every measure, but the gaps at this level are close to the run-to-run noise. The clearer signal is in which problems appeared where. Of the 20 sites with stable US results, five had at least one type of failure in a localized version that appeared in neither US run:
| Site type | Failure found only outside the US | Markets |
|---|---|---|
| Gaming platform | Sign-up birthday fields (day, month, year) with no accessible name | Germany, France, Japan |
| Website builder | Main search field with no label | Germany, France, Japan |
| Security software vendor | Currency selector with an invalid ARIA attribute; a promotional image with no text alternative; an unnamed video play button | Germany, France, Japan |
| E-commerce platform | A button with no accessible name | Germany, France |
| Open-source project site | Footer text that failed colour contrast | Germany, France, Japan |
Several of these matter more than the counts suggest. A screen reader user cannot complete a sign-up form whose date fields have no names, and cannot use a search box that is announced only as “edit text”. They are the kind of failure that stays invisible to a team that only ever tests the US page.
Across all 92 audits, the most common failure was colour contrast, found on 43 pages, followed by buttons without accessible names and images without text alternatives.
The code
The module below audits one page as a visitor in one market would see it, and flags block or error pages so they are never mistaken for the page you meant to test. It needs Playwright and axe-core.
import { readFileSync } from 'node:fs';
import { createRequire } from 'node:module';
const require = createRequire(import.meta.url);
const AXE_SOURCE = readFileSync(require.resolve('axe-core/axe.min.js'), 'utf8');
// Audit one page as a visitor in one market would see it: the browser's locale and time zone match the
// market, and the browser itself runs through an exit in that country.
export async function auditPage(browser, url, { locale, timezoneId }) {
const context = await browser.newContext({ locale, timezoneId });
const page = await context.newPage();
try {
const response = await page.goto(url, { waitUntil: 'domcontentloaded', timeout: 60000 });
await page.waitForTimeout(5000); // let client-side rendering and consent banners appear
await page.evaluate(AXE_SOURCE); // evaluate, not a script tag, so the page's CSP cannot block it
const result = await page.evaluate(() =>
window.axe.run(document, { runOnly: ['wcag2a', 'wcag2aa'], resultTypes: ['violations'] }));
const lang = await page.evaluate(() => document.documentElement.getAttribute('lang'));
const status = response ? response.status() : null;
return {
url,
finalUrl: page.url(),
status,
blocked: status === null || status >= 400, // a block or error page, not the page you meant to audit
title: await page.title(),
lang,
violations: result.violations.map((v) => ({ id: v.id, impact: v.impact, nodes: v.nodes.length })),
};
} finally {
await context.close();
}
}And running it for one market, with the browser routed through an exit in that country:
import { chromium } from 'playwright';
import { auditPage } from './audit.mjs';
// One browser per market, running through an exit in that country.
const browser = await chromium.launch({
proxy: {
server: 'http://p.shifter.io:443',
username: `${process.env.SHIFTER_PROXY_USER}-country-de`,
password: process.env.SHIFTER_PROXY_PASS,
},
});
const result = await auditPage(browser, 'https://shifter.io/de', { locale: 'de-DE', timezoneId: 'Europe/Berlin' });
console.log(result.status, result.blocked, result.lang, result.violations);
await browser.close();Run against our own German homepage, it reported no violations.
Two lessons from building it are baked in. The blocked flag exists because our first run audited block pages: two sites refused every exit, and their block pages produced convincing-looking results, including a page declared as US English that we briefly took for a mislabelled German one. And axe is injected with evaluate rather than a script tag, because strict Content Security Policies on some sites refuse injected scripts.
Building it into your process
- Audit every market you sell in, from that market. Use an exit in each country and match language and time zone, as described in matching proxy geo, timezone and locale, so the audit sees the same consent flows, redirects and regional components as local visitors.
- Find the versions from the site itself. Hreflang annotations list every market version, so the audit list stays complete as markets are added.
- Measure noise before trusting differences. Audit the same page twice and treat changes that also appear between identical runs as noise.
- Check that you audited the right page. Record the status, title and final URL, and discard block pages, error pages and redirects to another market. Our anti-bot stack lookup explains why a status code alone can mislead.
- Diff against the primary version. Report failures that exist in a market version but not in the version the team tests, since those are the ones nobody has seen.
- Run it on a schedule. Local campaigns and banners change weekly; change detection principles apply to accessibility too.
- Keep humans in the loop. The W3C is explicit that tools cannot check every accessibility requirement and that human judgement is required. Automated audits find regressions; they do not prove a site is accessible.
Why it matters now
Accessibility has long been a legal requirement for public sector websites in many countries. In the EU, the European Accessibility Act has applied since 28 June 2025 to a defined set of consumer products and services, including e-commerce, and the harmonised European standard used to meet it builds on WCAG. For a company selling across Europe, each country’s version of its site is part of what has to meet those requirements, not just the one its developers look at. This is general information, not legal advice; check the rules that apply in each of your markets with counsel.
The bottom line
Accessibility is usually tested in one place and shipped to many. In our audit of 23 multi-market sites, a quarter of those with stable results had failures that existed only outside the US, from unlabelled sign-up fields to images without text alternatives.
Test each market version as a local visitor would see it, separate real differences from noise, and diff against the version your team already checks. It costs a few minutes of automation per market, and it finds the problems your users there have been running into all along.
Sources and references
- Deque, axe-core, version 4.13, used for the audits.
- W3C Web Accessibility Initiative, Selecting web accessibility evaluation tools.
- Noerr, Accessibility in e-commerce: new obligations for online shop operators starting from June 2025.
- Audits run by Shifter on 4 October 2026 using the code above.