Integrating @axe-core/playwright Into an Existing Project

A suite that already exists is an asset, and the cheapest accessibility coverage available is the states it already reaches. This guide is part of Playwright Accessibility Plugin Integration, and it works from zero on a repository that has functional Playwright specs today: what to install without disturbing the pinned versions, how to add a scan at the exact point a spec has driven the UI into an interesting state, how to give the gate its own project so it can run independently of the functional run, and how to make the first run record the violations that already exist instead of turning every pull request red.

Root Cause

The instinctive first move is a new tests/a11y/ directory with a loop over routes, and as a breadth measure that is correct. As the only measure it wastes most of the money. Every scan in a route-loop pays for a fresh browser context, a sign-in, a cold navigation and a hydration wait before it evaluates anything, and what it evaluates is always a page’s initial state. The states that actually fail WCAG are the ones a user creates: an address form showing three validation errors, a results table sorted descending with an empty result set, a modal stacked over an open drawer, a quantity stepper at its maximum with a disabled increment button, a toast that appeared while a dialog was open. The existing suite has already paid to reach all of them, and adding one line at the end of each of those specs buys a scan of a state a route loop can never see.

The second failure mode is drift. A parallel accessibility suite duplicates route constants, selectors, sign-in helpers and test data. Nothing keeps the two copies in step, so when the functional suite’s authentication helper changes six months later, the accessibility suite quietly starts scanning the login page for every route and reports zero violations on twenty pages it never loaded. A retrofit that appends an assertion inside specs that already exist has no second copy to drift from, which is worth more over two years than any amount of harness design.

The third problem is arithmetic rather than architecture. An application that has never been scanned has violations — on a twenty-route matrix a mid-sized product typically returns somewhere between 40 and 200 nodes flagged at serious or critical on the very first run, most of them a handful of rules repeating across a shared header, a component library button and a date picker. Making that a required check on day one blocks every pull request for reasons unrelated to the change under review, and the check is disabled or marked non-required within the week. The first run has to measure, the second week has to hold the line, and only then does a ratchet make sense.

Scanning in place versus re-navigating for every scan The top row shows an existing checkout spec whose steps are sign in, cart with two items, address with a validation error, payment and confirm, with scans attached to three of those steps costing under a second each. The bottom row shows a parallel suite that creates a fresh context, signs in, navigates and scans, looping three times for forty-two seconds. Retrofit: scan where the spec already stands sign in cart 2 items address 3 errors shown payment confirm scan +0.9 s scan +0.9 s scan +0.9 s Parallel suite: pay for the journey again, per scan fresh context sign in again goto the route scan +0.9 s repeat per route: three scans cost 42 s, not 2.7 s A route loop also only ever sees initial states, never the error and stacked-layer states above.
The three most interesting states on this journey are free to scan inside the spec that produced them and expensive to reproduce anywhere else.

Configuration

Step one: install without disturbing what is pinned. The builder needs nothing but a Page object, so it does not force a Playwright upgrade. Check what the repository already resolves before changing anything.

npm ls @playwright/test          # the version the suite already pins
npm install --save-dev @axe-core/playwright
npm ls axe-core                  # must resolve to exactly one version

Do not add axe-core to package.json yourself; the builder depends on it, and a second pinned copy makes the injected version depend on hoisting order, which moves the baseline whenever the lockfile is regenerated. If the suite is on a Playwright release older than 1.37, the blob reporter and merge-reports are unavailable — that is a reason to plan an upgrade later, not a blocker for this retrofit, because testInfo.attach has been available far longer.

Step two: add a helper function, not a fixture. A fixture is the better long-term shape, but it requires changing import { test } from '@playwright/test' in every spec that wants a scan, and a pull request that touches forty spec files will not be reviewed carefully. A plain async helper can be dropped into any existing spec with one import line and no change to how test is obtained. Critically, it returns the blocking violations rather than asserting on them, which is what makes baseline mode possible later.

// tests/support/a11y.ts
import AxeBuilder from '@axe-core/playwright';
import type { Page, TestInfo } from '@playwright/test';
import type { Result } from 'axe-core';

const TAGS = ['wcag2a', 'wcag2aa', 'wcag21a', 'wcag21aa', 'wcag22aa'];
const BLOCKING = new Set(['critical', 'serious']);

// Returns the blocking violations. Deliberately asserts nothing.
export async function scanHere(
  page: Page,
  testInfo: TestInfo,
  label: string,
): Promise<Result[]> {
  const results = await new AxeBuilder({ page }).withTags(TAGS).analyze();

  await testInfo.attach(`axe-${label}.json`, {
    body: JSON.stringify(results, null, 2),
    contentType: 'application/json',
  });

  return results.violations.filter((v) => BLOCKING.has(v.impact ?? ''));
}

Step three: call it where the spec already stands. Below is a real functional spec with three lines added. The scan runs after the assertion that proves the error state exists, which means the DOM is guaranteed to be in the state the label claims — no extra wait, no extra navigation, no duplicated selector.

// tests/checkout.spec.ts — an existing spec, three lines added
import { test, expect } from '@playwright/test';
import { scanHere, recordOrGate } from './support/a11y';

test('checkout rejects an incomplete address', async ({ page }, testInfo) => {
  await page.goto('/cart');
  await page.getByRole('button', { name: 'Checkout' }).click();
  await page.getByLabel('Postcode').fill('');
  await page.getByRole('button', { name: 'Continue' }).click();
  await expect(page.getByRole('alert')).toContainText('Postcode is required');

  // The state is already exactly what we want to scan: three field errors.
  await recordOrGate(await scanHere(page, testInfo, 'checkout-address-errors'));
});

Step four: give the gate its own project. Two things need separating. The route matrix belongs in its own project so the accessibility gate can be run, sharded and timed independently of the functional suite. The scans embedded in functional specs cannot move — they belong to functional tests — so they get an environment switch instead, letting a functional-only run skip them entirely.

// playwright.config.ts — projects added around an existing config
import { defineConfig, devices } from '@playwright/test';

const STATE = 'playwright/.auth/user.json'; // written by the setup project

export default defineConfig({
  fullyParallel: true,
  retries: process.env.CI ? 1 : 0,
  use: { baseURL: process.env.BASE_URL ?? 'http://127.0.0.1:4173' },
  projects: [
    { name: 'setup', testMatch: /auth\.setup\.ts/ },
    {
      name: 'e2e',
      dependencies: ['setup'],
      testDir: 'tests',
      testIgnore: /a11y\//, // the matrix belongs to the other project
      use: { ...devices['Desktop Chrome'], storageState: STATE },
    },
    {
      name: 'a11y',
      dependencies: ['setup'],
      testDir: 'tests/a11y',
      timeout: 90_000, // a scan legitimately outlasts a click
      use: { ...devices['Desktop Chrome'], storageState: STATE },
    },
  ],
});

The gate now runs on its own with npx playwright test --project=a11y, and the functional suite runs unchanged with --project=e2e. Both reuse the same signed-in storageState, so nothing is duplicated.

Two projects sharing one authentication setup A setup project produces a stored authentication state. Both the e2e project and the a11y project declare it as a dependency, and each can be invoked separately with its own project flag on the command line. One sign-in, two independently runnable projects setup project writes storageState e2e project specs with inline scans a11y project tests/a11y route matrix --project=e2e functional job --project=a11y the gate, shardable declared as a dependency Inline scans stay inside functional specs; only the route matrix moves into the gated project.
Separating the projects lets the accessibility gate be timed, sharded and made a required check without touching the functional job.

Step five: make the first run record instead of fail. Counts are written per test to their own file so parallel workers never race on a shared JSON, and a small script merges them into the committed baseline. In gate mode the assertion is “no worse than the recorded number for this label”.

// tests/support/a11y.ts (continued)
import { mkdir, writeFile, readFile } from 'node:fs/promises';
import { join } from 'node:path';
import { expect, test } from '@playwright/test';

const COUNT_DIR = 'test-results/a11y-counts';

export async function recordOrGate(blocking: Result[]) {
  const info = test.info();
  const label = info.title.replace(/[^a-z0-9]+/gi, '-').toLowerCase();
  await mkdir(COUNT_DIR, { recursive: true });

  // One file per test per attempt: workers never write the same path.
  const file = join(COUNT_DIR, `${label}.${info.retry}.json`);
  const ids = blocking.map((v) => `${v.id}:${v.nodes.length}`).sort();
  await writeFile(file, JSON.stringify({ label, count: blocking.length, ids }));

  if (process.env.A11Y_MODE === 'record') return; // measuring week: never fail

  const baseline = JSON.parse(await readFile('a11y-baseline.json', 'utf8'));
  const allowed = baseline[label];
  // An unknown label is new coverage, not a regression: record it next run.
  if (allowed === undefined) return;
  expect(blocking.length, `${label} ids: ${ids.join(', ')}`).toBeLessThanOrEqual(allowed);
}
// scripts/write-a11y-baseline.mjs — run once after A11Y_MODE=record
import { readdir, readFile, writeFile } from 'node:fs/promises';

const dir = 'test-results/a11y-counts';
const files = (await readdir(dir)).filter((f) => f.endsWith('.json'));
const baseline = {};

for (const f of files.sort()) {
  const { label, count } = JSON.parse(await readFile(`${dir}/${f}`, 'utf8'));
  // Files are sorted, so the highest retry index for a label wins.
  baseline[label] = count;
}

const sorted = Object.fromEntries(Object.entries(baseline).sort());
await writeFile('a11y-baseline.json', JSON.stringify(sorted, null, 2) + '\n');
console.log(`recorded ${Object.keys(sorted).length} labels`);

Validation

Run the measuring pass first, then commit the file it produces. Nothing should fail, and the output tells the team the real size of the problem for the first time.

A11Y_MODE=record npx playwright test --project=a11y --project=e2e
node scripts/write-a11y-baseline.mjs
cat a11y-baseline.json
# {
#   "checkout-address-errors": 6,
#   "checkout-rejects-an-incomplete-address": 6,
#   "scan-account-orders": 11,
#   "scan-search-q-shoes": 9,
#   "scan-home": 4
# }

Now prove the gate bites. Delete an alt attribute from a component that renders on the search route, drop the recorded number by one, or lower a single count in the baseline by hand, then run without the record flag:

npx playwright test --project=a11y
# 1) scan /search?q=shoes ──────────────────────────────────────────
#    Error: scan-search-q-shoes ids: color-contrast:3, image-alt:1, label:6
#    expect(received).toBeLessThanOrEqual(expected)
#    Expected: <= 9
#    Received: 10

The failure message names every rule and its node count, and the attached JSON in the HTML report carries the selectors. Confirm the two projects are genuinely independent by running the functional suite alone — npx playwright test --project=e2e — and checking that it still passes with the accessibility baseline file deleted, which proves the inline scans are not silently gating the functional job during the measuring period.

Edge Cases and Conditional Guards

  • A spec that fails functionally never reaches its scan. The recordOrGate call sits after the assertions, so a broken checkout means no count file and a missing baseline key. Treat a missing key as new coverage to be recorded next run, never as zero, or the first flaky functional failure permanently sets a route’s allowance to nothing and the next green run fails on it.
  • Sharded runs split the count directory. Each shard writes only its own labels, so writing the baseline from one shard truncates it to a quarter of the suite. Download every shard’s test-results/a11y-counts into one directory before running the merge script, and never regenerate the baseline from a partial matrix.
  • Retries overwrite a count. The attempt index is part of the filename precisely so attempt zero and attempt one are both preserved, and the merge script’s sort makes the last attempt win. If the two attempts disagree by more than a node or two, that is an application race worth investigating rather than a number to record.

Pipeline Impact

During the measuring period the accessibility job runs with A11Y_MODE=record and is not a required check; it exists to publish a number to the run summary. The inline scans inside functional specs run in the same mode, so the functional job’s pass rate is untouched — an important political detail, because a retrofit that destabilises the existing suite on its first day will be reverted regardless of its merit. After one or two weeks the baseline is committed, A11Y_MODE is unset, and the a11y project becomes a required status check in branch protection while the inline scans start holding their own labels.

From there the numbers only go down, which is a policy question rather than a tooling one: setting up progressive accessibility thresholds in CI covers how to express the ceiling, and ratcheting violation budgets down each sprint covers the cadence. Two runtime costs are worth stating up front. The inline scans add roughly 0.9 seconds per call to specs that already exist, so twelve scans across the functional suite cost about eleven seconds spread over however many workers are running. The a11y project’s route matrix is a new job, and its cost is dominated by browser launch and navigation rather than by the scan itself, which is why it should be sharded from the start rather than after it becomes slow.

Baseline first, then a ceiling that steps down Bars show measured blocking violations at weeks zero, two, four, eight and twelve: one hundred thirty-seven, one hundred twenty-nine, one hundred four, fifty-eight and twelve. A stepped amber ceiling sits above them at one hundred thirty-seven, one hundred twenty, seventy and twenty, with week zero recorded in warning mode only. Measured violations against the committed ceiling measured ceiling 0 50 100 137 129 104 58 12 week 0 record only week 2 gate on week 4 week 8 week 12 Week zero fails nothing; the ceiling only ever moves down, and only when a run has already cleared it.
The ceiling is committed from a measured run and stepped down behind the real numbers, so no pull request is ever blocked by debt it did not create.

Common Pitfalls

  • Writing the baseline from a single shard, which silently drops three quarters of the labels and makes the gate meaningless for every route it forgot.
  • Deriving the label from the route instead of the test title, so two specs that visit the same route overwrite each other’s count and one of them is never gated.
  • Turning the gate on in the same pull request that introduces the scans, which guarantees the first reviewer sees a red build caused by pre-existing debt.
  • Letting the inline scans assert during the measuring period, so a functional spec fails for an accessibility reason before the team has agreed the gate exists.
  • Adding axe-core as a direct dependency at a pinned version, which changes the injected engine and moves every recorded count without any application change.
  • Committing a11y-baseline.json without a code owner, so it becomes the file people edit upward to make a build pass instead of fixing the violation.

FAQ

Should the baseline record counts or the actual violation identities? Record counts for the gate and identities for the report. A count comparison is stable and cheap: it fails when the number goes up and it does not churn when a component moves in the DOM. Identity comparison is stricter — it catches “one violation fixed, one new one introduced” — but it needs a normalised node key that survives refactoring, and every layout change produces a diff that has to be reviewed by hand. The helper above writes the sorted ids into the count file anyway, so the identity data is there for triage without being load-bearing for the gate.

How long should the measuring period last? Long enough for every label to have been recorded at least twice, which in practice means two weeks or about ten pipeline runs on an active repository. The purpose is not statistical rigour; it is to catch labels whose count varies between runs. Any label that moves between two record runs is either racing or genuinely nondeterministic, and it should be fixed or excluded before the gate turns on, because a flaky baseline is what teaches a team to ignore the check. Running the gate in warning mode first, as described in soak-testing a new accessibility gate in warning mode, is the same idea applied to the whole job.

Does the route matrix still earn its place once functional specs are scanning? Yes, for coverage that no functional spec provides. Functional tests concentrate on revenue-critical journeys, so a help centre, a legal page, a password-reset form and an empty-state dashboard typically have no spec at all. The matrix is the breadth layer and it is trivially cheap to extend — one array entry per route — while the inline scans are the depth layer. Keeping the tag list identical in both, ideally imported from the shared axe-core configuration and setup conventions, is what makes their counts comparable.

Back to Playwright Accessibility Plugin Integration