# Playwright Accessibility Plugin Integration

A Playwright suite already drives the application into the states that matter — signed in, cart populated, dialog open, filter applied, error banner showing — which makes it the cheapest place in a pipeline to ask an accessibility question. This guide is part of Web Accessibility Testing Fundamentals & Tool Selection, and it treats @axe-core/playwright as the primary scanning harness for that suite: the AxeBuilder API surface, a project-wide fixture that hands every spec an identically configured builder, result attachments that make a failed scan readable straight from the HTML report, sharding a large route matrix across workers, and pointing the same specs at a deploy preview instead of localhost.

# Problem Statement

@axe-core/playwright is not a plugin in the sense that cypress-axe is a plugin. It exports one class, AxeBuilder, and that class has no global configuration hook, no defineConfig entry, and no lifecycle registration. Every new AxeBuilder({ page }) starts from axe-core’s shipped defaults. In a suite of thirty specs written by six people over eighteen months, this guarantees rule-set drift: one spec runs wcag2a only, another adds best-practice, a third disables color-contrast because it was noisy on one page in 2024 and nobody removed the line. Violation counts from two specs are then not comparable with each other, and a count from last quarter is not comparable with this quarter’s. The gate becomes a number nobody trusts.

The second problem is debuggability. analyze() resolves to an AxeResults object holding violations, passes, incomplete and inapplicable, each entry carrying rule metadata, an impact, a help URL and a nodes array with CSS targets, html snippets and per-check failure summaries. That object is the entire value of the scan, and the standard expect(results.violations).toEqual([]) pattern throws almost all of it away: the assertion prints a truncated diff of a deeply nested structure, and the HTML report shows a wall of serialised JSON with no indication of which node on which page failed. Engineers respond by re-running the scan locally, which is exactly the loop CI was supposed to remove.

The third is cost. Injecting axe-core into a page means evaluating roughly half a megabyte of JavaScript in the browser context, and a full-tree analyze() on a dense dashboard takes 400–900 ms on a CI runner before navigation and hydration waits are counted. A forty-route matrix is therefore a minute or more of pure scan time on a single worker, on top of the two or three minutes the navigation costs. That is affordable only if the run is parallel, and parallelism only helps if the report can be reassembled afterwards. The fourth problem is that localhost is the wrong target: purged CSS drops a focus ring, a minifier collapses a <label> wrapper, a CDN rewrite strips a lang attribute. Those failures exist only in a built artifact, so the gate has to be able to run against the preview URL the deploy just produced.

Key implementation targets:

• One fixture that yields a preconfigured AxeBuilder, so tag selection and rule exclusions live in a single reviewed file instead of thirty specs.
• A documented tag set that maps to the WCAG level the organisation actually claims, with best-practice reported but never blocking.
• Scoping by Playwright locator rather than by duplicated CSS strings, plus a policy for the page-level rules that scoping breaks.
• The full AxeResults JSON attached to every test result, alongside a one-line-per-violation digest that renders inline in the HTML report.
• A shard-aware CI invocation whose blob reports merge into one HTML report and one machine-readable JSON artifact.
• A baseURL that comes from the environment, so the identical specs run against a local preview server or a deployed preview build.

# Prerequisites

• Node.js 20.x or newer — current @axe-core/playwright releases target Node 18+, and Playwright itself has dropped Node 16.
• @playwright/test 1.44 or newer, so testInfo.attach() is available inside fixtures and merge-reports accepts blob directories.
• Chromium plus its OS libraries on the runner: npx playwright install --with-deps chromium.
• A playwright.config.ts with baseURL read from process.env, and a PREVIEW_URL secret or output available to the CI job.
• Permission for the job to upload artifacts and to write a check summary, if PR annotations are wanted.
• An agreed blocking impact set — this guide gates on critical and serious and reports the rest.
• A route list (or a sitemap parser) committed to the repository, because an undeclared route matrix silently shrinks.

# 1. Install the Harness and Build a Shared Fixture

Install the binding as a dev dependency. It declares axe-core as its own dependency, so do not add axe-core to package.json at a different version — two copies in the tree means the version the builder injects depends on hoisting order, and the violation count changes when an unrelated package bumps.

npm install --save-dev @axe-core/playwright
# Verify which axe-core actually got hoisted before you trust any baseline:
npm ls axe-core          # expect exactly one resolved version

The fixture must hand back a builder, not results. Returning results forces every scoping or rule decision into the fixture’s option bag, which grows a parameter per special case until it is unreadable. Returning a builder lets a spec keep chaining — .include(), .exclude(), .disableRules(), .options() — while the shared defaults stay applied.

// tests/a11y/fixtures.ts
import { test as base, expect } from '@playwright/test';
import AxeBuilder from '@axe-core/playwright'; // a named { AxeBuilder } export also exists
import type { AxeResults, Result } from 'axe-core';

// The only impacts that fail a job. Everything else is reported, never blocking.
export const BLOCKING = new Set(['critical', 'serious']);

// One reviewed tag set for the whole repository. Changing it is a pull request.
export const GATE_TAGS = ['wcag2a', 'wcag2aa', 'wcag21a', 'wcag21aa', 'wcag22aa'];

type A11yFixtures = {
  axe: () => AxeBuilder;
};

export const test = base.extend<A11yFixtures>({
  axe: async ({ page }, use) => {
    // A factory, not a single instance: a builder is single-use per analyze().
    await use(() =>
      new AxeBuilder({ page })
        .withTags(GATE_TAGS)
        // Component-level specs re-enable this; see section 3.
        .disableRules(['region']),
    );
  },
});

export function blockingOf(results: AxeResults): Result[] {
  return results.violations.filter((v) => BLOCKING.has(v.impact ?? ''));
}

export { expect };

A spec now imports test from the fixture module instead of from @playwright/test, and the accessibility assertion is two lines longer than a functional one. The page.getByRole('main').waitFor() call is doing real work: it holds the scan until the framework has committed a render, which removes the largest single source of nondeterminism in a scan suite.

// tests/a11y/routes.spec.ts
import { test, expect, blockingOf } from './fixtures';

const ROUTES = ['/', '/search?q=shoes', '/product/ba-1180', '/account/orders'];

for (const route of ROUTES) {
  test(`scan ${route}`, async ({ page, axe }) => {
    await page.goto(route);
    await page.getByRole('main').waitFor(); // a render commit, not a timer
    const results = await axe().analyze();
    const blocking = blockingOf(results);
    // Assert on ids and counts so the failure message is one readable line.
    expect(blocking.map((v) => `${v.id} (${v.nodes.length})`)).toEqual([]);
  });
}

# 2. Tag Selection: What withTags Actually Runs

Every axe rule carries a tags array, and withTags compiles to axe’s runOnly option with type: 'tag'. The semantics are a union, not an intersection: a rule executes if it carries any one of the listed tags. Reading it as an intersection is the most common configuration error in this harness, and it produces a suite that reports zero violations because no rule carries both wcag2aa and wcag22aa.

The WCAG 2.2 tag is small. axe-core’s 2.2-specific additions amount to a handful of rules — target-size for SC 2.5.8 (Target Size Minimum) is the headline one — so withTags(['wcag22aa']) on its own runs almost nothing and looks like a green build. WCAG 2.2 is a superset of 2.1, which is a superset of 2.0, so the tag list has to be cumulative: the 2.0 A and AA tags, the 2.1 A and AA tags, and then wcag22aa on top. Level AAA rules live under wcag2aaa and should be left out of a gate unless the organisation genuinely claims AAA.

best-practice is the tag that causes arguments. It holds real, useful rules — region, landmark-one-main, page-has-heading-one, heading-order — none of which map to a success criterion. Running them in the blocking set means a component test that renders a button in isolation fails region because the button is not inside a landmark. Run best-practice in a second, non-blocking scan and route the output to the reporting and violation-tracking dashboards rather than to the gate.

Two builder methods interact badly and are worth stating plainly. withRules also compiles to runOnly, so calling withTags and withRules on the same builder means the second call wins and the first is silently discarded. disableRules, by contrast, writes into options.rules[id].enabled = false and composes correctly with either. The escape hatch for anything the fluent API does not cover is .options(), which merges a raw axe run-options object:

const results = await axe()
  .options({
    // Skip building the passes/inapplicable arrays. On a 4,000-node dashboard
    // this took the serialised result from 6.2 MB to 180 KB.
    resultTypes: ['violations', 'incomplete'],
    rules: {
      // Per-rule options: do not scroll the page hunting for background colours.
      'color-contrast': { options: { noScroll: true } },
    },
  })
  .analyze();

Rule-level tuning beyond this — custom check options, per-page allowances, the difference between incomplete and violations — belongs with the shared axe-core configuration and setup conventions, so that the same rule decisions apply whether the scan is driven by this harness, by a component test, or by a command-line run.

# 3. Scoping the Scan with Locators

include and exclude accept three shapes: a CSS selector string, an array of strings that describes a frame path, or a Playwright Locator. The locator form is the one worth adopting, because it reuses the selector logic already written for the functional suite instead of duplicating a brittle CSS string next to it.

// tests/a11y/dialog.spec.ts
import { test, expect, blockingOf } from './fixtures';

test('scan the booking dialog only', async ({ page, axe }) => {
  await page.goto('/booking');
  await page.getByRole('button', { name: 'Choose a slot' }).click();
  const dialog = page.getByRole('dialog', { name: 'Choose a slot' });
  await dialog.waitFor();

  const results = await axe()
    // Locators must resolve to exactly one element: strict mode applies here.
    .include(dialog)
    // The vendor availability calendar is an iframe we do not control.
    .exclude(['#availability-frame'])
    // Landmark rules cannot pass inside a dialog subtree; assert them per route.
    .disableRules(['region', 'landmark-one-main', 'page-has-heading-one'])
    .analyze();

  expect(blockingOf(results).map((v) => v.id)).toEqual([]);
});

Three behaviours decide whether scoping helps or quietly hides failures. First, exclude wins over include where the two overlap, which is what makes “scan main, but not the embedded map” expressible in one chain. Second, the array form is a frame path, not a selector list: ['#checkout-frame', 'form.payment'] means “inside the iframe matched by #checkout-frame, the element matched by form.payment”, and it is how a scan reaches into a same-origin iframe. Third, and most importantly, page-level rules cannot pass a scoped run. html-has-lang, document-title, landmark-one-main and region all assert something about the document, so including only a subtree either marks them inapplicable or reports them as failures of the subtree. The rule is simple: one unscoped scan per route owns the document-level rules, and every scoped component scan disables them explicitly.

# 4. Attaching Results to the Playwright Report

The single highest-value change to a scan suite is making the result object survive the assertion. testInfo.attach() writes a named blob into the test result, and the HTML report renders it under the test alongside traces and screenshots. Attach two things: the complete AxeResults JSON, which is what tooling and trend dashboards consume, and a short plain-text digest, which is what a human reads in the browser without downloading anything.

// tests/a11y/attach.ts
import type { TestInfo } from '@playwright/test';
import type { AxeResults } from 'axe-core';

export async function attachAxe(testInfo: TestInfo, results: AxeResults) {
  // Retry index in the name keeps attempt 1 and attempt 2 distinguishable.
  const suffix = testInfo.retry > 0 ? `-retry${testInfo.retry}` : '';

  await testInfo.attach(`axe${suffix}.json`, {
    body: JSON.stringify(results, null, 2),
    contentType: 'application/json', // offered as a download in the HTML report
  });

  const lines = results.violations.map((v) => {
    const targets = v.nodes.slice(0, 3).map((n) => n.target.join(' >> '));
    return `${v.impact}\t${v.id}\t${v.nodes.length} node(s)\t${targets.join(', ')}`;
  });
  const incomplete = results.incomplete.map((v) => `incomplete\t${v.id}`);

  await testInfo.attach(`axe-digest${suffix}.txt`, {
    body: [...lines, ...incomplete].join('\n') || 'no violations, no incomplete',
    contentType: 'text/plain', // rendered inline, so it is readable in one click
  });
}

The digest format is deliberately tab separated: it pastes into a spreadsheet, and it is trivially parseable by the annotation script described in structuring JSON violation output for Slack and GitHub annotations. Screenshotting the offending node is worth the extra six lines, because a picture of a 2.9:1 contrast failure ends an argument that a CSS selector does not:

// Screenshot the first blocking node when its target is a plain CSS selector.
const first = blockingOf(results)[0];
const target = first?.nodes[0]?.target[0];
if (typeof target === 'string') {
  const node = page.locator(target).first();
  if (await node.isVisible()) {
    await testInfo.attach(`node-${first.id}.png`, {
      body: await node.screenshot(),
      contentType: 'image/png',
    });
  }
}

Guard the type of target: axe returns a string for an ordinary element, but an array when the node lives inside an iframe or a shadow root, and page.locator() cannot consume that array. Attach on every run rather than only on failure. A passing run’s JSON is what lets a dashboard show that a route went from nine violations to zero, and a zero-violation attachment is a few hundred bytes.

# 5. The CI Invocation: Sharding and Preview URLs

Two configuration decisions turn a local suite into a pipeline job. The first is a baseURL that comes from the environment, so the identical specs run against a locally built preview or a deployed one. The second is an a11y project with its own timeout, because a scan legitimately takes longer than a click.

// playwright.config.ts
import { defineConfig, devices } from '@playwright/test';

const PREVIEW = process.env.PREVIEW_URL; // set by the deploy job on a PR
const LOCAL = 'http://127.0.0.1:4173';

export default defineConfig({
  testDir: 'tests',
  // Scans are independent, so let Playwright spread them across all workers.
  fullyParallel: true,
  forbidOnly: !!process.env.CI,
  retries: process.env.CI ? 1 : 0,
  workers: process.env.CI ? 3 : undefined,
  reporter: process.env.CI
    ? [['blob'], ['github']] // blob per shard, merged after the matrix
    : [['html', { open: 'never' }]],
  use: {
    baseURL: PREVIEW ?? LOCAL,
    // Preview deployments often sit behind a self-signed edge certificate.
    ignoreHTTPSErrors: !!PREVIEW,
    // A shared bypass header keeps the preview private without a login flow.
    extraHTTPHeaders: PREVIEW && process.env.PREVIEW_TOKEN
      ? { 'x-preview-bypass': process.env.PREVIEW_TOKEN }
      : {},
    trace: 'retain-on-failure',
  },
  projects: [
    {
      name: 'a11y',
      testDir: 'tests/a11y',
      timeout: 90_000, // injection plus analyze on a dense route
      use: { ...devices['Desktop Chrome'], reducedMotion: 'reduce' },
    },
  ],
  // Only start a local server when no preview URL was handed to the job.
  webServer: PREVIEW
    ? undefined
    : {
        command: 'npm run build && npm run preview -- --port 4173 --strictPort',
        url: LOCAL,
        reuseExistingServer: !process.env.CI,
        timeout: 240_000,
      },
});

Sharding splits the test list, not the files, and the split is deterministic for a given list: --shard=2/4 always contains the same tests as long as no test is added or renamed. That determinism matters because it makes a shard’s runtime predictable and a shard’s failures reproducible locally with the same flag. Combine sharding with workers multiplicatively — four shards of three workers is twelve concurrent browsers — and keep the per-shard worker count at or below the runner’s core count, since axe-core’s tree walk is CPU-bound and oversubscribed workers slow every scan at once.

name: a11y-scan
on:
  pull_request:
    paths:
      - 'src/**'
      - 'tests/a11y/**'
      - 'playwright.config.ts'
      - '.github/workflows/a11y-scan.yml'
concurrency:
  group: a11y-scan-${{ github.head_ref }}
  cancel-in-progress: true
jobs:
  scan:
    runs-on: ubuntu-24.04
    timeout-minutes: 25
    strategy:
      fail-fast: false # a failing shard must not hide the other three
      matrix:
        shard: [1, 2, 3, 4]
    steps:
      - uses: actions/checkout@v4
      - uses: actions/setup-node@v4
        with:
          node-version: '20'
          cache: npm
      - run: npm ci
      - run: npx playwright install --with-deps chromium
      - name: Scan shard ${{ matrix.shard }} of 4
        env:
          PREVIEW_URL: ${{ needs.deploy.outputs.preview_url }}
          PREVIEW_TOKEN: ${{ secrets.PREVIEW_BYPASS }}
        run: npx playwright test --project=a11y --shard=${{ matrix.shard }}/4
      - uses: actions/upload-artifact@v4
        if: always()
        with:
          name: blob-${{ matrix.shard }}
          path: blob-report/
          retention-days: 7
  report:
    needs: scan
    if: always()
    runs-on: ubuntu-24.04
    steps:
      - uses: actions/checkout@v4
      - uses: actions/setup-node@v4
        with:
          node-version: '20'
          cache: npm
      - run: npm ci
      - uses: actions/download-artifact@v4
        with:
          pattern: blob-*
          path: all-blobs
          merge-multiple: true
      - name: Merge the four shard reports into one
        run: npx playwright merge-reports --reporter=html,json all-blobs
      - uses: actions/upload-artifact@v4
        with:
          name: a11y-html-report
          path: playwright-report/
          retention-days: 14

# Pipeline Integration

The gate is the Playwright exit code and nothing else. A failed expect produces exit 1, the shard job fails, and fail-fast: false keeps the other three shards running so the pull request gets a complete picture rather than the first failure. Mark the report job — not the individual shards — as the required status check in branch protection, because it is the only job that observes all four shard outcomes and it is the job whose artifact a reviewer opens.

Three artifacts leave the pipeline. The merged playwright-report/ directory is for humans: it contains the per-test attachments, so the JSON, the digest and the node screenshot are one click from a failing test name. The merged report.json from --reporter=html,json is for machines, and it is the input to a trend series — commit it to a metrics store keyed by commit SHA so a violation count becomes a line on a chart rather than a fact that expires when the artifact does. The individual blob-* artifacts are the debugging path: a single shard’s blob can be turned back into a local HTML report without re-running anything.

PR annotations come from the github reporter, which writes failures as workflow annotations attached to the file and line of the failing assertion. That places the failure on the diff, but it names the spec, not the accessibility rule. For an annotation that says image-alt on img.hero-banner, parse the digest attachments in the report job and emit ::error file=…:: lines, or post one consolidated comment. The mechanics of merging shard output into a single reviewable artifact are covered in merging sharded accessibility reports into one artifact, and the shard-sizing arithmetic for larger matrices is in sharding axe-core scans across parallel CI jobs.

# Troubleshooting and Flaky-Test Mitigation

analyze() rejects with an evaluation error after a navigation. The builder captures a page reference at construction, but injection happens inside analyze(). If a navigation is in flight when analyze() runs, the execution context is destroyed mid-injection and the promise rejects with a context error rather than a violation. Always await the navigation, then build and analyze. Never wrap analyze() in Promise.all alongside a click that navigates.

A frame reports frame-tested as incomplete. axe cannot inject into a frame whose sandbox attribute omits allow-scripts, or into a cross-origin frame it has no execution context for. The result is an incomplete entry for frame-tested, not a violation, so a suite that only asserts on violations treats an unscanned third of the page as clean. Either exclude the frame explicitly — which makes the gap visible in the chain — or assert the frame’s contents in a separate spec that navigates directly to the frame’s own URL.

color-contrast lands in incomplete instead of passing or failing. Text over a gradient, a canvas, an image or a background-image gives axe no computable background colour, so it declines to judge. Teams read the empty violations array as a pass. Print the incomplete count in the digest — the attachment helper above does — and treat a rising incomplete count on a page as a signal to write a targeted assertion rather than as noise.

Strict-mode violation from a locator handed to include. page.getByRole('listitem') matching twelve elements throws when the builder resolves it. Chain .first(), or narrow with an accessible name, and prefer a locator that the functional suite already treats as unique.

Retries hide a genuine intermittent violation. With retries: 1, a scan that fails on attempt one and passes on attempt two reports as flaky and the job goes green. That is exactly the shape of a real race condition in the application — an aria-busy region that occasionally renders its error state after the scan. Because the attachment name carries the retry index, the two attempts’ JSON can be diffed directly; a repeated aria-required-children on attempt one and not attempt two is a hydration bug, not a runner bug.

The scan times out on a dense route. Injection plus a full-tree walk on a page with several thousand nodes can exceed a 30-second default. Raise the timeout on the a11y project only, so functional specs keep a tight timeout, and cut the result payload with resultTypes before reaching for a bigger runner.

Animation-driven contrast flake. A fading toast or a shimmering skeleton produces different computed colours on consecutive frames. Set reducedMotion: 'reduce' in the project’s use block, and where the app animates regardless, wait for the Web Animations API to settle before scanning: await page.waitForFunction(() => !document.getAnimations().some((a) => a.playState === 'running')).

# Common Pitfalls

• Importing test from @playwright/test in a spec that needs the axe fixture, so the fixture is undefined and the failure looks like a syntax problem rather than a wiring one.
• Reusing one AxeBuilder instance for two analyze() calls after a navigation; construct a fresh builder per scan, which is exactly what the factory fixture makes cheap.
• Calling both withTags and withRules on one builder and assuming they combine — the second call replaces runOnly and the first is discarded silently.
• Listing wcag22aa without the 2.0 and 2.1 tags, which runs a couple of rules and reports a clean page.
• Adding axe-core to package.json at a pinned version alongside @axe-core/playwright, producing two copies and a baseline that moves when the lockfile is regenerated.
• Asserting expect(results.violations).toEqual([]) on the raw array, which prints an unreadable diff; map to id and node count first and let the attachment carry the detail.
• Scoping with include and leaving region enabled, so every component scan fails a landmark rule that cannot pass inside a subtree.
• Sharding without fail-fast: false, which cancels three shards the moment one fails and leaves the reviewer with a quarter of the findings.

# FAQ

Should the accessibility scans live in the functional specs or in their own files? Both, for different reasons. A dedicated route-matrix spec gives broad coverage of initial page states cheaply and is easy to shard. Scans embedded in functional specs are the only way to reach interacted-with states — a validation error, an open menu, a loaded third page of results — because those states cost a page object and a login to reproduce. The practical split is a route matrix in tests/a11y/ for breadth plus a handful of axe() calls inside high-value flows for depth, all sharing the same fixture so the rule set is identical.

Does the scan need a separate Playwright project, or is a tag filter enough? A separate project earns its keep as soon as the scan needs a different setting from the functional suite — a longer timeout, reducedMotion: 'reduce', a single browser instead of three, or a different retry count. A --grep @a11y tag filter is faster to introduce and adequate while everything shares one configuration, but it cannot express “scan only in Chromium while functional tests run in three engines”. The retrofit path from a tag filter to a dedicated project is covered in integrating axe-core Playwright into an existing project.

How much does an accessibility scan add to a pull-request pipeline? On the matrix in this guide — sixty routes, four shards, three workers each — the scan job runs in roughly fifty seconds of wall clock plus about thirty-five seconds of runner startup and dependency install per shard, and the merge job adds ten to fifteen seconds. The dominant cost is not analyze() but browser launch and navigation, which is why adding scans to specs that already navigate is dramatically cheaper than a parallel suite that re-navigates every route.

What does this harness fundamentally not test? Anything temporal or behavioural. axe judges one DOM snapshot, so it cannot tell whether Tab moves through controls in a sensible order, whether a dialog traps focus, whether a focus ring is actually visible, or whether a status message was announced. Those need a runner driving the keyboard and reading state between presses, which is the subject of testing keyboard focus order with Playwright. Automated scanning of any kind reliably covers roughly 30–40% of WCAG failures; the harness makes that 30–40% free, it does not extend it.

Is there a reason to choose this over the Cypress equivalent if the team has no existing suite? For accessibility work specifically, yes — multi-tab and cross-origin reach, worker-level parallelism and the report attachment model all favour Playwright, and the trade-offs are worked through in comparing Playwright and Cypress for WCAG compliance testing. If a Cypress suite already exists and is healthy, the right move is almost always cypress-axe in that suite rather than a second runner nobody maintains.

# Related

• Integrating axe-core Playwright Into an Existing Project — the retrofit path, including a baseline instead of a day-one failure.
• Comparing Playwright and Cypress for WCAG Compliance Testing — the runner decision, framed around accessibility rather than general end-to-end testing.
• Testing Keyboard Focus Order With Playwright — the behavioural assertions this harness cannot make.
• axe-core Configuration & Setup — the run options and rule tuning the builder assembles.
• Cypress a11y Testing Workflows — the same job in the other runner, for teams already invested there.