# CI/CD Integration & Automated Quality Gating

An accessibility scan that runs on a schedule and mails out a PDF changes nothing, because by the time anyone reads it the regression is already in production and the person who wrote it has moved on. This section covers the other kind of scan: the one whose exit code decides whether a branch is allowed to merge, and the surrounding machinery — a report format, a committed budget, a branch policy, an annotation layer and a shard plan — that makes that decision fast enough and trustworthy enough for a team to leave switched on.

Key implementation targets:

• A pull-request workflow that installs dependencies, restores a cached browser binary, serves the built application and scans a fixed route list in under six minutes.
• A machine-readable violation report with a stable shape, so every downstream decision reads a field rather than parsing console output.
• A committed budget file that says how many failing nodes each rule is currently allowed, plus the ratchet policy that shrinks it.
• A blocking rule expressed as severity plus budget — critical and serious count against the gate, moderate and minor become annotations.
• A required status check in branch protection, with a diff-aware scan for pull requests and a full scan on the default branch.
• A rollout path that starts in warning mode, soaks for a fixed period, and only then gains the ability to fail a build.
• Sharded execution and affected-package detection so a 240-route scan does not add a quarter of an hour to every pull request.
• Trend reporting that answers “is this getting better” with a number, not with an impression.

Before writing any workflow, decide what the gate is actually allowed to do. Five strategies show up in real repositories, and they differ far more in operational cost than in technical difficulty — the hard part is never the YAML, it is choosing a rule the team will not disable in the third week.

# Core Principles

Shift-left validation is an economic argument, not an ideological one. A missing form label found by an external audit costs a finding, a remediation ticket, a sprint slot, a regression test and a re-audit; the same label found by a pull-request check costs one push. The multiplier is roughly the same one that applies to every other class of defect, and it collapses the moment feedback stops arriving where work happens. That is why an accessibility gate belongs in the same workflow run as the unit tests, with its result rendered in the pull-request checks list — not in a nightly job whose output lands in a channel nobody reads, and not in a quarterly audit whose findings arrive as a spreadsheet.

Severity thresholds are what stop a gate from being all-or-nothing. axe-core stamps every result with an impact of minor, moderate, serious or critical, and those labels are a genuine proxy for how badly a real user is stuck: a button with no accessible name is critical because a screen-reader user cannot operate it at all, while a redundant title attribute is minor because nothing is blocked. Blocking on critical and serious while reporting moderate and minor keeps the failure list short and every failure defensible. The moment a minor finding blocks a release, the argument shifts from “fix the bug” to “turn off the check”, and that argument only ever has one winner.

The automated-versus-manual split sets the ceiling on what any gate can promise. Automated scanners reliably detect roughly 30–40% of WCAG failures — the machine-decidable ones: an image with no alt, a control with no name, a contrast ratio below 4.5:1, an ARIA attribute pointing at an id that does not exist. The remaining majority needs a person: whether alt text conveys the same information as the image, whether the focus order tells a coherent story, whether an error message explains how to recover, whether a novel widget behaves the way its role implies. A gate that is sold internally as “we are now accessible” will be discredited by the first manual audit. Sold accurately as “the machine-checkable third can no longer regress”, it survives, and it makes the manual testing budget go much further because auditors stop spending their time on findings a script could have caught.

A gate must be deterministic before it is allowed to block. Determinism means the same commit produces the same violation list on every run, on every runner, in every timezone. Three things break it in practice: an unpinned browser build, because font rendering and layout affect the color-contrast sampling; an unstable scan point, because an early scan walks a DOM that has not finished rendering; and non-deterministic content, because a carousel that starts on a random slide fails a different rule each run. Fix all three before wiring the exit code to branch protection. A flaky accessibility check trains developers to click re-run, and a check people re-run reflexively is worse than no check, because it consumes runner minutes while teaching the team that its failures are noise.

A budget and a baseline are not the same instrument, and conflating them causes the most common gate failure of all. A baseline is a recorded set of specific known violations — this rule, on this selector, on this page — that the gate suppresses; it is precise, it keeps the failure list clean, and it silently expires the moment a component is refactored and the selectors move, at which point every suppressed violation reappears at once. A budget is a number: this rule may fail on at most seven nodes across the scanned routes. A budget survives refactoring, is trivial to ratchet down, and gives a legacy codebase a way to be gated at all, but it cannot tell a fixed violation from a moved one. Use a budget as the gate and a baseline only for genuinely permanent exceptions, each with an owner and a review date, so a suppression cannot outlive the reason it was added.

# GitHub Actions A11y Pipeline Setup

The workflow is where every other decision in this section is expressed, so it deserves more care than a copy-pasted step. GitHub Actions a11y pipeline setup covers the runner shape: a paths filter so the job does not run for a README change, a concurrency group keyed on the head ref so a force-push cancels the previous run, a pinned Node minor version, and an explicit timeout-minutes so a hung dev server fails in fifteen minutes instead of holding a runner for six hours.

Browser caching is the single biggest lever on runtime. A cold playwright install --with-deps chromium downloads roughly 130 MB and spends one to two minutes on it, on every run, forever. Setting PLAYWRIGHT_BROWSERS_PATH into the workspace and caching that directory turns the install into a cache restore of a few seconds, but only if the cache key includes something that changes when Playwright’s pinned revision changes — hash the lockfile, never the OS name alone, or the first Playwright bump will leave a stale binary in the cache and the job will fail with a missing-executable error that looks nothing like a dependency problem. Note that a restored binary still needs its shared libraries, so a cache hit must be followed by playwright install-deps rather than by nothing at all.

A matrix run is the right way to add dimensions the gate genuinely needs, and the wrong way to add dimensions it does not. Two viewports (a phone width and a desktop width) catch real responsive failures where a control collapses into an icon and loses its label. Two colour schemes catch the dark-mode contrast regressions that ship almost unchallenged in most codebases. A matrix of five browsers, by contrast, mostly produces five copies of the same violation, because axe-core evaluates the DOM and the accessibility tree rather than browser-specific rendering — the exception is contrast, which is worth checking on one engine only. Keep the matrix at two to four legs, set fail-fast: false so one leg’s failure does not hide the others, and name each leg so the check name in the pull-request list says which configuration failed.

Annotations are what make a failure actionable inside the review. Writing a markdown table into $GITHUB_STEP_SUMMARY costs one >> redirect and puts the rule ids, node counts and target selectors on the run page. Going further and posting file-level annotations or a single updated comment is worth it on a repository with many contributors — annotating pull requests with axe-core violation comments works through the token scopes and the update-in-place pattern that avoids twelve bot comments on a long-lived branch. For the blocking half, blocking pull requests on critical accessibility violations shows the impact filter and exit-code plumbing, and configuring GitHub Actions for automated WCAG checks covers the tag selection that decides which success criteria are in scope in the first place.

# Progressive Threshold Management

No real codebase passes a strict accessibility gate on the day it is switched on. A mid-sized application typically reports somewhere between eighty and four hundred failing nodes on a first full scan, most of them a handful of rules repeated across shared components. The choice is therefore not between gating and not gating; it is between gating at today’s number and gating at zero, and only one of those can be turned on this week. Progressive threshold management is the discipline of committing today’s number as a budget, blocking anything worse, and shrinking the number on a schedule.

The budget is a file in the repository, reviewed like code. Keeping it in version control means every change to the allowance appears in a diff with an author and a reviewer, which is the entire mechanism by which a budget stays honest — an allowance stored in a CI variable or a dashboard setting gets raised quietly at 5pm on a release day. Record the allowance per rule rather than as one total, because a single total lets a team fix seven easy contrast nodes and spend the headroom on a new missing-button-name failure. A per-rule budget makes that trade impossible: the contrast line goes down and the button-name line stays at zero.

Ratcheting turns the budget from a snapshot into a plan. The mechanics are simple — after a scan, if the found count for a rule is below its allowance, rewrite the allowance to the found count and commit it. That “auto-tighten on improvement” step is worth more than any manual schedule, because it captures incidental fixes: a developer who rewrites a component and happens to remove four contrast failures permanently locks in the improvement without filing anything. Layer a deliberate reduction on top — two nodes per rule per sprint, or a fixed percentage — and the budget reaches zero on a date somebody can name. Ratcheting violation budgets down each sprint covers the commit-back mechanics, including how to avoid the workflow fighting itself when two pull requests both tighten the same line.

Legacy code needs a different granularity again. A single global budget means an untouched legacy area’s failures are the headroom that new code spends, and a newly built feature can ship with real violations because an old admin screen is worse. Splitting the allowance by directory or route prefix fixes this: /legacy/ keeps a generous budget that shrinks slowly, while /checkout/ sits at zero from the first day. Per-directory accessibility budgets in legacy code describes the path-matching rules, and setting up progressive accessibility thresholds in CI walks the first-run audit that produces the initial numbers.

# Pull-Request Gating & Branch Policies

A workflow that exits non-zero has produced an opinion, not an enforcement. Until the check is listed as required in branch protection, the merge button stays green and the accessibility job is a suggestion — and every team discovers this the first time a red check gets merged past. Pull-request gating and branch policies covers the repository-side configuration: the exact check name to require, the interaction with merge queues, and the trap where a paths filter causes the job to be skipped so the required check never reports at all and the pull request waits forever.

That skipped-check trap deserves naming, because it is the most common way an accessibility gate quietly stops working. If the workflow has paths: ['src/**'] and a required status check with the same name, a documentation-only pull request never runs the job, the check stays pending, and the merge is blocked for a reason that has nothing to do with accessibility. The fix is a second job that always runs and reports the same check name as a pass when the paths filter excludes everything relevant, or dropping the paths filter and making the job itself decide quickly that there is nothing to scan. Either way the check must always report a conclusion.

Diff-aware scanning is what makes gating tolerable on a large site. Scanning 240 routes on every pull request is both slow and unfair: a change to one checkout component should not be blocked by a pre-existing contrast failure on a marketing page nobody touched. Mapping changed files to affected routes and scanning only those cuts the run to seconds and makes every reported failure attributable to the diff. The cost is a mapping to maintain and a blind spot — a change to a shared header affects every route, and a naive mapping will miss it — so pair a diff-aware pull-request scan with a full scan on the default branch. Gating only changed pages with a diff-aware scan covers the mapping strategies and the shared-dependency fallback.

Overrides need to exist, and they need to leave a trace. A gate with no escape hatch gets bypassed by an administrator at 2am during an incident, and nobody ever finds out; a gate with a documented escape hatch gets used three times a quarter and each use is a record. Implement the override as a label — a11y-override — that the workflow reads, so applying it is an audited action attributable to a person, and have the job post a comment naming the violations that were waived and open a follow-up issue automatically. Requiring accessibility status checks in branch protection covers both the protection rule and the audit trail around it.

# Auto-Fail vs Warning Workflows

The decision of what fails a build is a rollout decision, not a configuration one. A gate introduced at full strength on a codebase with two hundred existing violations blocks the next fifty pull requests, none of which caused the problem, and the predictable outcome is a pull request that deletes the workflow — usually with a reasonable justification attached. Auto-fail vs warning workflows treats warn-then-block as the default path rather than a compromise: run in warning mode long enough to prove the scan is stable and the numbers are real, then promote.

The soak period has a concrete purpose beyond politics. Warning mode is how the run-to-run variance is measured: scan the same commit ten times and count how many runs produce an identical violation list. Anything less than ten out of ten means the gate would have failed builds at random, and the cause is findable — an unpinned browser, a scan that fires before hydration, an A/B experiment flag, a relative timestamp that changes the contrast of a badge. Two weeks of warning-mode runs also produce the histogram that sets the initial budget honestly, rather than from a single audit that happened to catch a good day. Soak-testing a new accessibility gate in warning mode covers the variance measurement and the promotion criteria.

Exit codes are the actual interface between the scanner and the pipeline, and using more than two of them pays off immediately. Reserve 0 for pass, 1 for a real budget breach, and a distinct code — 2 is conventional — for a tooling failure: the dev server never came up, the browser crashed, the report file is missing. Without that separation, a broken container looks identical to an accessibility regression, and the on-call developer spends twenty minutes hunting a violation that was never there. Most scanner CLIs collapse everything into 1, which is a good reason to run the scan through a small script that owns the exit code rather than calling the CLI directly from the workflow step.

continue-on-error: true is the correct warning-mode primitive, with one wrinkle worth knowing: it makes the step non-fatal but still marks the run with a warning annotation, so the signal is visible without being fatal. It is not the same as if: always(), which controls whether a step runs at all, and it is not the same as swallowing the exit code with || true, which hides the failure completely and produces a job that is green even when the scan crashed. When the gate is promoted, removing continue-on-error should be the entire diff — if promotion requires rewriting the step, the warning mode was not modelling the blocking mode. Choosing exit codes for warning and blocking a11y jobs sets out the full code table and the wrapper script that emits it.

# Docker-Based Pipeline Execution

Contrast results depend on rendered pixels, and rendered pixels depend on the font stack, the browser build and the compositor. That is why an accessibility scan is one of the test types that genuinely benefits from a container: without a pinned image, a runner-image upgrade that swaps a font package can change color-contrast results on text that nobody edited, and the resulting failure is attributed to whichever pull request happened to be open. Docker-based pipeline execution covers building the scan image, pinning the browser and font layers, and running the job inside it.

Pin three versions explicitly and treat each bump as a reviewable change: the base image by digest rather than by a floating tag, the browser revision, and the axe-core version. The axe-core pin matters more than teams expect — a minor release can add rules or tighten an existing one, and the first pipeline run after an unpinned upgrade reports new violations on unchanged code. That is a good outcome delivered at a terrible moment. Pinning axe-core and bumping it deliberately turns those new rules into a scheduled piece of work with its own pull request and its own budget adjustment, rather than an ambush during someone else’s release.

Layer ordering is what keeps the image cheap. Put the system libraries and fonts in an early layer, the browser binary next, the dependency install after that, and the application source last, so a source change rebuilds one small layer instead of re-downloading a browser. A browser binary baked into the image also removes the download from the critical path entirely, which is a different trade-off from caching it in the workflow: the image is bigger and takes longer to pull, but the pull is one predictable operation instead of a cache that can miss. Caching axe-core browser binaries in CI containers compares the two approaches with real timings.

Containers also make a second scanner cheap to run alongside the first. Lighthouse contributes a score and a small set of audits that overlap heavily with axe, and its value in a gate is mostly as a coarse trend line rather than as a blocking check — score-based gates are noisy because a score aggregates weighted audits, so one new violation can move the number by a variable amount. Running it in the same container as the axe scan at least guarantees both tools see the same rendering. Running Lighthouse CI in a Docker-based pipeline covers the container flags and the assertion configuration, and the broader trade-off between the two tools belongs to the accessibility testing fundamentals and tool selection section.

# Reporting, Dashboards & Violation Tracking

Everything downstream of the scan reads the report, so the report’s shape is an interface and needs to be treated as one. A raw axe result object is a poor interface: it is large, it nests differently per rule, and it says nothing about which commit or which route produced it. Normalising it once — a flat list of findings each carrying a rule id, an impact, a route, a target selector, a short summary, the commit SHA and the axe version — means the budget comparator, the annotation writer, the Slack notifier and the dashboard loader all read the same fields. Reporting, dashboards and violation tracking covers that normalised schema and the sinks that consume it.

Which field drives which decision is worth being explicit about, because mixing them is how gates end up with strange behaviour. impact decides whether a finding counts toward the gate. The rule id decides which budget line it is compared against. The node target selector decides where the annotation is anchored. failureSummary is human-facing text for the comment and must never be parsed. And the incomplete array — results where axe could not decide, typically contrast over a background image — is triage material that belongs in a report and must never contribute to an exit code.

Routing matters as much as the schema. A pull-request finding belongs in the pull request, where the person who caused it is already looking; a default-branch trend belongs in a dashboard; a new critical violation on the default branch belongs in a channel, because it means the gate was bypassed or a scan gap exists. Sending everything to every sink produces alert fatigue within a fortnight — a Slack channel that fires on every pull-request warning is muted by the second week and provides no signal thereafter. Structuring JSON violation output for Slack and GitHub annotations covers the payload shapes for both sinks and the deduplication that keeps a long-running branch from posting the same finding twelve times.

Trends are the artefact leadership actually wants, and they need one more thing than the reports contain: a stable scan scope. Comparing this sprint’s 46 findings with last sprint’s 61 is meaningless if the route list grew by ten pages in between, so store the route count and the axe version alongside the counts, and treat a change in either as a break in the series rather than as a data point. Tracking accessibility violation trends across sprints covers the storage shape and the normalisation, and exporting accessibility results to compliance dashboards covers the mapping from rule ids to the success criteria an auditor or an accessibility statement needs. Findings that are not going to be fixed this quarter should be routed into accessibility debt triage and prioritisation rather than left to accumulate silently inside the budget.

# Monorepo Parallel Test Sharding

A gate’s credibility is partly a function of its runtime. Once the accessibility check takes fifteen minutes, developers stop waiting for it, start merging on the other green checks, and the required-check setting becomes the only thing keeping it alive. A 240-route scan on a single job is roughly fourteen minutes of wall clock; the same work across four jobs is roughly six, including the fixed install cost each job pays and the merge step at the end. Monorepo parallel test sharding covers how to split the work and how to put the results back together.

Shard on stable, evenly weighted units. A deterministic hash of the route path modulo the shard count gives an even split and, crucially, keeps a given route on the same shard between runs, which makes a shard’s timing comparable across builds and makes a slow shard diagnosable. Splitting alphabetically produces wildly uneven shards because route depth correlates with section; splitting randomly makes flakiness impossible to localise. Whatever the split, each shard must produce a self-contained report file named after the shard index, because merging is much easier than reconstructing which shard scanned what.

In a monorepo the sharper win is scanning less rather than scanning in parallel. A change to one package rarely affects every application, and a build graph already knows which applications depend on the changed package, so the scan list can be derived rather than hard-coded. That turns most pull requests into a scan of one application’s routes, and only a change to a shared design-system package into a full run. Scanning only affected packages in a Turborepo monorepo covers deriving the list from the dependency graph and the fallback when the graph cannot answer.

Sharding moves the gating decision to the end, which is the part teams underestimate. No individual shard can decide whether the budget was breached, because the budget is a total across the scanned scope, so each shard must upload a partial report and a final job must download all of them, merge them, run one comparison and own the exit code. That final job also needs to fail when a shard is missing entirely — a merge over three of four reports will happily report an improvement that is really a crashed job. Merging sharded accessibility reports into one artifact covers the merge, the completeness assertion and the deduplication of findings that appear on routes shared between shards.

# Reference Pipeline

The four files below are one complete gate. A scan script produces a normalised report from a fixed route list; a committed budget declares what is currently tolerated; a comparator writes the job summary and owns the exit code; and a workflow wires them together with a cached browser, an uploaded artifact and a single blocking step. Each of the seven topics above acts on one part of this pipeline, which is the fastest way to see where a change belongs.

The scan script owns the browser and the route list, and nothing else. Keeping the report-writing here and the decision-making elsewhere is what lets the same script be used for a warning-mode run, a blocking run and a nightly full crawl without any conditional logic.

// scripts/a11y/scan.mjs — usage: node scripts/a11y/scan.mjs http://127.0.0.1:4173
import { writeFileSync, mkdirSync } from 'node:fs';
import { chromium } from 'playwright';
import AxeBuilder from '@axe-core/playwright';

const baseUrl = process.argv[2] ?? 'http://127.0.0.1:4173';
const ROUTES = ['/', '/pricing', '/checkout', '/account/orders', '/support/contact'];

const browser = await chromium.launch();
const context = await browser.newContext({
  viewport: { width: 1280, height: 900 },
  reducedMotion: 'reduce', // animation mid-frame changes contrast sampling
  colorScheme: 'light',    // pin the theme; dark mode gets its own scan leg
});

const pages = [];
for (const route of ROUTES) {
  const page = await context.newPage();
  await page.goto(`${baseUrl}${route}`, { waitUntil: 'load' });
  await page.getByRole('main').waitFor({ state: 'visible' });
  const results = await new AxeBuilder({ page })
    .withTags(['wcag2a', 'wcag2aa', 'wcag21a', 'wcag21aa', 'wcag22aa'])
    .analyze();
  pages.push({
    url: route,
    violations: results.violations.map((v) => ({
      id: v.id,
      impact: v.impact,
      help: v.help,
      nodes: v.nodes.map((n) => ({ target: n.target, summary: n.failureSummary })),
    })),
    // Kept separate so no downstream step can accidentally gate on them.
    incomplete: results.incomplete.map((v) => ({ id: v.id, count: v.nodes.length })),
  });
  await page.close();
}
await browser.close();

mkdirSync('a11y', { recursive: true });
writeFileSync(
  'a11y/report.json',
  JSON.stringify(
    {
      generatedAt: new Date().toISOString(),
      commit: process.env.GITHUB_SHA ?? 'local',
      routeCount: ROUTES.length, // a change here breaks trend comparability
      pages,
    },
    null,
    2,
  ),
);
console.log(`scanned ${ROUTES.length} routes`);

The budget is committed next to the script. Every number in it is a debt with an owner; reviewedOn exists so a stale budget shows up in a quarterly review rather than becoming permanent by default.

{
  "blockingImpacts": ["critical", "serious"],
  "total": 12,
  "rules": {
    "color-contrast": 7,
    "label": 3,
    "link-name": 2,
    "button-name": 0,
    "aria-required-children": 0
  },
  "reviewedOn": "2026-07-13",
  "ratchet": { "everySprints": 1, "reduceTotalBy": 2 }
}

The comparator is the only file allowed to decide the job’s fate. It writes the job summary before it exits, so a failing run explains itself on the run page without anyone downloading an artifact, and it treats an unbudgeted rule as an allowance of zero — which is what makes a brand-new violation type fail even while old ones are tolerated.

// scripts/a11y/compare-budget.mjs
// usage: node scripts/a11y/compare-budget.mjs a11y/report.json a11y/budget.json
import { readFileSync, appendFileSync } from 'node:fs';

const [reportPath, budgetPath] = process.argv.slice(2);
const report = JSON.parse(readFileSync(reportPath, 'utf8'));
const budget = JSON.parse(readFileSync(budgetPath, 'utf8'));
const blocking = new Set(budget.blockingImpacts);

const found = new Map(); // rule id -> failing node count across all routes
for (const page of report.pages) {
  for (const violation of page.violations) {
    if (!blocking.has(violation.impact)) continue; // minor/moderate never block
    found.set(violation.id, (found.get(violation.id) ?? 0) + violation.nodes.length);
  }
}

const rows = [...found.entries()]
  .map(([id, count]) => {
    const allowed = budget.rules[id] ?? 0; // an unbudgeted rule gets no allowance
    return { id, count, allowed, over: count - allowed };
  })
  .sort((a, b) => b.over - a.over);

const total = [...found.values()].reduce((sum, n) => sum + n, 0);
const breaches = rows.filter((row) => row.over > 0);

const lines = ['### Accessibility budget', '', '| Rule | Found | Budget | Delta |', '|---|---|---|---|'];
for (const row of rows) {
  const delta = row.over > 0 ? '+' + row.over : String(row.over);
  lines.push('| `' + row.id + '` | ' + row.count + ' | ' + row.allowed + ' | ' + delta + ' |');
}
lines.push('', 'Blocking nodes: ' + total + ' of ' + budget.total + ' allowed.');
if (process.env.GITHUB_STEP_SUMMARY) {
  appendFileSync(process.env.GITHUB_STEP_SUMMARY, lines.join('\n') + '\n');
}
console.log(lines.join('\n'));

// Exit 1 only above the threshold — never merely because violations exist.
process.exit(breaches.length > 0 || total > budget.total ? 1 : 0);

The workflow contributes the environment and the plumbing. Note that the artifact upload runs with if: always() so a failing gate still leaves the evidence behind, and that the comparator is the last step so nothing after it can mask the exit code.

name: a11y-budget-gate
on:
  pull_request:
    branches: [main]
    paths:
      - 'src/**'
      - 'a11y/budget.json'
      - 'scripts/a11y/**'
      - '.github/workflows/a11y-budget-gate.yml'
permissions:
  contents: read
concurrency:
  group: a11y-budget-${{ github.head_ref || github.ref }}
  cancel-in-progress: true
jobs:
  budget-gate:
    runs-on: ubuntu-24.04
    timeout-minutes: 25
    env:
      PLAYWRIGHT_BROWSERS_PATH: ${{ github.workspace }}/.cache/ms-playwright
    steps:
      - uses: actions/checkout@v4
      - uses: actions/setup-node@v4
        with:
          node-version: '20.17.0' # pin the minor: a V8 bump can move contrast results
          cache: npm
      - run: npm ci
      - name: Restore the cached browser binary
        id: browsers
        uses: actions/cache@v4
        with:
          path: ${{ env.PLAYWRIGHT_BROWSERS_PATH }}
          # Lockfile hash in the key so a Playwright bump invalidates the cache.
          key: pw-chromium-${{ runner.os }}-${{ hashFiles('package-lock.json') }}
      - name: Download Chromium on a cache miss
        if: steps.browsers.outputs.cache-hit != 'true'
        run: npx playwright install --with-deps chromium
      - name: Install only the OS libraries on a cache hit
        if: steps.browsers.outputs.cache-hit == 'true'
        run: npx playwright install-deps chromium
      - name: Build and serve the application
        run: |
          npm run build
          npx http-server dist -p 4173 --silent &
          npx wait-on http://127.0.0.1:4173 -t 60000 # fail fast if it never boots
      - name: Scan the route list
        run: node scripts/a11y/scan.mjs http://127.0.0.1:4173
      - uses: actions/upload-artifact@v4
        if: always()
        with:
          name: a11y-report-${{ github.event.pull_request.number }}
          path: a11y/report.json
          retention-days: 30
      - name: Compare the report with the committed budget
        run: node scripts/a11y/compare-budget.mjs a11y/report.json a11y/budget.json

# WCAG 2.2 Coverage Mapping

A gate is only auditable if each blocked build maps to a success criterion. The table below covers the criteria the rule set in this section can genuinely enforce, the axe rule ids that carry the enforcement, and — for each one — the part of the criterion that stays outside the machine’s reach. That last column is the honest version of the 30–40% figure: for most criteria the gate covers a mechanical precondition, not the criterion itself.

Two rows in that table have no rule id, and they are not oversights. Focus visibility depends on rendered pixels and on the difference between focused and unfocused states, which a DOM walk cannot see; status messages depend on whether an announcement fired, which requires observing a live region over time rather than inspecting a snapshot. Both are reachable with authored checks and interaction assertions rather than with the default rule set — that work belongs to custom rule development and context-aware testing, and its output plugs into exactly the same budget file and comparator as everything else here.

Beyond those two, there are whole classes of criteria a gate should not pretend to hold. Anything that depends on meaning — alternative text quality, link purpose in context, error suggestion usefulness, heading accuracy — is human work. Anything that depends on a journey rather than a page, such as consistent help placement or completing a multi-step process without re-entering data, needs a scripted flow rather than a page scan. And anything that depends on a real assistive technology, such as whether a screen reader announces a custom widget usefully, needs a real screen reader. Write those exclusions down next to the budget file; a gate whose limits are documented is much harder to over-claim in a compliance conversation.

# Common Pitfalls

• Wiring the required status check to a job that a paths filter can skip, so unrelated pull requests wait on a check that will never report a conclusion.
• Letting the scanner CLI own the exit code, which collapses “the app never booted” and “there are three new violations” into the same failure signal.
• Blocking on moderate and minor impacts on day one, which makes the first argument about the gate an argument about deleting the gate.
• Comparing a total violation count instead of per-rule counts, so easy fixes on one rule pay for new failures on another and the total looks flat.
• Caching the browser directory with a key that does not change when the browser revision does, producing a missing-executable error on the first upgrade.
• Storing the budget in a CI variable or a dashboard rather than in the repository, where a raised allowance would need a reviewer.
• Counting incomplete results toward the gate, which turns every hero image with text over it into a blocking failure nobody can fix.
• Scanning before the application has settled, then attributing the resulting run-to-run variance to the scanner rather than to the missing wait.
• Merging sharded reports without asserting that every expected shard file arrived, so a crashed shard reads as an improvement.
• Leaving a baseline suppression in place with no owner and no review date, so a violation that was waived for one sprint is invisible three years later.
• Posting a new bot comment on every push, which trains reviewers to collapse the accessibility comment before reading it.

# FAQ

Should the accessibility gate block on the count of violations or on the severity of them? On both, in that order: severity decides what counts, and the budget decides how much of it is tolerated. Filtering to critical and serious keeps every blocking finding defensible, and comparing the filtered count against a per-rule budget lets a legacy codebase be gated today rather than after a remediation project. Blocking on severity alone works only on a codebase that is already at zero, which is a small minority of real repositories.

How long should a new gate run in warning mode before it is allowed to fail a build? Long enough to prove two things: that the scan is deterministic, and that the numbers are stable. In practice that is two to three weeks, or about twenty pull-request runs — enough to see whether the same commit produces the same violation list, and enough to catch the weekly cron job or the A/B experiment that changes the page. If the variance is not zero at the end of that period, promoting the gate just converts a measurement problem into a flaky build.

What happens to the gate when axe-core is upgraded and new rules appear? New rules find violations on unchanged code, so an unpinned upgrade turns a green pipeline red for reasons nobody in the pull request caused. Pin the axe-core version, upgrade it in a dedicated pull request, run the scan in warning mode on that branch to see the new findings, then commit the budget adjustment in the same change. The upgrade then arrives as a reviewed piece of work with a known cost instead of as an interruption.

Can a diff-aware scan replace a full-site scan entirely? No, because the mapping from changed files to affected routes is always approximate, and shared code breaks it: a change to a header component or a global stylesheet can affect every route in the application. Use the diff-aware scan as the fast pull-request gate and run a full scan on the default branch — nightly or on merge — so the two together cover both attribution and completeness. The full scan is also what keeps the trend series honest, since its scope does not change with the diff.

Where do violations go when the team decides not to fix them this quarter? Into the budget as an explicit allowance with an owner, and into a triage queue as a scored item — never into a rule exclusion. An allowance keeps the finding visible and countable, so the ratchet still applies pressure and the trend line still tells the truth; a disabled rule removes the finding from every future report and quietly lowers the ceiling on what the gate can ever catch. If a rule genuinely does not apply to the codebase, that is a documented exclusion with a written justification, which is a different artefact from a deferral.

# Related

• GitHub Actions A11y Pipeline Setup — the runner, the browser cache, the matrix legs and the annotation step.
• Progressive Threshold Management — how to commit today’s violation count as a budget and ratchet it to zero.
• Monorepo Parallel Test Sharding — split a slow full-site scan across jobs and merge the reports before gating.
• Custom Axe Rules & Context-Aware Testing — author the checks that cover the criteria the default rule set leaves ungated.
• Web Accessibility Testing: Fundamentals & Tools — pick and configure the scanner whose output this gate consumes.