Gating Only Changed Pages With a Diff-Aware Scan

This guide is part of Pull Request Gating & Branch Policies. It solves the scope problem: given a diff, work out the smallest set of rendered routes that could have regressed, scan those, and escalate to everything when the diff touches something shared — without ever changing the name of the check that branch protection requires.

Root Cause

Scanning every route on every pull request stops being viable somewhere between forty and a hundred routes. A Playwright run that loads a page, waits for hydration and runs an axe pass costs roughly four to eight seconds per route on a standard hosted runner, so 220 routes is twenty-five minutes of wall clock before any sharding. Teams facing that number do not tolerate it; they move the scan to a nightly schedule, and a nightly scan is not a gate. The regression still lands, and it is discovered by whoever reads the morning report — which after three weeks is nobody.

The obvious fix is to scan only what changed, and the obvious fix does not work naively, because a diff is a list of source files and a scan needs a list of URLs. There is no general mapping between the two. A React component in src/components/ renders on an unknown number of routes. A file in src/routes/checkout/review.tsx probably corresponds to /checkout/review, but “probably” is doing a lot of work: the router may have a base path, the route may be parameterised, and the file may be a layout that wraps forty children. Guessing the URL from the path is the failure mode that makes diff-aware gating untrustworthy — it produces a green check for a page nobody looked at.

The harder case is the one that determines whether the whole approach is safe. A one-line change to packages/design-system/src/Button.tsx can alter the accessible name, the focus ring contrast or the disabled semantics of every button in the product. The honest answer to “which pages changed?” for that diff is “all of them”, and a mapping that returns two routes because two route files import Button directly is worse than no mapping at all: it reports success over a change with the widest possible blast radius. Any diff-aware gate therefore needs three things — an explicit mapping where one exists, a dependency-graph fallback where it does not, and a threshold above which it gives up and scans everything.

From changed files to a scan scope The changed file list splits into files matched by route manifest globs and files resolved through the reverse import graph. The two results merge into one route set, which a share threshold then routes either to a partial diff scan or to a full-site scan. Scope resolution runs before any browser starts git diff against the merge base 17 files routes.json globs explicit, owner-tagged 11 files matched reverse import graph walk up to route entries 6 files unmatched route set share of total 9 of 52 routes mode: diff scan 9 routes mode: full scan all 52 under 40% over 40% Both branches report under the same check name, so the branch policy never sees the difference.
The resolver is a pure function from a file list to a route list plus a mode, which makes it unit-testable without a browser.

Configuration

Get a trustworthy changed-file list

Use the merge base, not the base branch tip. A two-dot diff against origin/main reports every file that differs, including files other people changed after this branch was cut, which inflates the scope on any long-lived branch. The three-dot form asks the right question: what did this branch change since it diverged.

# Requires actions/checkout with fetch-depth: 0 — a shallow clone has no merge base.
BASE="${GITHUB_BASE_REF:-main}"
git fetch --no-tags origin "$BASE"
git diff --name-only --diff-filter=ACMRT "origin/${BASE}...HEAD" > changed.txt
# A: added  C: copied  M: modified  R: renamed  T: type changed
# Deletions are excluded on purpose: a removed file has no route left to scan.
wc -l < changed.txt

Own the mapping explicitly

The mapping between source and route is a product decision, so write it down. A manifest keyed by route, with the globs that render it and the team that owns it, doubles as the routing table for assigning failures — the same data that assigning violation ownership with CODEOWNERS uses to decide who gets the ticket.

{
  "totalRoutes": 52,
  "fullScanTriggers": [
    "packages/design-system/**",
    "src/styles/tokens.css",
    "src/app/layout.tsx"
  ],
  "routes": [
    { "path": "/", "sources": ["src/app/page.tsx"], "owner": "@acme/web" },
    { "path": "/checkout/cart", "sources": ["src/app/checkout/**"],
      "owner": "@acme/payments" },
    { "path": "/checkout/review", "sources": ["src/app/checkout/**"],
      "owner": "@acme/payments" },
    { "path": "/product/pa11y-mug", "sources": ["src/app/product/**"],
      "owner": "@acme/catalog", "note": "representative id for /product/[slug]" },
    { "path": "/legal/terms", "sources": ["content/legal/**"], "owner": "@acme/web" }
  ]
}

A framework that already knows its routes can supply half of this for free. A Next.js App Router build writes .next/app-path-routes-manifest.json, mapping each page module to its URL pattern, and reading it removes the risk of the hand-written manifest drifting away from reality. Use the framework manifest as the source of truth for which routes exist and the hand-written file for what it cannot know: which parameterised route to visit with which representative id, and who owns what.

Fall back to the import graph

Files that no route glob claims — shared components, hooks, utilities — need the reverse import graph. Generate the forward graph once with a static analyser, invert it, and walk upward from each changed file until reaching a module that a route claims.

// a11y/scripts/resolve-scope.mjs
// Usage: node a11y/scripts/resolve-scope.mjs changed.txt graph.json
// Produce graph.json first:  npx madge --json --extensions ts,tsx src packages
import { readFileSync } from 'node:fs';
import { minimatch } from 'minimatch';

const [changedFile, graphFile] = process.argv.slice(2);
const changed = readFileSync(changedFile, 'utf8').split('\n').filter(Boolean);
const graph = JSON.parse(readFileSync(graphFile, 'utf8')); // { file: [imports] }
const manifest = JSON.parse(readFileSync('a11y/routes.json', 'utf8'));

// Invert the dependency graph: for each module, who imports it.
const importers = new Map();
for (const [file, deps] of Object.entries(graph)) {
  for (const dep of deps) {
    if (!importers.has(dep)) importers.set(dep, []);
    importers.get(dep).push(file);
  }
}

const routesFor = (file) =>
  manifest.routes.filter((r) => r.sources.some((g) => minimatch(file, g)));

// Breadth-first walk up the importer chain, stopping at any file a route claims.
function climb(file, seen = new Set()) {
  if (seen.has(file)) return [];
  seen.add(file);
  const direct = routesFor(file);
  if (direct.length > 0) return direct;
  return (importers.get(file) ?? []).flatMap((up) => climb(up, seen));
}

const forced = changed.some((f) =>
  manifest.fullScanTriggers.some((g) => minimatch(f, g)),
);
const resolved = [...new Set(changed.flatMap((f) => climb(f).map((r) => r.path)))];
const share = resolved.length / manifest.totalRoutes;
// Above 40% the partial scan costs nearly as much as the full one and is
// harder to trust, so escalate rather than half-cover the site.
const mode = forced || share > 0.4 ? 'full' : 'diff';
const routes = mode === 'full' ? manifest.routes.map((r) => r.path) : resolved;

console.log(JSON.stringify({ mode, routes, share: Number(share.toFixed(2)) }));
Blast radius of a shared-component change Button.tsx is imported by Toolbar, Modal, DataGrid and FormRow, which between them appear on 38 of the 52 routes. A bar shows 38 of 52 against a threshold marker at 21 routes, so the resolver escalates to a full scan. One shared component, most of the site Button.tsx 4 lines changed Toolbar Modal DataGrid FormRow 38 of 52 routes resolved by the climb Resolved share against the escalation threshold threshold 21 routes 73% resolved Above the threshold the resolver returns mode: full, so no design-system change is partly scanned.
The escalation rule is what makes the mapping safe to trust: a wide diff produces a wide scan rather than a confidently narrow one.

Keep one job name for both modes

This is the constraint that shapes the workflow. Branch protection requires an exact check name, so the scope decision must not change the shape of the job graph — no matrix over the resolved routes, because a matrix appends its values to the check name and a diff-derived matrix produces a different context on every pull request. One job, one name, a loop inside it.

# .github/workflows/a11y-diff.yml
name: accessibility (diff-aware)
on:
  pull_request:
    types: [opened, synchronize, reopened, ready_for_review]
  schedule:
    - cron: '0 3 * * *' # nightly full scan on the default branch
permissions:
  contents: read
jobs:
  a11y-scope:
    # Fixed name, fixed count: this is the required context in every mode.
    name: a11y-scope
    runs-on: ubuntu-24.04
    timeout-minutes: 25
    steps:
      - uses: actions/checkout@v4
        with:
          fetch-depth: 0 # merge-base resolution needs full history
      - uses: actions/setup-node@v4
        with:
          node-version: '20'
          cache: npm
      - run: npm ci
      - name: Resolve the scan scope
        id: scope
        run: |
          if [ "${{ github.event_name }}" = "schedule" ]; then
            echo 'plan={"mode":"full","routes":[]}' >> "$GITHUB_OUTPUT"
            exit 0
          fi
          git fetch --no-tags origin "${GITHUB_BASE_REF}"
          git diff --name-only --diff-filter=ACMRT \
            "origin/${GITHUB_BASE_REF}...HEAD" > changed.txt
          npx madge --json --extensions ts,tsx src packages > graph.json
          plan=$(node a11y/scripts/resolve-scope.mjs changed.txt graph.json)
          echo "plan=${plan}" >> "$GITHUB_OUTPUT"
          echo "Scan plan: \`${plan}\`" >> "$GITHUB_STEP_SUMMARY"
      - run: npx playwright install --with-deps chromium
      - name: Scan the resolved routes
        env:
          A11Y_PLAN: ${{ steps.scope.outputs.plan }}
        # The runner script reads mode and routes from A11Y_PLAN and loops;
        # mode=full ignores the route list and reads a11y/routes.json instead.
        run: npm run a11y:scan -- --plan-from-env --fail-on critical
      - uses: actions/upload-artifact@v4
        if: always()
        with:
          # Mode in the artifact name so a partial run is never mistaken for full.
          name: a11y-${{ fromJSON(steps.scope.outputs.plan).mode }}-${{ github.run_id }}
          path: a11y-results/
          retention-days: 21

The scheduled branch is not optional. A diff-aware gate is, by construction, blind to any route the diff does not reach — including routes broken by a dependency upgrade, a CDN font change, a content edit made through a CMS, or a violation introduced by a merge that the gate scoped narrowly. The nightly full scan is what closes that gap, and its results are the ones that belong in the trend series described in tracking accessibility violation trends across sprints. In a monorepo the same reasoning generalises from routes to packages, which is the subject of scanning only affected packages in a Turborepo monorepo.

Diff-aware coverage against scheduled coverage Four routes across five weekdays. Shaded columns mark the nightly full scan covering every route. Marks show which routes pull-request scans reached: the checkout, product and account routes were each scanned on some days, while the legal terms route was reached only by the nightly run. What the schedule covers that the diff cannot Mon Tue Wed Thu Fri /checkout/cart /product/pa11y-mug /account/settings /legal/terms /legal/terms: no pull request reached this route all week nightly full scan: every route, mode full, updates the baseline pull-request scan: resolved routes only, mode diff, gates the merge
The gate and the schedule answer different questions: one blocks a specific change, the other establishes what the site currently looks like.

Validation

Test the resolver, not the browser. The interesting behaviour is entirely in the mapping, and it runs in milliseconds against fixtures.

// a11y/scripts/resolve-scope.test.mjs — node --test a11y/scripts/
import { test } from 'node:test';
import assert from 'node:assert/strict';
import { execFileSync } from 'node:child_process';
import { writeFileSync } from 'node:fs';

const run = (files) => {
  writeFileSync('/tmp/changed.txt', files.join('\n'));
  return JSON.parse(
    execFileSync('node', ['a11y/scripts/resolve-scope.mjs',
                          '/tmp/changed.txt', 'a11y/fixtures/graph.json'],
                 { encoding: 'utf8' }),
  );
};

test('a route file resolves to exactly its own route', () => {
  const plan = run(['src/app/checkout/review.tsx']);
  assert.equal(plan.mode, 'diff');
  assert.deepEqual(plan.routes.sort(), ['/checkout/cart', '/checkout/review']);
});

test('a design-system change forces a full scan', () => {
  const plan = run(['packages/design-system/src/Button.tsx']);
  assert.equal(plan.mode, 'full');
  assert.equal(plan.routes.length, 52);
});

test('an unreferenced utility resolves to no routes and scans nothing', () => {
  const plan = run(['src/lib/format-currency.test.ts']);
  assert.equal(plan.mode, 'diff');
  assert.deepEqual(plan.routes, []);
});

On a real pull request, read the plan out of the step summary before trusting the green tick. A one-line summary such as {"mode":"diff","routes":["/checkout/cart","/checkout/review"],"share":0.04} tells a reviewer precisely what was and was not covered, which is the difference between a scoped gate and an unexamined one. Print the same line as an annotation if reviewers do not open run summaries.

Edge Cases and Conditional Guards

  • Deleted and renamed routes. A pull request that removes src/app/promo/page.tsx resolves to /promo, which now returns 404, and the scanner reports a violation on the error page. Excluding deletions with --diff-filter=ACMRT handles the removal, but a rename needs both sides: intersect the resolved route list with the routes present in the built output before scanning, and drop anything that no longer exists.
  • Parameterised and data-driven routes. /product/[slug] cannot be visited without a value, and picking a random id makes the scan non-deterministic. Pin a representative value in the manifest — ideally a fixture record seeded by the test environment — and add a second entry for the interesting variant, such as a product with no image, since that is where an alt regression surfaces.
  • Changes with no import edge at all. Design tokens, a global stylesheet, a locale catalogue, a font file in public/, or a Content-Security-Policy header change can alter contrast or accessible names across the site while appearing in no module graph. These belong in fullScanTriggers as explicit paths; the graph will never find them, and a resolver that returns zero routes for a token change is the worst possible outcome.

Pipeline Impact

The runtime effect is the point of the exercise. On a 52-route application at roughly six seconds per route, a full scan is about five and a half minutes of scanning plus two minutes of install and build; a typical feature pull request resolving to four routes spends under thirty seconds scanning. The resolver itself adds ten to twenty seconds, mostly the static analysis pass, which is worth caching between runs keyed on the lockfile and the source tree hash.

Reporting has to become mode-aware or it will lie. A diff run that finds three violations has not established that the site has three violations, so a partial result must never be written to the baseline or the trend series — that is how a scoped scan gets misread as a large improvement and a threshold gets ratcheted down onto a number that was never measured. Tag every result with its mode, let only mode: full runs update the baseline, and keep the ratchet driven by scheduled runs as described in progressive threshold management.

The gate contract itself is unchanged, which is the whole design constraint. a11y-scope reports once per commit with one conclusion in both modes, so the ruleset needs no knowledge that scoping exists and no edit when the escalation threshold is tuned. Combined with the merge-queue run described in the parent guide — which always scans everything, because it runs once per merge rather than once per push — the result is fast feedback on pull requests and complete coverage at the point where the code actually enters the default branch.

Common Pitfalls

  • Deriving URLs from file paths by convention instead of a manifest, which quietly scans the wrong page whenever the router has a base path, a route group, or a rewrite.
  • Using a two-dot diff against the base branch tip, so a branch that has been open for two weeks resolves to everything other teams have changed since.
  • Running the scan without fetch-depth: 0, leaving no merge base and producing either an error or a diff of the entire tree.
  • Fanning the resolved routes into a job matrix, which makes the check name depend on the diff and breaks the required status check on the first pull request with a different route count.
  • Omitting the escalation trigger for design tokens and global CSS, so the widest-reaching changes in the codebase are gated by the narrowest possible scan.
  • Writing diff-mode violation counts into the baseline or dashboard, which fabricates an improvement trend out of reduced coverage.
  • Shipping the diff-aware gate without the scheduled full scan, leaving every route that nobody has edited recently permanently unverified.

FAQ

Is a diff-aware gate safe enough to be the only accessibility check on a pull request? Yes, provided two conditions hold: shared code escalates to a full scan, and a scheduled full scan runs at least nightly. Without escalation the gate is unsound, because the highest-impact changes get the smallest scope. Without the schedule it is incomplete, because nothing ever re-verifies a route that has not been edited. With both, the residual risk is a regression caused by a change the import graph cannot see, which is what the fullScanTriggers list exists to cover.

Why not shard the full scan across parallel jobs instead of narrowing it? Sharding and scoping solve different problems and compose well. Sharding cuts wall-clock time at the cost of runner minutes and needs the shard results merged before the gate can conclude; scoping cuts both time and minutes but only for narrow diffs. Most pipelines end up doing both — scope on pull requests, shard the scheduled full run — and the sharding mechanics are covered in the monorepo and parallel-sharding guides.

What happens when the import graph is wrong? It will be, in specific ways: dynamic import() with a computed path, a module resolved through a bundler alias the analyser does not know, or a component injected through a plugin registry. Each of those breaks an edge and shrinks the resolved set. Treat the graph as a heuristic that only ever adds routes, keep the explicit manifest authoritative for the paths that matter, and rely on the escalation triggers plus the nightly run rather than on the graph being complete.

Back to Pull Request Gating Branch Policies