Accessibility Debt Triage & Prioritization
The first full-site accessibility scan produces a number nobody planned for. A mid-size application with 400 routes and a decade of markup behind it will return somewhere between 3,000 and 20,000 findings, and the honest reaction to that report is that it is unreadable rather than actionable. This guide is part of Automated Remediation & Accessibility Fixing Patterns, and it deals with the half of remediation that no codemod solves: deciding which of those thousands of findings gets fixed first, who fixes it, and how the remaining pile is described to the people funding the work. A gate stops the bleeding — it prevents finding 3,001 — but it does nothing about the first three thousand, and a team that installs a gate without a triage model ends up with a permanently red baseline it has learned to ignore.
Problem Statement
The scanner’s report is organised for the scanner’s convenience, not for a work queue. axe-core emits an array of violations, each holding an array of nodes, grouped by rule; the same broken button appears under button-name on 34 routes as 34 separate nodes, and a single unreadable legacy admin table produces 190 findings in one place. Counted naively, that legacy table looks like the most urgent thing in the codebase and the checkout button looks like a rounding error. Counted by the number of users harmed, the ranking inverts.
The obvious sort key is the impact field, and it is the single most common triage mistake. impact rates the rule, not the instance. axe assigns critical to image-alt because a missing alternative text is categorically severe when it happens on content that matters; it has no idea whether this particular <img> is a decorative spacer on an internal report builder that eleven employees reach through a bookmark, or the only label on the primary call to action of the signup flow. A critical finding on an unreachable admin page matters less than a moderate colour-contrast finding on the checkout button, and any ordering that cannot express that will be overruled by the first engineer who reads it — after which the whole queue loses authority.
Four factors actually determine priority, and all four are outside the scanner’s knowledge. User-journey reach is how many real sessions traverse the route and whether the affected element sits on the critical path through it. Blocking-versus-degrading harm is whether the defect stops a user completing the task or merely makes it unpleasant: an unlabelled submit control with no accessible name blocks; a 4.2:1 contrast ratio on body copy degrades. Fix cost is the honest engineering estimate, because a queue that ignores cost stalls on the twelve findings that each need a design decision while forty one-line fixes wait behind them. Blast radius is how many other findings one change clears — the property that makes fixing a design-system component worth ten times fixing the same defect in a page.
Those four are also the only inputs that make a report legible to leadership. “We have 4,812 accessibility violations” invites no decision. “312 findings sit on the four journeys that carry 82% of sessions; 47 of them block task completion; 29 of those 47 are in six shared components and one team owns all six” is a staffing conversation with an obvious next move.
Key implementation targets:
- A normalised findings table with one stable row per (rule, route, element) pair, so the same defect keeps its identity across runs and can be counted, assigned and closed.
- A priority score built from journey reach, blocking-versus-degrading harm, fix cost and blast radius, with the raw inputs preserved next to the score so any ranking can be argued with.
- Priority bands (P1/P2/P3) rather than a 4,812-row ordering, because a queue is consumed in sprint-sized chunks and nobody negotiates over rank 703 versus rank 704.
- A grouping pass that collapses findings by owning component, exposing the small set of shared-component fixes that clear hundreds of rows.
- One tracker issue per component group, deduplicated by fingerprint, routed to a named owning team rather than a shared triage backlog.
- A weekly regeneration job that reports the queue’s movement, and a quarterly report expressed in journeys and blocking defects rather than in raw counts.
Prerequisites
1. Normalising the Raw Report into a Findings Table
Everything downstream depends on a finding having a stable identity. If a row’s id changes between runs, the queue cannot tell a fixed finding from a moved one, progress reporting becomes impossible, and every regenerated issue is a duplicate. The identity has to come from something that survives a re-render: the rule id, the route, and a selector path that is not positional. axe’s node.target is an array of CSS selectors — one per frame — and it is the best available anchor, but it degrades badly when the DOM has no ids and axe falls back to :nth-child chains. Where the application can emit a component marker, prefer it; where it cannot, accept the churn and handle it in the review step.
The normaliser flattens the nested report into newline-delimited JSON, one object per (rule, node) pair. NDJSON rather than a single JSON array is deliberate: a 12,000-row array is awkward to diff in a pull request and awkward to stream, while NDJSON sorts, greps and diffs with ordinary tools.
// a11y/triage/normalize.mjs
// Usage: node a11y/triage/normalize.mjs artifacts/scan/*.json > triage/findings.ndjson
import { readFileSync } from 'node:fs';
import { createHash } from 'node:crypto';
// Identity = rule + route + selector path. Stable across runs unless the DOM moves.
function fingerprint(rule, route, target) {
return createHash('sha1').update(`${rule}|${route}|${target}`).digest('hex').slice(0, 12);
}
// Positional selectors churn on every reorder; flag them instead of trusting them.
const POSITIONAL = /:nth-(child|of-type)\(/;
for (const file of process.argv.slice(2)) {
const report = JSON.parse(readFileSync(file, 'utf8'));
const route = new URL(report.url).pathname;
for (const violation of report.violations) {
for (const node of violation.nodes) {
const target = node.target.flat().join(' >> '); // flat() handles iframe paths
process.stdout.write(
JSON.stringify({
id: fingerprint(violation.id, route, target),
rule: violation.id,
// Node impact beats rule impact: axe downgrades some nodes per instance.
impact: node.impact ?? violation.impact,
route,
target,
unstableAnchor: POSITIONAL.test(target),
component: node.html.match(/data-component="([^"]+)"/)?.[1] ?? null,
html: node.html.slice(0, 200), // enough to recognise, small enough to store
wcag: violation.tags.filter((t) => /^wcag\d/.test(t)),
seenAt: report.timestamp,
}) + '\n',
);
}
}
}
Two fields in that output earn their place. unstableAnchor marks rows whose identity will probably change on the next run, which is the difference between a queue that reports honest progress and one that reports noise; the review step in section 5 treats a spike in unstable anchors as a data-quality bug rather than as new debt. component reads a build-time marker, which is the cheapest reliable bridge from a DOM finding back to source — the same marker that ownership routing depends on, worked through in assigning violation ownership with CODEOWNERS.
Deduplicate before scoring, not after. Two overlapping rules frequently report the same node: link-name and image-alt both fire on an image-only link with no alternative text, and aria-required-attr plus a custom combobox rule both fire on the same broken widget. Left in, that node inflates its component’s apparent blast radius and pulls the component up the queue for the wrong reason. Collapse rows that share a route, a target and a WCAG success criterion, keep the highest-impact rule as the label, and record the collapsed rule ids in a alsoReportedBy array so the fixer knows both gates have to go green.
2. Computing the Priority Score
The score exists to produce an ordering that survives an argument. That constrains its design more than accuracy does: it must be recomputable from checked-in inputs, it must be coarse enough that small input errors do not reorder the top of the queue, and every row must carry its inputs alongside its result. A score whose derivation is invisible gets treated as an opinion, and the first team to disagree with its own P1 will simply reorder locally.
The model here is deliberately ordinal. Reach comes from the route-reach export bucketed into three levels; harm is binary; cost is a divisor with three levels; blast radius is a multiplier capped so that one enormous component cannot dominate the whole queue. Keep the levels and the bucket boundaries in a11y/triage/weights.json — a checked-in, reviewed file — so that changing them is a pull request with a diff rather than an edit to a script somebody ran on a laptop. The companion journeys.json holds two arrays of route paths, primary and secondary, derived from the reach export in the same review.
{
"reach": { "primaryJourney": 5, "secondary": 2, "internalOnly": 1 },
"harm": { "blocksTask": 3, "degradesTask": 1 },
"cost": { "oneLine": 1, "componentChange": 2, "needsDesign": 4 },
"blastRadiusCap": 8,
"bands": { "p1": 12, "p2": 4 },
"reviewedBy": "a11y-guild",
"reviewedOn": "2026-07-06"
}
The scorer applies that file and emits bands. Note that it never discards an input: factors travels with every row so the tracker issue, the dashboard and the arguing engineer all see the same derivation. The full weighted variant — normalised traffic weights, an estimated fix-size field and a ranked table with the inputs printed next to the score — is developed in scoring accessibility violations by user impact; this coarser version is what a first triage pass needs on day one.
// a11y/triage/band.mjs
// Usage: node a11y/triage/band.mjs triage/findings.ndjson > triage/queue.ndjson
import { readFileSync } from 'node:fs';
const w = JSON.parse(readFileSync('a11y/triage/weights.json', 'utf8'));
const journeys = JSON.parse(readFileSync('a11y/triage/journeys.json', 'utf8'));
// Rules whose failure removes the only path to completing the task.
const BLOCKING_RULES = new Set([
'button-name', 'link-name', 'label', 'select-name', 'input-button-name',
'aria-required-attr', 'frame-title', 'aria-hidden-focus',
]);
const COST = { 'color-contrast': 'needsDesign', 'image-alt': 'oneLine' };
const rows = readFileSync(process.argv[2], 'utf8').trim().split('\n').map(JSON.parse);
// Blast radius: how many other rows this component+rule pair would clear at once.
const radius = new Map();
for (const r of rows) {
const key = `${r.component ?? r.route}|${r.rule}`;
radius.set(key, (radius.get(key) ?? 0) + 1);
}
for (const r of rows) {
const tier = journeys.primary.includes(r.route)
? 'primaryJourney'
: journeys.secondary.includes(r.route) ? 'secondary' : 'internalOnly';
const harm = BLOCKING_RULES.has(r.rule) ? 'blocksTask' : 'degradesTask';
const cost = COST[r.rule] ?? 'componentChange';
const spread = Math.min(radius.get(`${r.component ?? r.route}|${r.rule}`), w.blastRadiusCap);
const score = (w.reach[tier] * w.harm[harm] * Math.sqrt(spread)) / w.cost[cost];
console.log(JSON.stringify({
...r,
factors: { tier, harm, cost, spread }, // inputs travel with the result
score: Number(score.toFixed(2)),
band: score >= w.bands.p1 ? 'P1' : score >= w.bands.p2 ? 'P2' : 'P3',
}));
}
Math.sqrt(spread) is the one non-obvious choice. Blast radius should raise priority, but linearly it swamps everything else: a component with 190 instances would outrank a blocking defect on the payment button by two orders of magnitude. A square root keeps a 34-instance component ahead of a 4-instance one while leaving reach and harm as the dominant terms. The same reasoning applies to the blastRadiusCap — one pathological legacy table should not define the top of the queue.
Three properties of this table are worth stating explicitly, because they are what teams get wrong when they build a score for the first time.
| Factor | Source of truth | Refresh cadence | Failure when omitted |
|---|---|---|---|
| Journey reach | Analytics export, route manifest | Quarterly | Internal tools outrank revenue paths |
| Blocking vs degrading | Reviewed rule list, specialist sign-off | On rule-set change | Contrast noise buries missing labels |
| Fix cost | Rule class plus fixer estimate | Per grooming session | Queue stalls behind design decisions |
| Blast radius | Computed from the findings table | Every run | Component fixes look like page fixes |
3. Grouping by Owning Component to Find High-Leverage Fixes
The single most useful transformation of a large findings table is the one that stops treating rows as independent. In a component-based application most findings are not distinct defects; they are one defect observed many times. A Button variant that renders an icon with no accessible name produces a button-name finding on every route that renders it, and the report shows 34 problems where there is one. Fix the component and 34 rows close in a single pull request that touches nine lines.
Grouping therefore serves two purposes at once. It compresses the queue into something a person can read — 4,812 rows become roughly 180 groups — and it surfaces the leverage ordering, which is usually the fastest route to a large numeric drop that buys political room for the slower work. Group on the pair (component, rule) rather than on the component alone: a DataTable can simultaneously have a missing caption defect that is one fix and a per-instance header-scope problem that is genuinely 40 separate content fixes, and collapsing those into one group hides the difference.
// a11y/triage/group.mjs
// Usage: node a11y/triage/group.mjs triage/queue.ndjson > triage/groups.md
import { readFileSync } from 'node:fs';
const rows = readFileSync(process.argv[2], 'utf8').trim().split('\n').map(JSON.parse);
const groups = new Map();
for (const r of rows) {
// Rows without a component marker group by route: they are page-local fixes.
const key = `${r.component ?? `route:${r.route}`}|${r.rule}`;
const g = groups.get(key) ?? {
component: r.component ?? `route:${r.route}`,
rule: r.rule,
band: r.band,
routes: new Set(),
count: 0,
topScore: 0,
};
g.count += 1;
g.routes.add(r.route);
g.topScore = Math.max(g.topScore, r.score);
// A group inherits the highest band any of its rows reached.
if (r.band === 'P1' || (r.band === 'P2' && g.band === 'P3')) g.band = r.band;
groups.set(key, g);
}
const ranked = [...groups.values()].sort(
(a, b) => b.topScore - a.topScore || b.count - a.count,
);
console.log('| Component | Rule | Findings | Routes | Band | Top score |');
console.log('|---|---|---|---|---|---|');
for (const g of ranked.slice(0, 40)) {
console.log(
`| ${g.component} | ${g.rule} | ${g.count} | ${g.routes.size} ` +
`| ${g.band} | ${g.topScore.toFixed(1)} |`,
);
}
const singles = ranked.filter((g) => g.count === 1).length;
console.error(`${ranked.length} groups, ${singles} one-off findings, ${rows.length} rows`);
The console.error line matters more than it looks. The ratio of groups to rows is the leverage metric for the whole backlog: a codebase where 4,812 rows collapse into 180 groups has a tractable problem, and one where they collapse into 3,900 groups has a markup problem that no component fix will solve and that needs a different plan — usually a per-directory budget of the kind described in progressive threshold management, applied so legacy areas stop getting worse while new work is held to the full standard.
There is a trap in leverage-first ordering, and it is worth naming before the section on scheduling. A shared-component fix is cheap in rows and expensive in coordination: it lands in a package that a dozen applications consume, it needs a version bump and a rollout, and it can regress visual snapshots in products whose teams never heard of the accessibility programme. Budget for that rollout as part of the fix rather than treating the merge as the finish line, and confirm each cleared group with a re-scan rather than an assumption — the verification discipline the rest of automated remediation and accessibility fixing patterns applies to codemods applies equally here.
4. Routing to Owners and Issue Trackers
A queue nobody owns is a report. The transition from report to work happens when a group becomes a tracker issue with a named team on it, and the mechanics of that step decide whether the programme survives its first quarter. Two rules do most of the work: create issues per group, never per finding, and never create an issue whose owner is a shared bucket.
Per-group issues keep the tracker usable. Filing 4,812 issues destroys the tracker for every other purpose, generates 4,812 notifications, and guarantees that the eventual clean-up is itself a project. One issue per (component, rule) group carries the same information in 180 items, each with a checklist of affected routes and a fingerprint list in the body so the next run can recognise it.
// a11y/triage/file-issues.mjs
// Usage: TRIAGE_TOKEN=… node a11y/triage/file-issues.mjs triage/queue.ndjson
// Input rows must already carry an `owner` field from the ownership pass.
import { readFileSync } from 'node:fs';
import { execFileSync } from 'node:child_process';
const REPO = 'acme/storefront';
const rows = readFileSync(process.argv[2], 'utf8').trim().split('\n').map(JSON.parse);
const p1p2 = rows.filter((r) => r.band !== 'P3'); // P3 rides along with other work
// Existing issues carry their group key in the title, so matching is exact.
const open = JSON.parse(
execFileSync('gh', [
'issue', 'list', '--repo', REPO, '--label', 'a11y-debt',
'--state', 'open', '--limit', '500', '--json', 'title',
]).toString(),
).map((i) => i.title);
const groups = new Map();
for (const r of p1p2) {
const key = `${r.component ?? `route:${r.route}`} / ${r.rule}`;
const g = groups.get(key) ?? { rows: [], band: r.band, owner: r.owner };
g.rows.push(r);
if (r.band === 'P1') g.band = 'P1';
groups.set(key, g);
}
for (const [key, g] of groups) {
const title = `a11y debt: ${key}`;
if (open.includes(title)) continue; // idempotent: re-runs add nothing
if (!g.owner) {
// No owner is a routing bug, not a low-priority ticket. Fail the job.
console.error(`unowned group: ${key} (${g.rows.length} findings)`);
process.exitCode = 1;
continue;
}
const body = [
`**Band:** ${g.band} · **Owner:** ${g.owner} · **Findings:** ${g.rows.length}`,
`**Blocks task completion:** ${g.rows[0].factors.harm === 'blocksTask' ? 'yes' : 'no'}`,
'',
'### Affected routes',
...[...new Set(g.rows.map((r) => r.route))].map((x) => `- [ ] ${x}`),
'',
'### Fingerprints (do not edit — used to close this issue automatically)',
'```',
...g.rows.map((r) => r.id),
'```',
].join('\n');
execFileSync('gh', [
'issue', 'create', '--repo', REPO, '--title', title, '--body', body,
'--project', 'Accessibility debt',
// Repeated --label flags: one for the programme, one for the band, one for
// the owning team, so every tracker filter people actually use is covered.
'--label', 'a11y-debt',
'--label', g.band.toLowerCase(),
'--label', `owner:${g.owner}`,
]);
}
The unowned case deliberately sets a non-zero exit code. An “unowned” or “needs triage” label is where findings go to be forgotten: it has no standup, no capacity and no one who feels bad when it grows. Failing the routing job forces a human to either extend CODEOWNERS or add a route-to-package mapping, and that five-minute fix is what keeps the queue trustworthy. Shared components need a further distinction — the owner of a Button defect is the design-system team even though the finding was observed on a marketing page — and the resolution order that gets that right is the subject of the ownership guide linked in section 1.
Set a work-in-progress cap when the issues are created. Ten open P1 issues per team is a plan; sixty is a wish, and a team facing sixty will pick by convenience rather than by band, which silently discards the whole scoring exercise. Hold the surplus in the queue file and let the weekly job promote replacements as issues close.
5. The Recurring Review
Triage is not a project with an end date; it is a job that runs on a schedule and a meeting that reads its output. Regenerate the queue weekly from a fresh full scan, diff it against last week’s, and publish three numbers: groups closed, groups opened, and P1 groups still open past their target sprint. Anything more granular is for the people doing the work, not for the review.
name: a11y-debt-triage
on:
schedule:
- cron: '0 5 * * 1' # Monday 05:00 UTC, before the weekly planning meeting
workflow_dispatch:
permissions:
contents: write
issues: write
jobs:
regenerate-queue:
runs-on: ubuntu-24.04
timeout-minutes: 45
steps:
- uses: actions/checkout@v4
- uses: actions/setup-node@v4
with:
node-version: '20'
cache: npm
- run: npm ci
- name: Full-coverage scan of every route in the manifest
run: npm run scan:all -- --out artifacts/scan # writes one JSON per route
- name: Normalise, band and group
run: |
node a11y/triage/normalize.mjs artifacts/scan/*.json > triage/findings.ndjson
node a11y/triage/band.mjs triage/findings.ndjson > triage/queue.ndjson
node a11y/triage/group.mjs triage/queue.ndjson > triage/groups.md
- name: Diff against last week and write the review summary
run: node a11y/triage/weekly-diff.mjs triage/queue.ndjson >> "$GITHUB_STEP_SUMMARY"
- name: File or refresh tracker issues
env:
TRIAGE_TOKEN: ${{ secrets.TRIAGE_TOKEN }}
run: node a11y/triage/file-issues.mjs triage/queue.ndjson
- name: Commit the queue so its history is reviewable
run: |
git config user.name 'a11y-triage-bot'
git config user.email 'a11y-triage-bot@users.noreply.github.com'
git add triage/
git diff --quiet --cached || git commit -m 'chore(a11y): weekly triage queue'
git push
- uses: actions/upload-artifact@v4
if: always()
with:
name: a11y-triage-queue
path: triage/
retention-days: 90
Committing the queue is what makes the history auditable. git log -p triage/queue.ndjson answers “when did this finding first appear and when did it close” without a database, and the 90-day artifact retention covers the same question for the raw reports. Teams that want charts rather than diffs push the same rows into the store described in reporting dashboards and violation tracking, which is also where the two-curve view — prevented versus remaining — belongs; the mechanics of that longitudinal measurement, and of setting quarter targets from measured throughput, are in burning down an accessibility backlog across quarters.
The review meeting needs one standing agenda item that is not a number: which groups were deferred, and why. A deferral with a written reason (“blocked on the Q4 design-token migration”) is a legitimate engineering decision and stays in the queue with a date. A deferral without one is the beginning of a queue nobody believes in, and the second time a P1 group is silently rolled forward, the band has stopped meaning anything.
Pipeline Integration
Triage and gating are separate jobs with separate exit-code contracts, and conflating them is the fastest way to break both. The gate runs on pull requests, scans only what changed, and exits non-zero on a new blocking violation. The triage job runs on a schedule, scans everything, and exits zero even when it finds 4,812 rows — because the backlog existing is not a build failure. The one exception is the routing step: an unowned group exits non-zero, since that is a configuration defect the pipeline can actually demand somebody fix.
Artifacts flow one way. The triage job publishes triage/queue.ndjson as a build artifact and commits it to the repository; the gate job reads the committed file to answer “is this finding already known”, which is what lets a pull-request gate block only new findings while a legacy page still has forty old ones. That baseline handoff is the join between this guide and the threshold ratchet: the queue is the list of exceptions, and the ratchet is the mechanism that shrinks the allowance as groups close. Keep the two files in the same commit so a fix that closes a group and lowers a budget arrives atomically.
Pull-request annotations should quote the band, not the raw score. A reviewer seeing P1 · blocks task completion · Button/button-name · 34 findings has enough to make a merge decision; a reviewer seeing score 14.7 has to go and read a script. Where the tracker and the pipeline are joined, close issues from the pipeline rather than by hand: when a scheduled run finds that every fingerprint listed in an open issue is absent, post the re-scan link and close it. That automated close is the whole reason the fingerprint block sits in the issue body.
Troubleshooting and Flaky-Test Mitigation
The queue regenerates with mostly new ids every week. Positional selectors are the cause: node.target fell back to :nth-child chains and any reorder rewrites the fingerprint. Check the proportion of rows with unstableAnchor: true; above roughly 15% the queue’s identity is unreliable. The fix is to emit a component marker in non-production builds so the anchor is semantic, and in the meantime to group and report at the (component, rule) level, which is stable even when individual rows churn.
Findings appear and disappear between identical runs. These are timing artifacts, not debt: a scan that fired before hydration finished, a lazily rendered popup, a virtualised list that painted a different window of rows. Feeding them into the queue produces phantom groups and phantom progress. Require a finding to appear in at least two of the last three scheduled runs before it is banded, and route single-appearance findings to a flaky file for investigation of the scan, not of the markup.
One legacy area dominates every band. A single unmaintained admin surface with 190 findings will fill P2 and starve everything else, and the honest answer is usually that it should not be in the same queue. Give it its own directory budget, freeze it at its current count so it cannot get worse, and score it separately; otherwise every weekly review is a conversation about a surface nobody plans to modernise this year.
P1 count rises but no regression happened. Almost always a coverage change: routes were added to the manifest, a new locale was scanned, or a rule was enabled. Record the number of scanned routes and the rule-set version alongside each run, and report coverage growth as its own line rather than letting it contaminate the debt trend. A backlog that grows because you started looking harder is good news reported badly.
Two teams both claim a group and neither fixes it. This is the shared-component case surfacing as a scheduling failure. Resolve it by rule, not by negotiation: if the defect reproduces in the component’s own test fixture, it belongs to the component’s owner; if it only reproduces with the consumer’s props or content, it belongs to the consumer. Write the outcome into the routing config so the same argument does not recur next quarter.
The score is quietly ignored. The symptom is teams working their P2s while P1s sit open. The cause is nearly always an input nobody believes — a reach figure from a two-year-old analytics export, or a blocksTask list that classifies contrast failures as blocking. Re-derive the inputs in front of the team once, in a review, and either the objection is real and the weights file changes, or the ordering survives with the team’s agreement. Do not defend a score by pointing at the script.
Common Pitfalls
- Sorting the queue on axe’s
impactfield and calling it triage, which ranks a decorative image on an internal tool above an unlabelled payment control. - Filing one issue per finding, which floods the tracker and guarantees the cleanup becomes its own project.
- Creating an “unowned” or “needs triage” label as the destination for unroutable findings, which is a queue with no standup, no capacity and no advocate.
- Scoring before deduplicating overlapping rules, so a node reported by two rules doubles its component’s apparent blast radius.
- Multiplying by blast radius linearly, which lets one 190-instance legacy table outrank every blocking defect in the product.
- Keeping the weights in a script instead of a reviewed file, so the ranking cannot be audited and any disagreement becomes a conversation about somebody’s code.
- Closing a group when the pull request merges rather than when a re-scan confirms the fingerprints are gone, which is how “fixed” counts drift away from reality.
- Letting the triage job’s exit code fail the build on backlog size, which trains everyone to ignore the job that was supposed to make the backlog visible.
- Reporting a single total to leadership, which hides a regression whenever the number of new findings and the number of fixed findings move together.
FAQ
Is the impact field useless for prioritisation?
No — it is a useful input and a bad sort key. impact encodes real expertise about how severe a rule’s failure mode is in general, which is exactly what you want when deciding whether a rule blocks the gate at all. What it cannot express is anything about this instance: the route it sits on, whether the element is on the task path, or how many places share the defect. Use it to set the gate’s blocking threshold, and use the four contextual factors to order the backlog.
How precise do the reach numbers need to be? Much less precise than teams assume. The model only needs to distinguish “on a primary journey”, “on a secondary journey” and “internal or rarely reached”, which a 90-day pageview export answers unambiguously for the overwhelming majority of routes. Chasing per-route session percentages adds decimal places and no reordering, and it makes the score depend on an analytics pipeline that changes more often than the route manifest does.
What happens to findings the scanner reports as incomplete?
Keep them out of the debt queue and in their own list. incomplete means axe could not determine an answer — a collapsed disclosure, a closed shadow root, an element whose contrast could not be computed over an image — so banding them produces work items that may not be defects at all. Review them in the same weekly meeting as a separate agenda item, and promote any that a human confirms into the findings table as ordinary rows.
Should the backlog block releases? The backlog should not, and the gate should. Blocking releases on a pre-existing 4,812-row baseline stops all delivery and gets the gate disabled within a week; blocking on newly introduced blocking violations is enforceable from day one. The bridge between the two positions is a shrinking allowance per area rather than a single global switch, which is why the queue and the threshold ratchet are designed to read the same committed baseline file.
How is progress reported to leadership without hiding a regression? Report two counts and one ratio: blocking findings remaining on primary journeys, new findings introduced since the gate went live, and the share of groups with a named owner. The first should fall, the second should stay at zero, and the third should approach 100% quickly because it costs configuration rather than engineering. A single combined total is the failure mode — twelve fixes and twelve new regressions net to zero movement and read as a stalled programme rather than as an urgent gate problem.
Related
- Scoring Accessibility Violations by User Impact — the weighted formula, its inputs and the ranked table it emits.
- Assigning Violation Ownership with CODEOWNERS — mapping a DOM node to a source path to a team, and failing loudly when it cannot.
- Burning Down an Accessibility Backlog Across Quarters — the two curves, throughput-derived targets and the exit criterion.
- Progressive Threshold Management — how to ratchet the allowance down without blocking legacy work.
- Reporting Dashboards & Violation Tracking — where queue rows become trends an engineering manager reads weekly.