# Exporting Accessibility Results to Compliance Dashboards

The person who signs an accessibility conformance statement does not read violation counts. They read a list of WCAG 2.2 success criteria with a status against each one, and they need to know which of those statuses came from a machine and which came from a human. This guide is part of Reporting, Dashboards & Violation Tracking, and it builds the one export that turns scan output into that list — including the column most exports omit, which is the one recording that a criterion was never automatically checked at all.

The export is small: one row per success criterion in scope, a status, the rules that produced it, and the run id that proves it. Getting it right is entirely a matter of what the statuses are allowed to say.

# Root Cause

axe-core does not emit success criteria. It emits tags, and the criteria are packed inside them alongside three other kinds of label. A single rule’s tags array might read ["cat.color", "wcag2aa", "wcag143", "TTv5", "ACT"]: cat.color is a category for grouping in the docs, wcag2aa is a conformance level, wcag143 is the criterion 1.4.3, and TTv5 and ACT are cross-references to other test suites. Three of those five are noise for this purpose, and the two that matter need different parsers — the criterion form is digits only, the level form always contains letters. A dashboard fed the raw tag array cannot answer “show me every failure of SC 1.4.3 last quarter”, because the criterion it wants is spelled wcag143 and buried in an array.

The harder problem is the shape of the result. A findings-first export — group the violations by criterion and print the groups — produces a document that lists only failures, and any criterion absent from that list reads as satisfied. That inference is false, and it is false in two different ways. Some criteria have no automated rule in the enabled set at all: nothing checked SC 2.4.3 (Focus Order) or SC 1.4.11 (Non-text Contrast), so their absence from the failure list means nothing whatsoever. Others have a rule that checks a necessary condition rather than the criterion: axe can prove an img has an alt attribute, which is required by SC 1.1.1 (Non-text Content), but it has no way to judge whether the text describes the image, so “no failure” is at best “not disproved”.

This is the concrete form of the well-known statement that automated scanners reliably detect only 30–40% of WCAG failures. Counted by criterion rather than by instance, the picture is starker: of the 55 Level A and AA success criteria in WCAG 2.2, a typical enabled axe rule set carries criterion tags for around twenty, and only a handful of those are decided end to end by a machine.

So the export has to be criterion-first: start from an inventory of every criterion in scope, left-join the findings onto it, and give the join a status vocabulary honest enough that a criterion nobody checked cannot be mistaken for one that passed.

# Configuration

Two inputs and one script. The first input is the criterion inventory: the 55 Level A and AA criteria, each with a hand-declared judgement of how far automation can go. That judgement is a human decision — it belongs in version control, it gets reviewed, and the validation step below fails the build when it drifts out of step with the rules actually running.

The file is CSV with title last, because several criterion titles contain commas and putting the free-text field at the end means the parser never needs quoting rules.

# a11y/compliance/criteria.csv — WCAG 2.2 Level A and AA, the criteria in scope.
# automation: mechanical = a rule decides the criterion end to end
#             partial    = a rule checks one necessary condition only
#             none       = no rule in the enabled set tags this criterion
criterion,level,automation,title
1.1.1,A,partial,Non-text Content
1.2.1,A,none,Audio-only and Video-only (Prerecorded)
1.2.2,A,none,Captions (Prerecorded)
1.2.3,A,none,Audio Description or Media Alternative (Prerecorded)
1.3.1,A,partial,Info and Relationships
1.3.2,A,none,Meaningful Sequence
1.3.3,A,none,Sensory Characteristics
1.4.1,A,none,Use of Color
1.4.2,A,partial,Audio Control
2.1.1,A,partial,Keyboard
2.1.2,A,none,No Keyboard Trap
2.1.4,A,none,Character Key Shortcuts
2.2.1,A,partial,Timing Adjustable
2.2.2,A,partial,Pause Stop Hide
2.3.1,A,none,Three Flashes or Below Threshold
2.4.1,A,partial,Bypass Blocks
2.4.2,A,partial,Page Titled
2.4.3,A,none,Focus Order
2.4.4,A,partial,Link Purpose (In Context)
2.5.1,A,none,Pointer Gestures
2.5.2,A,none,Pointer Cancellation
2.5.3,A,mechanical,Label in Name
2.5.4,A,none,Motion Actuation
3.1.1,A,mechanical,Language of Page
3.2.1,A,none,On Focus
3.2.2,A,none,On Input
3.2.6,A,none,Consistent Help
3.3.1,A,none,Error Identification
3.3.2,A,partial,Labels or Instructions
3.3.7,A,none,Redundant Entry
4.1.2,A,partial,Name Role Value
1.2.4,AA,none,Captions (Live)
1.2.5,AA,none,Audio Description (Prerecorded)
1.3.4,AA,partial,Orientation
1.3.5,AA,partial,Identify Input Purpose
1.4.3,AA,mechanical,Contrast (Minimum)
1.4.4,AA,partial,Resize Text
1.4.5,AA,none,Images of Text
1.4.10,AA,none,Reflow
1.4.11,AA,none,Non-text Contrast
1.4.12,AA,partial,Text Spacing
1.4.13,AA,none,Content on Hover or Focus
2.4.5,AA,none,Multiple Ways
2.4.6,AA,none,Headings and Labels
2.4.7,AA,none,Focus Visible
2.4.11,AA,none,Focus Not Obscured (Minimum)
2.5.7,AA,none,Dragging Movements
2.5.8,AA,partial,Target Size (Minimum)
3.1.2,AA,partial,Language of Parts
3.2.3,AA,none,Consistent Navigation
3.2.4,AA,none,Consistent Identification
3.3.3,AA,none,Error Suggestion
3.3.4,AA,none,Error Prevention (Legal Financial Data)
3.3.8,AA,none,Accessible Authentication (Minimum)
4.1.3,AA,none,Status Messages

The second input is the normalised findings stream produced in the parent guide, plus any one raw scan file. The raw file is needed for a reason that is easy to miss: the only way to know which criteria the rule set claims to check is to enumerate every rule that executed, which means reading passes and inapplicable as well as violations. A rule that matched nothing still proves it ran.

// a11y/compliance/export.mjs
// usage: node a11y/compliance/export.mjs findings.ndjson axe-raw/home.json \
//          a11y/compliance/criteria.csv
import { readFileSync, writeFileSync } from 'node:fs';

const [ndjson, sampleReport, inventoryPath] = process.argv.slice(2);
const CRITERION = /^wcag(\d)(\d)(\d+)$/;   // wcag143 -> 1.4.3, wcag1412 -> 1.4.12

// --- inventory: title is the last column so a comma in it needs no quoting.
const inventory = readFileSync(inventoryPath, 'utf8').split('\n')
  .filter((l) => l.trim() && !l.startsWith('#') && !l.startsWith('criterion,'))
  .map((line) => {
    const [criterion, level, automation, ...rest] = line.split(',');
    return { criterion, level, automation, title: rest.join(',').trim() };
  });

// --- which criteria does the enabled rule set claim to check? Every rule that
// executed counts, including the ones that matched nothing.
const sample = JSON.parse(readFileSync(sampleReport, 'utf8'));
const executed = [...sample.violations, ...sample.passes,
                  ...sample.incomplete, ...sample.inapplicable];
const covering = new Map();                // criterion -> Set of rule ids
for (const rule of executed) {
  for (const tag of rule.tags) {
    const m = CRITERION.exec(tag);
    if (!m) continue;
    const sc = `${m[1]}.${m[2]}.${m[3]}`;
    if (!covering.has(sc)) covering.set(sc, new Set());
    covering.get(sc).add(rule.id);
  }
}

// --- findings, bucketed per criterion and split by outcome.
const rows = readFileSync(ndjson, 'utf8').split('\n').filter(Boolean).map(JSON.parse);
const run = rows[0] ?? {};
const failed = new Map();
const unresolved = new Map();
for (const r of rows) {
  const bucket = r.status === 'violation' ? failed : unresolved;
  for (const sc of r.criteria) {
    if (!bucket.has(sc)) bucket.set(sc, { nodes: 0, rules: new Set(), routes: new Set() });
    const e = bucket.get(sc);
    e.nodes += 1;
    e.rules.add(r.ruleId);
    e.routes.add(r.route);
  }
}

function statusFor(entry) {
  if (failed.has(entry.criterion)) return 'fail';
  // No rule looked. Absence of a failure carries no information at all.
  if (entry.automation === 'none') return 'not-tested';
  // A rule looked and could not decide: a human has to finish the job.
  if (unresolved.has(entry.criterion)) return 'needs-review';
  // A rule looked and found nothing. Only a criterion a rule decides end to
  // end may be reported as passing on that basis.
  return entry.automation === 'mechanical' ? 'pass-automated' : 'no-automated-failure';
}

const HEADER = ['criterion', 'level', 'automation', 'status', 'failing_nodes',
  'failing_routes', 'failing_rules', 'covering_rules', 'evidence_run', 'commit',
  'scanner', 'ruleset_hash', 'scanned_at', 'title'];

const out = inventory.map((entry) => {
  const f = failed.get(entry.criterion);
  return {
    criterion: entry.criterion,
    level: entry.level,
    automation: entry.automation,
    status: statusFor(entry),
    failing_nodes: f ? f.nodes : 0,
    failing_routes: f ? f.routes.size : 0,
    failing_rules: f ? [...f.rules].sort().join(' ') : '',
    covering_rules: [...(covering.get(entry.criterion) ?? [])].sort().join(' '),
    evidence_run: run.runId ?? 'no-run',
    commit: run.commit ?? 'no-commit',
    scanner: `axe-core ${run.axeVersion ?? 'unknown'}`,
    ruleset_hash: run.rulesetHash ?? 'unknown',
    scanned_at: run.startedAt ?? 'unknown',
    title: entry.title,
  };
});

// CSV for the spreadsheet, JSON for whatever reads the dashboard.
const csv = [HEADER.join(',')].concat(
  out.map((r) => HEADER.map((k) => String(r[k]).replaceAll(',', ';')).join(',')),
).join('\n');
writeFileSync('compliance-export.csv', csv + '\n');
writeFileSync('compliance-export.json', JSON.stringify({
  generatedAt: new Date().toISOString(),
  scope: 'WCAG 2.2 Level A and AA',
  evidenceRun: run.runId ?? 'no-run',
  criteria: out,
}, null, 2));

const tally = {};
for (const r of out) tally[r.status] = (tally[r.status] ?? 0) + 1;
console.log(Object.keys(tally).sort().map((k) => `${k}=${tally[k]}`).join(' '));

Five statuses, and the difference between the last three is the entire value of the document:

Every row carries evidence_run, and that column is what separates a report from an assertion. The run id resolves in the findings store to a commit, a scanner version, a ruleset hash and a route count, so a reviewer six months later can establish exactly what was scanned. A row without it is a number somebody typed.

Run the export from a release workflow rather than on every pull request. A conformance snapshot is a per-release artifact, and generating one per commit produces a hundred contradictory documents a week.

- name: Export the criterion rollup
  if: always()                 # a failing gate still owes a compliance snapshot
  run: |
    node a11y/compliance/export.mjs findings.ndjson axe-raw/home.json \
      a11y/compliance/criteria.csv
- name: Retain the snapshot with the release
  if: always()
  uses: actions/upload-artifact@v4
  with:
    name: compliance-snapshot-${{ github.sha }}
    path: |
      compliance-export.csv
      compliance-export.json
      a11y/compliance/criteria.csv    # the judgement calls as reviewed at this sha
    retention-days: 90

# Validation

The export is correct when it has exactly one row per criterion in scope and when the inventory’s judgement calls still match the rules that ran. Both are shell assertions, and the second is the one that catches real rot: somebody enables a new rule tagged wcag249, and a criterion the inventory still calls none silently starts being checked.

node a11y/compliance/export.mjs findings.ndjson axe-raw/home.json \
  a11y/compliance/criteria.csv

# 1. One header plus 55 criteria: nothing dropped by a parsing slip.
test "$(wc -l < compliance-export.csv)" -eq 56

# 2. Drift both ways: not-tested rows must have no covering rule, and every
#    criterion the inventory claims coverage for must have one.
node -e "
const rows = require('./compliance-export.json').criteria;
const stale = rows.filter(r => r.status === 'not-tested' && r.covering_rules);
const orphan = rows.filter(r => r.automation !== 'none' && !r.covering_rules);
if (stale.length) {
  console.error('inventory says not-tested but a rule covers: ' +
    stale.map(r => r.criterion).join(' '));
}
if (orphan.length) {
  console.error('inventory claims coverage but no rule ran for: ' +
    orphan.map(r => r.criterion).join(' '));
}
if (stale.length || orphan.length) process.exit(1);
console.log('inventory matches the enabled rule set');
"

# 3. Every row must carry provenance.
! grep -q ',no-run,' compliance-export.csv

A healthy run prints the status tally followed by the drift check. The tally is the sentence to read aloud in a compliance meeting: three criteria are demonstrably satisfied by machine, thirteen more had a rule run without finding anything, and thirty-five were never automatically checked at all.

fail=4 needs-review=2 no-automated-failure=11 not-tested=35 pass-automated=3
inventory matches the enabled rule set

# Edge Cases and Conditional Guards

• Custom rules with criterion tags. A rule authored in-house is only visible to this export if its tags array carries the criterion in axe’s own wcagNNN form; a tag like sc-4.1.2 parses to nothing and the criterion stays not-tested. When adding coverage through custom rules and context-aware testing, tag the rule with wcag412 and update the inventory’s automation value in the same commit, or the drift check will fail the build — which is the intended behaviour.
• Partial scans. A run that only reached eight of forty routes produces far fewer failures, and every criterion it never visited quietly becomes no-automated-failure. Refuse to export from a run whose scan_status is not complete; a snapshot from a partial scan is worse than no snapshot, because it looks authoritative.
• Criteria that are out of scope rather than untested. A product with no audio or video does not need SC 1.2.2 (Captions) evaluated, but deleting the row hides the decision. Add a fourth automation value, not-applicable, with a written justification column, so the reviewer sees a reasoned exclusion instead of a gap.

# Pipeline Impact

The export step never sets the job status. It runs with if: always() precisely so that a red gate still produces a snapshot, and it exits non-zero only when its own validation fails — a stale inventory, a parse error, a missing run id — which is a data-integrity failure rather than an accessibility one. Keep it in a separate job from the gate so the two cannot be confused in the run summary, and keep the artifact retention aligned with the retention period of the conformance statement itself, not with the seven-day default that suits raw scan JSON. If the same numbers also feed a standing panel view, wire the JSON output into the datasource described in visualizing WCAG compliance trends with Grafana rather than having the dashboard re-derive criteria from raw tags.

# Common Pitfalls

• Publishing a failure-only list, so the 35 criteria nothing checked are rendered as an empty space that reads as compliance.
• Collapsing no-automated-failure into pass, which converts a tooling limit into a conformance claim in a document someone signs.
• Deriving coverage from violations alone, so a rule that ran and matched nothing is indistinguishable from a rule that was never enabled.
• Exporting the criterion as the packed tag wcag143 instead of 1.4.3, forcing every consumer to re-derive the mapping and eventually derive it differently.
• Omitting evidence_run, leaving a spreadsheet of statuses nobody can trace to a commit, a scanner version or a route list.
• Generating a snapshot on every pull request, producing dozens of conflicting documents and no clear answer to “what did we claim at release”.

# FAQ

Why not just report the percentage of WCAG criteria passing? Because the numerator would be dishonest. A single percentage has to place not-tested somewhere, and putting it in the numerator overstates conformance while putting it in the denominator implies a failure that was never observed. Report three counts — failing, satisfied by automation, not automatically tested — and let the reader see the shape of the evidence rather than a number that hides it.

Who owns the automation column, and how often does it change? An accessibility specialist owns it, and it changes only when the enabled rule set changes, which for most teams is a few times a year. It is a judgement about what a rule can prove, not a fact about the tool, so it is reviewed like any other code change. The drift check exists because the rule set changes more often than anyone remembers to revisit the inventory.

Can this export replace an accessibility audit? No, and it is most useful when it says so explicitly. It is the machine-verifiable slice of the evidence a conformance statement needs, which is roughly a third of the criteria and none of the judgement calls about meaning, order or clarity. Its job is to shrink the manual audit to the criteria a machine genuinely cannot decide, and to prove which ones those were on a specific commit — the same division of labour that governs every gate in the CI/CD integration and automated quality gating section.

# Related

• Reporting, Dashboards & Violation Tracking — the record shape and run metadata this export reads.
• Visualizing WCAG Compliance Trends with Grafana — putting the criterion statuses on a standing panel.
• CI/CD Integration & Automated Quality Gating — where the scan that produces this evidence is gated and scheduled.