Reporting, Dashboards & Violation Tracking for Accessibility Pipelines

A passing or failing exit code tells a developer whether the build broke, but it tells an engineering manager almost nothing about which WCAG criteria are slipping, where the regressions cluster, or whether the team is trending toward compliance. This guide is part of CI/CD Integration & Automated Quality Gating, and it covers the reporting layer that sits between your scanner and your humans: normalizing raw JSON, routing it to the right surface, persisting it, and visualizing the trend over time.

The core problem is that axe-core and Lighthouse emit deeply nested, tool-specific JSON that nobody reads directly. To make that output actionable you need to normalize it into a single shape, then fan it out to four destinations: inline pull-request annotations, a chat channel, a durable store, and a time-series view.

Key implementation targets

  • Normalize axe-core results.violations and Lighthouse audit output into one stable schema
  • Route normalized findings to GitHub PR annotations and a Slack or Teams channel
  • Persist every run with stable rule IDs and WCAG 2.2 SC mapping for an audit trail
  • Append per-commit severity counts to a time series so trends are queryable
Reporting data flow Scanner JSON enters a normalizer which fans out to a PR comment, Slack, a dashboard store, and a time-series database. Scanner JSON axe-core / Lighthouse Normalizer stable schema PR annotation Slack / Teams Compliance store Time-series DB
One normalizer fans scanner output out to four reporting destinations.

Prerequisites

1. Normalize the JSON schema

Both axe-core and Lighthouse describe accessibility violations, but they disagree on field names, severity vocabularies, and nesting. Collapse them into one flat record per finding so every downstream consumer reads the same shape. Each normalized record carries a stable ruleId, an impact bucket, the WCAG criterion, and the affected selectors.

// normalize.js — collapse axe-core or Lighthouse JSON into one schema
const fs = require('fs');

// Map Lighthouse score weight to axe-style impact buckets
function lhImpact(score) {
  if (score === 0) return 'serious';
  if (score < 0.5) return 'moderate';
  return 'minor';
}

function fromAxe(report, meta) {
  return report.violations.flatMap(v =>
    v.nodes.map(n => ({
      ruleId: v.id,                         // stable across runs, e.g. "color-contrast"
      impact: v.impact,                     // critical | serious | moderate | minor
      wcag: (v.tags.find(t => t.startsWith('wcag2')) || 'n/a'),
      help: v.help,
      selector: n.target.join(' '),
      ...meta,                              // commit, branch, url, timestamp
    }))
  );
}

function fromLighthouse(report, meta) {
  const a11y = report.categories.accessibility.auditRefs;
  return a11y
    .map(ref => report.audits[ref.id])
    .filter(a => a.score !== null && a.score < 1)
    .map(a => ({
      ruleId: a.id,
      impact: lhImpact(a.score),
      wcag: 'n/a',                          // Lighthouse does not emit SC numbers directly
      help: a.title,
      selector: '(page-level)',
      ...meta,
    }));
}

const file = process.argv[2];
const raw = JSON.parse(fs.readFileSync(file, 'utf8'));
const meta = {
  commit: process.env.GITHUB_SHA || 'local',
  branch: process.env.GITHUB_REF_NAME || 'local',
  url: process.env.SCAN_URL || 'http://localhost:3000',
  timestamp: new Date().toISOString(),
};

const records = raw.lighthouseVersion ? fromLighthouse(raw, meta) : fromAxe(raw, meta);
fs.writeFileSync('normalized.json', JSON.stringify(records, null, 2));
console.log(`Normalized ${records.length} findings`);

This single normalization step is what lets the same downstream code work whether the team standardizes on axe-core configuration or on Lighthouse CI baselines.

2. Route to GitHub annotations and Slack

With one schema, routing is a pair of small adapters. GitHub Actions reads ::error workflow commands from stdout and renders them inline on the changed files. Slack reads a Block Kit JSON payload posted to an incoming webhook. The detailed transforms — including grouping by impact — live in the dedicated guide on structuring JSON violation output for Slack and GitHub annotations.

// route.js — emit GitHub annotations and post a Slack summary
const https = require('https');
const records = require('./normalized.json');

// (a) GitHub Actions annotations — printed to stdout, picked up by the runner
for (const r of records) {
  // Annotations cannot point at a DOM selector, so we surface it in the message
  console.log(`::error title=${r.ruleId} (${r.impact})::${r.help} — ${r.selector}`);
}

// (b) Slack summary grouped by impact
const byImpact = records.reduce((acc, r) => {
  acc[r.impact] = (acc[r.impact] || 0) + 1;
  return acc;
}, {});
const summary = Object.entries(byImpact)
  .map(([k, v]) => `*${k}*: ${v}`)
  .join('  ');

const payload = JSON.stringify({
  text: `A11y scan for ${process.env.GITHUB_REF_NAME || 'local'} — ${summary || 'no violations'}`,
});

const hook = new URL(process.env.SLACK_WEBHOOK_URL);
const req = https.request(hook, { method: 'POST', headers: { 'Content-Type': 'application/json' } });
req.write(payload);
req.end();

3. Persist normalized findings to a store

Annotations and chat messages are ephemeral. For audit trails and trend math you need a durable record keyed by commit. A single-file SQLite database is enough to start and travels well as a CI artifact; teams with a shared Postgres swap the connection string. Each row carries the stable rule ID and WCAG criterion so an auditor can reconstruct exactly what failed on any commit.

// persist.js — append findings to a SQLite table
const Database = require('better-sqlite3');
const records = require('./normalized.json');

const db = new Database('a11y.db');
db.exec(`CREATE TABLE IF NOT EXISTS findings (
  commit_sha TEXT, branch TEXT, url TEXT, ts TEXT,
  rule_id TEXT, impact TEXT, wcag TEXT, selector TEXT
)`);

const insert = db.prepare(`INSERT INTO findings
  (commit_sha, branch, url, ts, rule_id, impact, wcag, selector)
  VALUES (@commit, @branch, @url, @timestamp, @ruleId, @impact, @wcag, @selector)`);

const tx = db.transaction(rows => rows.forEach(r => insert.run(r)));
tx(records);
console.log(`Persisted ${records.length} rows`);

The export and schema details — including SARIF for code-scanning dashboards — are expanded in exporting accessibility results to compliance dashboards.

4. Visualize the trend over time

Once findings persist per commit, you can collapse them into a daily or per-sprint severity count and chart the burn-down. A lightweight approach appends counts to a time series; the mechanics of computing sprint deltas are covered in tracking accessibility violation trends across sprints, and wiring that series into live panels is covered in visualizing WCAG compliance trends with Grafana.

-- Per-commit severity rollup, newest first
SELECT commit_sha,
       SUM(impact = 'critical') AS critical,
       SUM(impact = 'serious')  AS serious,
       SUM(impact = 'moderate') AS moderate,
       SUM(impact = 'minor')    AS minor
FROM findings
GROUP BY commit_sha
ORDER BY MAX(ts) DESC;

Pipeline Integration

Run the four steps in sequence after the scan, uploading the SQLite file and normalized.json as build artifacts so the data survives the ephemeral runner.

- name: Report a11y results
  if: always()                       # run even when the gate fails the job
  env:
    SLACK_WEBHOOK_URL: ${{ secrets.SLACK_WEBHOOK_URL }}
  run: |
    node normalize.js axe-results.json
    node route.js                    # prints ::error annotations + posts Slack
    node persist.js
- name: Upload a11y artifacts
  if: always()
  uses: actions/upload-artifact@v4
  with:
    name: a11y-report
    path: |
      normalized.json
      a11y.db

The annotation step does not itself set the job status. Keep your blocking decision in the gate step (see auto-fail vs warning workflows) so reporting and gating stay decoupled.

Troubleshooting & Flaky-Test Mitigation

  • Empty normalized.json: confirm the scanner actually wrote JSON; a non-zero exit with || true upstream can leave a truncated file. Validate with node -e "require('./axe-results.json')".
  • Duplicate Slack messages on retries: key the post on GITHUB_RUN_ID and skip if already sent, or only post from the final attempt.
  • Annotation flooding: GitHub renders at most 10 annotations of each level per step. Cap output and link to the full artifact for the remainder.
  • Selector noise across runs: virtualized lists produce shifting selectors; persist the ruleId count rather than the selector when computing trends.

Common Pitfalls

  • Posting raw axe JSON to Slack — it is unreadable and rate-limits the channel; always normalize and summarize first.
  • Storing the human-readable help text as the primary key instead of the stable ruleId, which breaks trend joins when copy changes.
  • Letting the reporting step’s own failure (e.g. a Slack 500) fail the build; wrap network calls so reporting never blocks the gate.
  • Mapping Lighthouse and axe severities differently across pages, which makes cross-tool dashboards meaningless.

FAQ

Should reporting run before or after the quality gate? Run the scan once, then run reporting and gating as separate steps that both read the same JSON. Use if: always() on the reporting step so you still get a dashboard entry and Slack alert when the gate fails the job.

Do I need a real database, or is SQLite enough? SQLite is enough for a single repo with one pipeline; it is a single file you can upload as an artifact. Move to Postgres only when multiple repos write to a shared compliance store or when you need concurrent writers.

How do I avoid leaking the Slack webhook URL? Store it as an encrypted CI secret and reference it only through an environment variable. Never echo the variable in a run step, and never commit it to the normalized output.

In This Section

Back to CI/CD Integration Automated Quality Gating