# Building a Grafana Dashboard for WCAG Compliance Trends

Grafana is the layer where an accessibility store stops being a database and starts being something a team looks at on a Monday. Getting it right is mostly two decisions: the shape of the rows the dashboard queries, and the ruthless restriction of the dashboard to panels that change somebody’s behaviour. This guide is part of Reporting, Dashboards & Violation Tracking, and it covers the time-series table to write into a SQL data source, the four panels that earn their space, the dashboard JSON committed to the repository so panel changes are reviewable, and the alert rule that fires on new violations rather than on the total.

# Root Cause

The instinct is to push a gauge to a metrics backend and chart it. That fails for three reasons specific to accessibility scans. A CI scan is a batch event, not a sample: it happens four times a day at irregular intervals, so a scrape-based backend either misses it entirely or, via a push gateway, retains the last value forever — which means a pipeline that has been broken for a week renders as a confident flat line rather than as missing data. Second, the interesting dimensions are high-cardinality by nature: rule id crossed with route crossed with impact crossed with success criterion is thousands of label combinations per run, and a commit label makes it unbounded. Third, and worst, a gauge is a single number, so the panel that gets built is a total — and a total is the one accessibility figure that means nothing on its own.

A SQL-backed data source inverts all three problems. Rows are events with an explicit timestamp, so a gap in the data is visibly a gap. Dimensions are columns, so grouping by rule or by criterion is a GROUP BY rather than a cardinality budget. And because the store is queried with SQL, a panel can ask a question that a metrics query language cannot express at all — “how many of today’s failing elements were not failing when we shipped v4.12.0” is a set difference, not an aggregation.

The last thing the schema has to make explicit is that a rule maps to more than one success criterion. link-name carries both WCAG 2.2 SC 4.1.2 (Name, Role, Value) and SC 2.4.4 (Link Purpose in Context), so a row grain that includes the criterion column will repeat the same three failing elements twice. That is the correct grain for a criterion panel and a trap for every other panel, which is why the write step also creates a rule-grain view and every impact or route panel reads the view instead of the table.

# Configuration

The table Grafana reads is derived, not primary. It is written once per run from the normalised findings, which keeps the dashboard queries free of joins and lets the detailed findings expire on their own retention schedule without taking the charts with them. Two small dimension tables sit beside it: one row per scanner or rule-set change for the annotation query, and one row per release for the new-since-release panel.

-- a11y/grafana/schema.sql — the read model Grafana is pointed at.
CREATE TABLE IF NOT EXISTS wcag_timeseries (
  observed_at TEXT    NOT NULL,   -- run start, ISO-8601 with timezone
  commit_sha  TEXT    NOT NULL,
  branch      TEXT    NOT NULL,
  route       TEXT    NOT NULL,
  rule_id     TEXT    NOT NULL,
  impact      TEXT    NOT NULL,   -- critical | serious | moderate | minor
  criterion   TEXT    NOT NULL,   -- '1.4.3'; a rule may produce several rows
  nodes       INTEGER NOT NULL,   -- failing elements for this rule on this route
  new_nodes   INTEGER NOT NULL,   -- of those, fingerprints unseen on this branch
  series_id   TEXT    NOT NULL,   -- scanner version + effective rule set
  PRIMARY KEY (observed_at, route, rule_id, criterion)
);

-- Rule grain: exactly one row per rule per route per run, criteria collapsed.
CREATE VIEW IF NOT EXISTS wcag_ts_rule AS
SELECT observed_at, commit_sha, branch, route, rule_id, impact, series_id,
       MIN(criterion) AS lead_criterion,
       MAX(nodes)     AS nodes,       -- identical across the criterion rows
       MAX(new_nodes) AS new_nodes
FROM wcag_timeseries
GROUP BY observed_at, commit_sha, branch, route, rule_id, impact, series_id;

CREATE TABLE IF NOT EXISTS scanner_change (
  changed_at   TEXT PRIMARY KEY,
  from_version TEXT NOT NULL,
  to_version   TEXT NOT NULL,
  series_id    TEXT NOT NULL,
  note         TEXT
);

CREATE TABLE IF NOT EXISTS release (
  tag         TEXT PRIMARY KEY,
  released_at TEXT NOT NULL,
  commit_sha  TEXT NOT NULL
);

CREATE INDEX IF NOT EXISTS wcag_ts_time_idx ON wcag_timeseries (observed_at, branch);
CREATE INDEX IF NOT EXISTS wcag_ts_rule_idx ON wcag_timeseries (rule_id);

-- One view per panel. The dashboard file then holds a view name rather than a
-- 300-character query, so a panel diff is readable in a pull request.
CREATE VIEW IF NOT EXISTS panel_impact AS
SELECT strftime('%Y-%m-%dT%H:%M:%SZ', observed_at) AS time,
       impact                                      AS metric,
       SUM(nodes)                                  AS value
FROM wcag_ts_rule
WHERE branch = 'main'
GROUP BY observed_at, impact;

CREATE VIEW IF NOT EXISTS panel_new_since_release AS
SELECT COALESCE(SUM(t.new_nodes), 0) AS value
FROM wcag_ts_rule t
WHERE t.branch = 'main'
  AND t.observed_at > (SELECT MAX(released_at) FROM release);

CREATE VIEW IF NOT EXISTS annot_scanner_change AS
SELECT strftime('%Y-%m-%dT%H:%M:%SZ', changed_at)         AS time,
       'axe-core ' || from_version || ' to ' || to_version AS text,
       'scanner,coverage'                                 AS tags
FROM scanner_change;

The projection runs immediately after the findings for a run are loaded. It counts distinct fingerprints per route and rule, marks the ones that have never been seen on this branch, and fans the result out across the criteria attached to each rule.

-- a11y/grafana/project.sql — parameter :run_id, executed once per run.
INSERT OR REPLACE INTO wcag_timeseries
  (observed_at, commit_sha, branch, route, rule_id, impact, criterion,
   nodes, new_nodes, series_id)
SELECT r.started_at, r.commit_sha, r.branch, f.route, f.rule_id, f.impact,
       fc.criterion,
       COUNT(DISTINCT f.fingerprint),
       COUNT(DISTINCT CASE WHEN prior.fingerprint IS NULL
                           THEN f.fingerprint END),   -- unseen before this run
       p.series_id
FROM findings f
JOIN runs r          ON r.run_id = f.run_id
JOIN series_point p  ON p.run_id = f.run_id
JOIN finding_criteria fc
  ON fc.run_id = f.run_id AND fc.fingerprint = f.fingerprint
LEFT JOIN (
  SELECT DISTINCT f2.fingerprint
  FROM findings f2
  JOIN runs r2 ON r2.run_id = f2.run_id
  WHERE r2.branch = 'main' AND f2.status = 'violation'
    AND r2.started_at < (SELECT started_at FROM runs WHERE run_id = :run_id)
) prior ON prior.fingerprint = f.fingerprint
WHERE f.run_id = :run_id
  AND f.status = 'violation'      -- incomplete results are a review queue, not a trend
GROUP BY r.started_at, f.route, f.rule_id, f.impact, fc.criterion;

Precomputing new_nodes at write time is deliberate. The alert rule further down evaluates every ten minutes, and an alert query that recomputes a set difference over months of findings on each evaluation will eventually time out and put the rule into a no-data state — which, if noDataState is misconfigured, reads as a firing alert at three in the morning. The definition of “new” it depends on is the fingerprint identity described in tracking accessibility violation trends across sprints; the dashboard consumes that decision rather than re-litigating it.

- name: Project the Grafana read model
  if: always() && github.ref == 'refs/heads/main'
  run: |
    RUN="${GITHUB_RUN_ID}.${GITHUB_RUN_ATTEMPT}"
    sqlite3 a11y.db < a11y/grafana/schema.sql
    sqlite3 a11y.db "$(sed "s/:run_id/'${RUN}'/g" a11y/grafana/project.sql)"

# The Four Panels That Earn Their Space

A dashboard is judged by how many panels somebody would notice were missing. Four is enough for accessibility, and each maps to a decision.

The stacked series is the headline. Stacking by impact rather than plotting one total makes a shift in composition visible: a flat total whose critical band is growing while its minor band shrinks is a worsening application, and a single line hides that completely.

-- Panel 1: violations by impact, stacked. Grafana time_series format.
SELECT strftime('%Y-%m-%dT%H:%M:%SZ', observed_at) AS time,
       impact                                      AS metric,
       SUM(nodes)                                  AS value
FROM wcag_ts_rule
WHERE branch = 'main'
  AND $__timeFilter(observed_at)   -- Grafana substitutes the dashboard range
GROUP BY observed_at, impact
ORDER BY observed_at;

The rules table answers a different question: where the leverage is. It reads the latest run only, because a thirty-day window mixes a rule that was fixed last Tuesday into today’s priorities.

-- Panel 2: top rules on the most recent run, by failing elements.
SELECT rule_id            AS "Rule",
       impact             AS "Impact",
       lead_criterion     AS "WCAG SC",
       SUM(nodes)         AS "Elements",
       COUNT(DISTINCT route) AS "Routes"
FROM wcag_ts_rule
WHERE branch = 'main'
  AND observed_at = (SELECT MAX(observed_at) FROM wcag_ts_rule
                     WHERE branch = 'main')
GROUP BY rule_id, impact, lead_criterion
ORDER BY 4 DESC
LIMIT 10;

The single stat is the only number on the dashboard anyone should feel responsible for today. It is scoped to the window since the last production release, so it is a question about this release rather than about the history of the codebase.

-- Panel 3: stat — failing elements that did not exist at the last release.
WITH shipped AS (
  SELECT MAX(released_at) AS at FROM release
)
SELECT COALESCE(SUM(t.new_nodes), 0) AS value
FROM wcag_ts_rule t, shipped
WHERE t.branch = 'main'
  AND t.observed_at > shipped.at;

The annotation query is what stops the dashboard lying. Every panel above is conditional on the rule set, so a version bump draws a step that looks exactly like a regression. Rendering the change as a vertical marker on the time axis answers the question before it is asked.

-- Annotation: one marker per scanner or rule-set change.
SELECT strftime('%Y-%m-%dT%H:%M:%SZ', changed_at) AS time,
       'axe-core ' || from_version || ' to ' || to_version AS text,
       'scanner,coverage' AS tags
FROM scanner_change
WHERE $__timeFilter(changed_at)
ORDER BY changed_at;

Each of those statements is then stored as a view in the same schema file, with the $__timeFilter macro left out of the view and applied by the panel. That split matters for review: the aggregation logic lives in a .sql file where a diff is legible line by line, and the dashboard JSON carries SELECT time, metric, value FROM panel_impact instead of three hundred characters of SQL on one line that no reviewer will read.

# Dashboard as Code

A dashboard edited in the browser is a production change with no diff, no review and no history, and the first person who drags a panel while exploring will save it for everyone. Commit the JSON, provision it read-only, and treat a panel change like any other pull request — which means a reviewer can see that a WHERE clause quietly dropped critical from a chart.

{
  "uid": "a11y-wcag-trends",
  "title": "Accessibility: WCAG trends",
  "editable": false,
  "schemaVersion": 39,
  "refresh": "15m",
  "time": { "from": "now-90d", "to": "now" },
  "tags": ["accessibility", "wcag"],
  "annotations": {
    "list": [
      {
        "name": "Scanner changes",
        "enable": true,
        "iconColor": "red",
        "datasource": { "type": "grafana-sqlite-datasource", "uid": "a11y-sql" },
        "target": {
          "rawSql": "SELECT time, text, tags FROM annot_scanner_change",
          "format": "table"
        }
      }
    ]
  },
  "panels": [
    {
      "id": 1,
      "type": "stat",
      "title": "New failing elements since last release",
      "gridPos": { "h": 4, "w": 8, "x": 0, "y": 0 },
      "datasource": { "type": "grafana-sqlite-datasource", "uid": "a11y-sql" },
      "targets": [
        {
          "refId": "A",
          "rawSql": "SELECT value FROM panel_new_since_release",
          "format": "table"
        }
      ],
      "fieldConfig": {
        "defaults": {
          "thresholds": {
            "mode": "absolute",
            "steps": [
              { "color": "green", "value": null },
              { "color": "red", "value": 1 }
            ]
          }
        },
        "overrides": []
      }
    },
    {
      "id": 2,
      "type": "timeseries",
      "title": "Violations by impact",
      "gridPos": { "h": 10, "w": 16, "x": 0, "y": 4 },
      "datasource": { "type": "grafana-sqlite-datasource", "uid": "a11y-sql" },
      "targets": [
        {
          "refId": "A",
          "rawSql": "SELECT time, metric, value FROM panel_impact WHERE $__timeFilter(time)",
          "format": "time_series"
        }
      ],
      "fieldConfig": {
        "defaults": {
          "custom": {
            "drawStyle": "bars",
            "stacking": { "mode": "normal" },
            "fillOpacity": 70
          }
        },
        "overrides": []
      }
    }
  ]
}

Provisioning points Grafana at the directory and forbids in-place edits, so the committed file is always the running dashboard.

# grafana/provisioning/dashboards/a11y.yaml
apiVersion: 1
providers:
  - name: accessibility
    orgId: 1
    folder: Accessibility
    type: file
    disableDeletion: true
    allowUiUpdates: false      # UI edits cannot outlive a reload
    updateIntervalSeconds: 60
    options:
      path: /etc/grafana/dashboards
      foldersFromFilesStructure: false

# The Alert That Fires on New Violations

An alert on the total is an alert on the wrong number. It fires when six routes are added to the manifest, it fires when a scanner upgrade enables a rule, and it stays silent in the week that five real defects were introduced alongside seven unrelated fixes. Alert on new_nodes instead: a value above zero means at least one element is failing that was not failing before, which is the only condition worth waking someone for. Two guards keep it truthful — the evaluation window must contain a run whose series_id matches the previous run’s, and noDataState must be OK, because a quiet pipeline is an operations problem rather than an accessibility regression.

# grafana/provisioning/alerting/a11y-new-violations.yaml
apiVersion: 1
groups:
  - orgId: 1
    name: accessibility
    folder: Accessibility
    interval: 10m
    rules:
      - uid: a11y-new-violations
        title: New accessibility violations on main
        condition: OVER_ZERO
        for: 0m                # event-driven, not sampled: no need to persist
        noDataState: OK        # a day with no scan is not a regression
        execErrState: OK
        labels:
          severity: ticket
        annotations:
          summary: A failing element appeared that was not failing before.
          runbook_url: /accessibility/new-violation-triage
        data:
          - refId: NEW
            relativeTimeRange: { from: 86400, to: 0 }
            datasourceUid: a11y-sql
            model:
              refId: NEW
              format: table
              rawSql: >
                SELECT COALESCE(SUM(t.new_nodes), 0) AS value
                FROM wcag_ts_rule t
                WHERE t.branch = 'main'
                  AND t.observed_at >= datetime('now', '-1 day')
                  AND t.series_id = (SELECT series_id FROM wcag_ts_rule
                                     ORDER BY observed_at DESC LIMIT 1)
          - refId: OVER_ZERO
            datasourceUid: __expr__
            model:
              refId: OVER_ZERO
              type: threshold
              expression: NEW
              conditions:
                - evaluator: { type: gt, params: [0] }

# Validation

Validate the dashboard the way any other committed artifact is validated: parse it, assert the invariants that break silently in Grafana, and only then import it. A panel whose datasource.uid no longer matches the provisioned data source renders as an empty graph with no error, which is the most expensive failure mode on this page.

#!/usr/bin/env bash
# a11y/grafana/verify.sh — run in CI on every change to the dashboard JSON.
set -euo pipefail
DASH=grafana/dashboards/a11y-wcag-trends.json

jq -e '.uid and .title' "$DASH" > /dev/null          # a missing uid forks the dashboard
jq -e '.editable == false' "$DASH" > /dev/null        # UI edits must not be savable
jq -e '[.panels[] | select((.targets // []) | length == 0)] | length == 0' \
  "$DASH" > /dev/null                                # every panel must query something
jq -e '[.panels[].datasource.uid, .annotations.list[].datasource.uid]
       | unique == ["a11y-sql"]' "$DASH" > /dev/null  # one provisioned datasource

# Every rawSql must survive the engine that will actually run it.
jq -r '[.panels[].targets[].rawSql, .annotations.list[].target.rawSql][]' "$DASH" \
| while read -r sql; do
    printf 'EXPLAIN %s;\n' "${sql//\$__timeFilter(*)/1=1}" \
      | sqlite3 a11y.db > /dev/null
  done
echo "dashboard JSON verified"

Then confirm the read model answers the four panel questions with numbers a human recognises. The stat panel is the one to check by hand, because a zero there is either genuinely good news or a broken release table.

sqlite3 -header -column a11y.db "
  SELECT (SELECT MAX(released_at) FROM release)              AS shipped_at,
         (SELECT COUNT(*) FROM wcag_timeseries)              AS ts_rows,
         (SELECT COUNT(*) FROM wcag_ts_rule)                 AS rule_rows,
         (SELECT SUM(new_nodes) FROM wcag_ts_rule
           WHERE branch = 'main'
             AND observed_at > (SELECT MAX(released_at) FROM release)) AS new_since;"
# Expected shape — rule_rows is always lower than ts_rows because of the
# criterion fan-out, and new_since must be 0 immediately after a release:
# shipped_at            ts_rows  rule_rows  new_since
# 2026-07-21T16:02:11Z  1487     1103       3

# Edge Cases and Conditional Guards

• No scan in the dashboard window. With noDataState: OK the alert stays silent, which is correct, but the panels then show a stale last value as though it were current. Add "nullPointMode": "null" behaviour by leaving gaps unfilled and treat a flat right-hand edge as suspicious; a separate freshness stat on MAX(observed_at) is cheaper than explaining the flat line twice.
• A route renamed rather than fixed. Renaming /pricing to /plans retires every fingerprint on the old route and creates the same number of new ones, so the stat panel spikes and the alert fires on a rename. Annotate route-manifest changes in scanner_change as well, and suppress the alert for that evaluation exactly as a scanner bump is suppressed.
• SQLite locked during the load. Grafana reading the same file the reporting job is writing will occasionally return database is locked. Enable WAL mode on the store, point the data source at a read-only copy published after the load step, or move the read model to Postgres once more than one pipeline writes to it.

# Pipeline Impact

The projection step and the dashboard verification live on opposite sides of the pipeline. Projection runs in the reporting job with if: always(), produces no artifact of its own, and must never influence the exit code — a Grafana outage is not an accessibility failure, and a gate that can be blocked by a dashboard will be deleted by the first person it inconveniences. Verification, by contrast, is a normal required check on pull requests that touch grafana/, and it should fail hard: a malformed dashboard merged to the default branch is invisible until someone opens it a week later. The blocking decision about violations themselves stays where it belongs, in the gate configured through progressive threshold management, and the alert here is a notification path rather than a second gate — give it the same warning-mode soak described in soak-testing a new accessibility gate in warning mode before it starts paging anyone. For an audience that needs a signed statement rather than a live panel, the same read model exports through exporting accessibility results to compliance dashboards.

# Common Pitfalls

• Summing the nodes column across the criterion fan-out, which reports a link-name failure twice and inflates every AA total.
• Alerting on the total instead of on new findings, so the dashboard pages the team for a manifest edit and stays quiet for a genuine regression.
• Leaving allowUiUpdates: true, after which the running dashboard and the committed JSON diverge and nobody can say which panel query produced last month’s screenshot.
• Reusing a dashboard uid across two files, or changing it in a commit, either of which creates a second dashboard and leaves half the team looking at the abandoned one.
• Charting an accessibility score alongside violation counts as if they were the same axis; the two move independently, as Lighthouse accessibility score versus axe violation counts sets out.
• Pointing the data source directly at the file the loader writes, then blaming Grafana for intermittent database is locked errors.

# FAQ

Why a SQL data source rather than pushing metrics to a time-series database? Because every useful accessibility panel groups by a dimension — rule, route, impact, success criterion — and at least one of them needs a set difference against an earlier state. In a metrics backend those dimensions are labels, and rule crossed with route crossed with criterion is thousands of series per run with a commit label making it unbounded. SQL also makes a gap in the data visible as a gap, whereas a push gateway retains the last value indefinitely and renders a dead pipeline as a reassuring flat line.

What should the alert do when there is no recent scan? Nothing. Set noDataState: OK and monitor pipeline health separately, because an alert that fires on missing data trains the team to ignore the alert during every holiday week and every infrastructure migration. Pair that with a freshness panel on MAX(observed_at) so a stalled scan is visible on the dashboard, and route that condition to the people who own the pipeline rather than to the people who own the components.

Can this dashboard be shared across several repositories? Yes, provided the read model carries a repo column and the dashboard exposes it as a template variable, so each panel query filters on $repo. Keep one row grain and one committed JSON file rather than forking the dashboard per team; a fork drifts within a month and the two versions then disagree about the same numbers. Aggregating across repositories is only meaningful for new-versus-fixed movement, never for absolute counts, because the scanned surface differs per application.

# Related

• Reporting, Dashboards & Violation Tracking — the parent guide defining the findings and criterion records this read model is projected from.
• Tracking Accessibility Violation Trends Across Sprints — where the new, fixed and carried-over definitions the panels display come from.
• CI/CD Accessibility Quality Gating — the section covering the gates and thresholds this dashboard observes but never enforces.