# Merging Sharded Accessibility Reports into One Artifact

Four parallel jobs produce four JSON files, and a branch-protection rule can read exactly one answer. This guide is part of Monorepo & Parallel Test Sharding, and it covers the aggregation step that turns N shard reports into a single document: how to count a problem that appears on many routes once, how to reconcile results that were inconclusive in one shard and conclusive in another, and how to notice that a shard is missing before the missing data becomes a green check.

# Root Cause

Concatenating the shard reports is the obvious implementation and it produces an unusable report. A design-system header with a 3.9:1 contrast ratio on its utility links appears in every route that renders the header, so a 240-route run reports it 240 times. The pull-request comment leads with “312 violations”, the author opens it, sees the same color-contrast entry forty screens deep, and closes the tab. There are in fact eleven distinct problems in that run, one of which needs a single CSS change to fix 240 occurrences, and nothing in the concatenated output says so.

Raw counts also make any threshold meaningless. If the gate blocks on “no more than five violations”, the number of violations is partly a function of how many routes happened to be in scope — a pull request that selects two packages passes and the identical change on a branch that selects six fails, because the same shared component was rendered by more routes. Budgets have to be expressed in distinct problems, and distinct problems only exist after deduplication.

The third failure is quieter and worse. A merge implementation that iterates over the shard reports it can find will happily merge three files, report zero blocking findings, and exit 0 — while shard 2, whose 60 routes contained the only serious violation in the run, was cancelled by a runner eviction and uploaded nothing. Absence of a report is not absence of violations, but the naive merge cannot tell the difference. The same applies to partial reports: uploading with if: always() means a shard that crashed at route 40 of 62 still contributes a file, and that file’s contents look exactly like a healthy shard’s contents unless it records how far it got.

# Configuration

The merge script does four things in order: collect the reports, fingerprint every failing node, reconcile statuses, and check what it received against what the plan promised. It writes the merged document and the step summary, and it deliberately does not decide the build result — that belongs to the verdict job, which owns the exit code.

// a11y/scripts/merge-reports.mjs
// node a11y/scripts/merge-reports.mjs --plan shard-plan.json --input a11y/in \
//   --out a11y/merged.json --summary "$GITHUB_STEP_SUMMARY"
import { createHash } from 'node:crypto';
import { appendFileSync, readdirSync, readFileSync, writeFileSync } from 'node:fs';
import { join } from 'node:path';

const args = new Map();
for (let i = 2; i < process.argv.length; i += 2) {
  args.set(process.argv[i].replace(/^--/, ''), process.argv[i + 1]);
}
const plan = JSON.parse(readFileSync(args.get('plan'), 'utf8'));

// download-artifact@v4 with a pattern creates one directory per artifact, so
// recurse rather than assuming every report sits at the top level.
function findReports(dir) {
  return readdirSync(dir, { withFileTypes: true }).flatMap((entry) => {
    const path = join(dir, entry.name);
    if (entry.isDirectory()) return findReports(path);
    return entry.name.endsWith('.json') ? [path] : [];
  });
}
const reports = findReports(args.get('input'))
  .map((path) => JSON.parse(readFileSync(path, 'utf8')));

// Targets carry render-order indexes and framework-generated ids that differ
// between routes for what is one component, so normalise before hashing.
function normalizeTarget(target) {
  return (Array.isArray(target) ? target : [target])
    .map((selector) => String(selector)
      .replace(/:nth-child\(\d+\)/g, ':nth-child(n)')
      .replace(/:nth-of-type\(\d+\)/g, ':nth-of-type(n)')
      .replace(/#[^\s>,]*\d[^\s>,]*/g, '#generated')
      .trim())
    .join(' > ');
}

function fingerprint(finding, node) {
  // The first line of failureSummary names the check that failed; the rest
  // contains measured values that differ per route and must stay out.
  const check = (node.failureSummary ?? '').split('\n')[0];
  return createHash('sha1')
    .update([finding.id, check, normalizeTarget(node.target)].join('|'))
    .digest('hex')
    .slice(0, 12);
}

const RANK = { pass: 0, incomplete: 1, violation: 2 };
const merged = new Map();

for (const report of reports) {
  for (const finding of report.findings ?? []) {
    const status = finding.incomplete ? 'incomplete' : 'violation';
    for (const node of finding.nodes ?? []) {
      const key = fingerprint(finding, node);
      const entry = merged.get(key) ?? {
        fingerprint: key,
        ruleId: finding.id,
        impact: finding.impact ?? 'moderate',
        help: finding.help ?? '',
        wcagTags: (finding.tags ?? []).filter((tag) => tag.startsWith('wcag')),
        status,
        occurrences: [],
      };
      // A violation on one route outranks an incomplete on another: one shard
      // proved the failure, so the merged status is a violation.
      if (RANK[status] > RANK[entry.status]) entry.status = status;
      const where = { route: finding.route, shard: report.shardIndex };
      if (!entry.occurrences.some((o) => o.route === where.route)) {
        entry.occurrences.push({ ...where, html: (node.html ?? '').slice(0, 240) });
      }
      merged.set(key, entry);
    }
  }
}

// Completeness is judged against the plan, never against the files on disk.
const seen = new Set(reports.map((report) => report.shardIndex));
const scanned = new Set(reports.flatMap((report) => report.routesScanned ?? []));
const planned = new Set(plan.shards.flatMap((shard) => shard.routes));
const problems = [];

for (let index = 0; index < plan.shardTotal; index += 1) {
  if (!seen.has(index)) problems.push(`shard ${index} uploaded no report`);
}
for (const report of reports) {
  if (report.partitionHash !== plan.partitionHash) {
    problems.push(`shard ${report.shardIndex} ran partition ${report.partitionHash}`);
  }
  if (report.complete !== true) {
    problems.push(`shard ${report.shardIndex} stopped after ` +
      `${(report.routesScanned ?? []).length} of ${report.routesPlanned.length} routes`);
  }
}
const unscanned = [...planned].filter((route) => !scanned.has(route));
if (unscanned.length > 0) {
  problems.push(`${unscanned.length} planned route(s) never scanned, ` +
    `first: ${unscanned.slice(0, 3).join(', ')}`);
}

const IMPACT = ['critical', 'serious', 'moderate', 'minor'];
const ordered = [...merged.values()].sort((a, b) =>
  IMPACT.indexOf(a.impact) - IMPACT.indexOf(b.impact) ||
  b.occurrences.length - a.occurrences.length);

const document = {
  schemaVersion: 2,
  run: {
    partitionHash: plan.partitionHash,
    shardsExpected: plan.shardTotal,
    shardsSeen: seen.size,
    routesPlanned: planned.size,
    routesScanned: scanned.size,
    complete: problems.length === 0,
    problems,
  },
  findings: ordered.filter((entry) => entry.status === 'violation'),
  incomplete: ordered.filter((entry) => entry.status === 'incomplete'),
};
writeFileSync(args.get('out'), `${JSON.stringify(document, null, 2)}\n`);

const lines = [
  `### Accessibility: ${document.findings.length} distinct finding(s) ` +
  `across ${document.run.routesScanned} route(s)`,
  '',
  `Shards ${document.run.shardsSeen}/${document.run.shardsExpected} · ` +
  `partition \`${plan.partitionHash}\``,
  '',
  '| Impact | Rule | Routes | Example route |',
  '|---|---|---|---|',
];
for (const entry of document.findings.slice(0, 15)) { // 1 MB summary cap
  lines.push(`| ${entry.impact} | \`${entry.ruleId}\` | ` +
    `${entry.occurrences.length} | ${entry.occurrences[0].route} |`);
}
if (document.incomplete.length > 0) {
  lines.push('', `${document.incomplete.length} inconclusive result(s) need review.`);
}
for (const problem of problems) lines.push('', `**Incomplete run:** ${problem}`);
appendFileSync(args.get('summary'), `${lines.join('\n')}\n`);

console.log(`merged ${reports.length} report(s) into ${args.get('out')}`);
// Always exit 0: the verdict job owns the build result, so a merge failure and
// a violation stay distinguishable in the checks list.

The RANK table is the reconciliation rule in one line. axe returns incomplete when a node cannot be judged from the DOM it was given — a collapsed disclosure, an element inside a closed shadow root, a colour whose background could not be resolved. The same component can be inconclusive on a route where it renders collapsed and a clear violation on a route where it renders open, and the honest merged answer is the violation, because one shard obtained the evidence. The reverse never applies: a pass on thirty routes does not soften a violation on the thirty-first, so passes are not merged at all.

# Validation

Test the merge with fixtures rather than with a real pipeline run, because the interesting cases — a missing shard, a stale partition, a truncated report — are all hard to produce on demand and trivial to write as JSON.

// a11y/scripts/merge-reports.test.mjs — run with: node --test a11y/scripts
import { test } from 'node:test';
import assert from 'node:assert/strict';
import { execFileSync } from 'node:child_process';
import { mkdirSync, mkdtempSync, readFileSync, writeFileSync } from 'node:fs';
import { tmpdir } from 'node:os';
import { join } from 'node:path';

function shardReport(index, routes, extra = {}) {
  return {
    shardIndex: index,
    shardTotal: 2,
    partitionHash: 'aaaa1111bbbb2222',
    routesPlanned: routes,
    routesScanned: routes,
    complete: true,
    findings: routes.map((route) => ({
      route,
      id: 'color-contrast',
      impact: 'serious',
      help: 'Elements must meet contrast ratio thresholds',
      tags: ['wcag2aa', 'wcag143'],
      nodes: [{
        target: ['header > nav > a:nth-child(3)'],
        failureSummary: 'Fix any of the following:\n  contrast is 3.91:1',
        html: '<a class="util-link" href="/help">Help</a>',
      }],
    })),
    ...extra,
  };
}

function runMerge(reports) {
  const dir = mkdtempSync(join(tmpdir(), 'a11y-merge-'));
  mkdirSync(join(dir, 'in'), { recursive: true });
  writeFileSync(join(dir, 'plan.json'), JSON.stringify({
    shardTotal: 2,
    partitionHash: 'aaaa1111bbbb2222',
    shards: [
      { index: 0, routes: ['/pricing', '/about'] },
      { index: 1, routes: ['/careers'] },
    ],
  }));
  for (const report of reports) {
    // One directory per artifact, matching download-artifact@v4 with a pattern.
    mkdirSync(join(dir, 'in', `a11y-shard-${report.shardIndex}`));
    writeFileSync(
      join(dir, 'in', `a11y-shard-${report.shardIndex}`, 'report.json'),
      JSON.stringify(report),
    );
  }
  execFileSync('node', [
    'a11y/scripts/merge-reports.mjs',
    '--plan', join(dir, 'plan.json'),
    '--input', join(dir, 'in'),
    '--out', join(dir, 'merged.json'),
    '--summary', join(dir, 'summary.md'),
  ]);
  return JSON.parse(readFileSync(join(dir, 'merged.json'), 'utf8'));
}

test('the same problem on three routes is one finding', () => {
  const merged = runMerge([
    shardReport(0, ['/pricing', '/about']),
    shardReport(1, ['/careers']),
  ]);
  assert.equal(merged.findings.length, 1);
  assert.equal(merged.findings[0].occurrences.length, 3);
  assert.equal(merged.run.complete, true);
});

test('a missing shard is an incomplete run, not a pass', () => {
  const merged = runMerge([shardReport(0, ['/pricing', '/about'])]);
  assert.equal(merged.findings.length, 1);
  assert.equal(merged.run.complete, false);
  assert.match(merged.run.problems.join(' '), /shard 1 uploaded no report/);
});

test('a truncated shard report is detected', () => {
  const truncated = shardReport(1, ['/careers']);
  truncated.routesScanned = [];
  truncated.complete = false;
  const merged = runMerge([shardReport(0, ['/pricing', '/about']), truncated]);
  assert.equal(merged.run.complete, false);
  assert.match(merged.run.problems.join(' '), /stopped after 0 of 1 routes/);
});

Running those three tests takes under a second and covers the failure modes that matter:

$ node --test a11y/scripts
▶ merge-reports.test.mjs
  ✔ the same problem on three routes is one finding (238ms)
  ✔ a missing shard is an incomplete run, not a pass (191ms)
  ✔ a truncated shard report is detected (188ms)
# pass 3
# fail 0

The step summary produced by a healthy four-shard run reads as one table, and the gate that consumes it reads only merged.json:

### Accessibility: 3 distinct finding(s) across 240 route(s)

Shards 4/4 · partition `9f2c41ab77e0d3b5`

| Impact | Rule | Routes | Example route |
|---|---|---|---|
| serious | `color-contrast` | 40 | /pricing |
| serious | `aria-dialog-name` | 6 | /checkout/payment |
| moderate | `heading-order` | 2 | /admin/reports |

4 inconclusive result(s) need review.

# Edge Cases and Conditional Guards

• Over-normalised targets. Stripping every numeric id and :nth-child index makes two genuinely different elements share a fingerprint when the only thing distinguishing them was the index — two adjacent icon buttons in a toolbar, for example. Keep the first html snippet per occurrence and spot-check that a finding’s occurrences describe the same element; if they do not, add the accessible role or a stable data-component attribute to the fingerprint input rather than loosening the normalisation.
• A route present in two shards. A manifest assembled from per-package route files can list /settings twice, so the same finding arrives from two shards for the same route. The occurrence push above guards on route for exactly this reason, which keeps the occurrence count equal to the number of affected pages rather than the number of times the page was scanned.
• Reports from a previous run. A re-run after a rebase can leave old artifacts in the same workflow run when artifact names are not scoped. Comparing each report’s partitionHash against the plan’s turns that into a named problem instead of a silently merged mixture of two commits’ results, which is why the shard reports echo the hash at all.

# Pipeline Impact

The merged document is the only file anything downstream reads, and its run.complete flag is what separates two very different failures. A complete run with blocking findings is exit code 1: a content or code problem the pull-request author fixes. An incomplete run is exit code 2: a missing shard, a stale partition, a truncated report — an infrastructure problem that the author cannot fix and should not be asked to. Keeping those codes distinct is what stops “re-run the job” from becoming the standard response to every red check, and the general policy for that split is in choosing exit codes for warning and blocking a11y jobs.

Size discipline matters more than it looks. Each shard’s raw report on a 240-route run is around 3 MB because axe includes html for every node; the merged document with occurrences capped to one entry per route and html truncated to 240 characters comes out at roughly 180 KB, which is small enough to attach to a pull-request comment and to post to a dashboard on every run. The step summary is capped at 1 MB per step, so the table is truncated to fifteen rows with the full list left in the artifact — the formatting conventions for the comment and the annotations are covered in structuring JSON violation output for Slack and GitHub annotations.

Because occurrences are preserved, the merged file is also the right input for trend tracking: distinct-finding count is the metric that moves when someone fixes something, while occurrence count moves when routes are added or removed. Plot both, keep the fingerprint as the join key between runs, and a finding that persists across twenty runs becomes visible as one long-lived item rather than as noise — which is the shape the trend views in reporting dashboards and violation tracking expect. Retain merged reports for thirty days and shard reports for seven; the shard files are only ever needed to debug a merge.

# Common Pitfalls

• Iterating over the shard reports that exist instead of the shards the plan promised, which turns a cancelled job into a clean pass.
• Fingerprinting on the whole failureSummary, whose measured values differ per route, so the same problem produces one finding per page anyway.
• Fingerprinting on the raw target selector, which splits one component into as many findings as it has sibling positions.
• Dropping the occurrence list after deduplication, leaving a report that says a problem exists but not where to verify the fix.
• Letting a pass in one shard cancel a violation in another by merging passes alongside violations, which quietly hides route-specific failures.
• Exiting non-zero from the merge script as well as the verdict job, so a genuine violation and a broken pipeline produce the same red check with different messages.

# FAQ

Why fingerprint at all instead of grouping by rule id? Grouping by rule id alone merges unrelated problems: a run with a low-contrast header link and a low-contrast disabled button in a form both report color-contrast, and collapsing them into one item means fixing the header appears to fix nothing. The fingerprint adds the failed check and the normalised element path, which separates those two while still collapsing the header link across every route that renders it. Rule id is the right grouping for a dashboard summary, not for the list a developer works through.

What should happen when a shard reports a finding the plan says it should not have scanned? Treat it as a problem, not as a finding to merge. It means the shard read a different partition than the merge was given — usually a stale plan artifact after a rebase, occasionally a hardcoded route list left behind in a spec. The partitionHash comparison catches it, and the route-level check catches the narrower case where the hash matches but a spec scanned something extra. Merging it anyway produces a verdict about a set of routes nobody chose.

Can the merge run inside the last shard instead of in its own job? No, and the reason is ordering rather than capability. With fail-fast: false there is no defined “last” shard, and any shard that also merges will do so before its siblings finish or will not run at all if it is the one that failed. A separate job with needs on the whole matrix and if: always() is the only structure that runs exactly once, after everything, including after failures — the matrix settings that make this work are in sharding axe-core scans across parallel CI jobs.

# Related

• Monorepo & Parallel Test Sharding — the shard plan and the required status check this merged document feeds.
• Sharding axe-core Scans Across Parallel CI Jobs — how the per-shard reports and the partition hash are produced.
• CI/CD Integration & Automated Quality Gating — the surrounding gate that reads exit codes 0, 1 and 2 differently.