# Annotating Pull Requests With axe-core Violation Comments

Every accessibility finding in a pipeline eventually competes for one scarce resource: thirty seconds of the pull-request author’s attention. This guide is part of GitHub Actions a11y Pipeline Setup, and it covers the three surfaces GitHub offers for spending those thirty seconds well — a single sticky comment that is edited in place rather than reposted, inline annotations attached to a file and a line through the Checks API, and a markdown table written to the step summary that needs no token whatsoever — plus the size limits and the fork restriction that decide which of them you can actually rely on.

# Root Cause

axe-core identifies a failing element by CSS selector. Every node in the violations array carries a target array such as ["#cart", "li:nth-child(3) > button"], and that is the only locator it has, because axe runs in a browser against a rendered DOM and has no idea which file produced the element. GitHub’s review surfaces work in the opposite coordinate system: a review comment needs a file path plus a position in the diff, and a Checks API annotation needs path, start_line and end_line. Nothing in the scanner’s output answers those fields, so any tool promising inline accessibility annotations is doing a mapping step somewhere, and the quality of that mapping is the whole story.

The second failure is accumulation. The naive reporting step posts a comment each time it runs, so an ordinary pull request — twelve pushes across two days of review — ends with twelve near-identical bot comments, eleven of which are wrong. Reviewers learn to collapse them, and after that the comment surface is worthless even when it is correct. The fix is an upsert: mark the comment with a hidden HTML marker, search the existing comments for that marker, then edit the one you find instead of creating a new one. It is four API calls and it turns the comment from noise into a live status panel that always describes the current head commit.

The third is permission. Reporting needs a write scope, and on a pull request opened from a public fork the automatically provisioned token is read-only regardless of what the workflow’s permissions block requests. Both the comment endpoint and the check-run endpoint answer 403 in that situation, and a reporting step that treats its own 403 as fatal takes the run red for a reason that has nothing to do with accessibility. Writing to the file named by $GITHUB_STEP_SUMMARY is the one surface with no token involved at all — it is a shell redirect into a file the runner uploads — which is why it should be the primary output and the other two should be enhancements.

# Configuration

The reporting job needs the merged scan output and two scopes, and it should never be the job that decides the merge — that separation is the reason the gate described in blocking pull requests on critical accessibility violations lives in its own job with a read-only token.

# Fragment of the report job: three surfaces, three steps, one artifact.
    permissions:
      contents: read
      pull-requests: write      # sticky comment
      checks: write             # annotation-bearing check run
    steps:
      - name: Step summary (always works, no token used)
        run: node scripts/a11y/summary.mjs a11y-out >> "$GITHUB_STEP_SUMMARY"
      - name: Sticky comment (skipped for forks, where the token is read-only)
        if: github.event.pull_request.head.repo.full_name == github.repository
        run: node scripts/a11y/sticky.mjs a11y-out
        env:
          GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}
          PR_NUMBER: ${{ github.event.pull_request.number }}
      - name: Inline annotations for mappable findings
        if: github.event.pull_request.head.repo.full_name == github.repository
        run: node scripts/a11y/annotate.mjs a11y-out
        env:
          GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}
          HEAD_SHA: ${{ github.event.pull_request.head.sha }}

The summary script is the one that must never fail. It reads the shard reports, groups findings by rule, and prints a markdown table on standard output; the workflow redirects that into the summary file. Grouping matters more than completeness here — a table with one row per failing node is unreadable at forty rows, while one row per rule with a node count and one example selector fits on a screen and still tells the author where to start.

// scripts/a11y/summary.mjs — prints a markdown table; needs no credentials.
import { readdirSync, readFileSync } from 'node:fs';
import { join } from 'node:path';

const dir = process.argv[2] ?? 'a11y-out';
const MAX_ROWS = 25;              // keeps the table well inside the 1 MiB cap
const byRule = new Map();

for (const file of readdirSync(dir).filter((f) => f.endsWith('.json'))) {
  const shard = JSON.parse(readFileSync(join(dir, file), 'utf8'));
  for (const page of shard.results) {
    for (const violation of page.violations) {
      const key = `${violation.id}::${page.route}`;
      const row = byRule.get(key) ?? {
        id: violation.id,
        impact: violation.impact,
        route: page.route,
        nodes: 0,
        example: violation.nodes[0]?.target.join(' '),
      };
      row.nodes += violation.nodes.length;
      byRule.set(key, row);
    }
  }
}

const rank = { critical: 0, serious: 1, moderate: 2, minor: 3 };
const rows = [...byRule.values()].sort(
  (a, b) => rank[a.impact] - rank[b.impact] || b.nodes - a.nodes,
);

if (rows.length === 0) {
  console.log('## Accessibility: no violations across all scanned routes');
  process.exit(0);
}
console.log(`## Accessibility: ${rows.length} rule/route pair(s) failing\n`);
console.log('| Rule | Impact | Route | Nodes | First selector |');
console.log('|---|---|---|---|---|');
for (const r of rows.slice(0, MAX_ROWS)) {
  // Backticks around the selector stop markdown eating > and * characters.
  console.log(
    `| \`${r.id}\` | ${r.impact} | ${r.route} | ${r.nodes} | \`${r.example}\` |`,
  );
}
if (rows.length > MAX_ROWS) {
  console.log(`\n${rows.length - MAX_ROWS} further pair(s) omitted; see the artifact.`);
}

The sticky comment is the same data plus an identity. The marker is an HTML comment, so it is invisible in the rendered thread but present in the body field the API returns, which makes “find my previous comment” a substring search rather than a stored comment id. Pagination is not optional: a long review thread easily exceeds one page, and a script that only checks the first hundred comments starts posting duplicates on exactly the pull requests where duplicates hurt most.

// scripts/a11y/sticky.mjs — one comment per pull request, edited in place.
import { execFileSync } from 'node:child_process';
import { readFileSync } from 'node:fs';

const MARKER = '<!-- a11y-report:sticky -->';
const LIMIT = 65_536;             // GitHub rejects issue comment bodies above this
const dir = process.argv[2] ?? 'a11y-out';
const pr = process.env.PR_NUMBER;
const token = process.env.GITHUB_TOKEN;
const repo = process.env.GITHUB_REPOSITORY;
const headers = {
  Authorization: `Bearer ${token}`,
  Accept: 'application/vnd.github+json',
  'X-GitHub-Api-Version': '2022-11-28',
};

// The summary script is the single source of truth for the table body.
const table = execFileSync('node', ['scripts/a11y/summary.mjs', dir], {
  encoding: 'utf8',
});
let body = `${MARKER}\n${table}\n<sub>Updated by the a11y workflow.</sub>`;
if (body.length > LIMIT) {
  body = `${body.slice(0, LIMIT - 120)}\n\n<sub>Truncated; full report in the artifact.</sub>`;
}

// 1. Walk every page of comments looking for the marker.
let existing = null;
for (let page = 1; page <= 10 && !existing; page += 1) {
  const url =
    `https://api.github.com/repos/${repo}/issues/${pr}/comments` +
    `?per_page=100&page=${page}`;
  const batch = await (await fetch(url, { headers })).json();
  if (!Array.isArray(batch) || batch.length === 0) break;
  existing = batch.find((c) => c.body?.includes(MARKER)) ?? null;
}

// 2. PATCH the comment we own, or POST once if this is the first run.
const target = existing
  ? `https://api.github.com/repos/${repo}/issues/comments/${existing.id}`
  : `https://api.github.com/repos/${repo}/issues/${pr}/comments`;
const res = await fetch(target, {
  method: existing ? 'PATCH' : 'POST',
  headers,
  body: JSON.stringify({ body }),
});
if (!res.ok) {
  // 403 on a fork pull request is expected; never fail the job for it.
  console.error(`comment ${existing ? 'update' : 'create'} failed: ${res.status}`);
  process.exit(res.status === 403 ? 0 : 1);
}
console.log(existing ? `updated comment ${existing.id}` : 'created the comment');

Inline annotations are the strongest surface and the only one that needs help from the application. Because the Checks API demands a path and a line, something in the build has to record where a component came from. The cheapest reliable mechanism is a build-time attribute: emit data-a11y-src="CartRow" on component roots in non-production builds, keep a manifest mapping each component name to its file and declaration line, and walk up from the failing node’s selector to the nearest ancestor carrying the attribute. That resolves most findings to a file and a plausible line; the rest fall back to the summary, which is why the summary is never optional. Structuring the JSON so this mapping is possible at all is covered in structuring JSON violation output for Slack and GitHub annotations.

// scripts/a11y/annotate.mjs — one check run carrying up to 50 annotations
// per API call, built from the component manifest written at build time.
import { readdirSync, readFileSync, existsSync } from 'node:fs';
import { join } from 'node:path';

const dir = process.argv[2] ?? 'a11y-out';
const repo = process.env.GITHUB_REPOSITORY;
const sha = process.env.HEAD_SHA;
const headers = {
  Authorization: `Bearer ${process.env.GITHUB_TOKEN}`,
  Accept: 'application/vnd.github+json',
  'X-GitHub-Api-Version': '2022-11-28',
};
// Shape: { "CartRow": { "path": "src/components/CartRow.tsx", "line": 42 } }
const manifest = existsSync('a11y-src-manifest.json')
  ? JSON.parse(readFileSync('a11y-src-manifest.json', 'utf8'))
  : {};

const annotations = [];
for (const file of readdirSync(dir).filter((f) => f.endsWith('.json'))) {
  const shard = JSON.parse(readFileSync(join(dir, file), 'utf8'));
  for (const page of shard.results) {
    for (const violation of page.violations) {
      for (const node of violation.nodes) {
        // The scan writes node.component from the nearest data-a11y-src ancestor.
        const source = manifest[node.component];
        if (!source) continue;    // unmappable: the summary table still has it
        annotations.push({
          path: source.path,
          start_line: source.line,
          end_line: source.line,
          annotation_level: 'failure',
          title: `${violation.id} (${violation.impact})`,
          message: `${violation.help} on ${page.route}`,
          raw_details: node.html.slice(0, 4000), // long form, shown on expand
        });
      }
    }
  }
}

const base = `https://api.github.com/repos/${repo}/check-runs`;
const created = await fetch(base, {
  method: 'POST',
  headers,
  body: JSON.stringify({
    name: 'a11y annotations',
    head_sha: sha,
    status: 'completed',
    // Neutral, not failure: the gate job owns the merge decision.
    conclusion: annotations.length ? 'neutral' : 'success',
    output: {
      title: `${annotations.length} mapped finding(s)`,
      summary: 'Unmapped findings are listed in the run summary table.',
      annotations: annotations.slice(0, 50), // hard API ceiling per request
    },
  }),
});
const run = await created.json();
for (let i = 50; i < annotations.length; i += 50) {
  await fetch(`${base}/${run.id}`, {
    method: 'PATCH',
    headers,
    body: JSON.stringify({
      output: {
        title: `${annotations.length} mapped finding(s)`,
        summary: 'Unmapped findings are listed in the run summary table.',
        annotations: annotations.slice(i, i + 50),
      },
    }),
  });
}
console.log(`published ${annotations.length} annotation(s)`);

# Validation

Open a pull request that introduces one known failure — an icon-only button with no accessible name is ideal, because it produces a button-name violation on a single node with a stable selector. Then push twice more with unrelated commits and check that the comment count has not moved.

# One comment, regardless of how many pushes have happened:
gh pr view 412 --json comments \
  --jq '[.comments[] | select(.body | contains("a11y-report:sticky"))] | length'
# 1

# The comment body reflects the newest head commit, not the first one:
gh pr view 412 --json comments \
  --jq '.comments[] | select(.body | contains("a11y-report:sticky")) | .updatedAt'
# 2026-07-25T11:42:08Z

# The summary table rendered even though the token was never used:
gh run view 1849321 --json jobs --jq '.jobs[] | select(.name=="report") | .conclusion'
# success

# Annotations landed on the check run rather than the workflow job:
gh api repos/acme/storefront/commits/$(gh pr view 412 --json headRefOid \
  --jq .headRefOid)/check-runs --jq '.check_runs[] | "\(.name) \(.output.annotations_count)"'
# a11y annotations 3
# a11y / gate 0

The negative case matters as much: fix the violation, push, and confirm the same comment now reads “no violations” rather than disappearing. A comment that deletes itself when clean leaves reviewers unable to tell “scanned and passed” from “never ran”, which is the exact ambiguity the sticky comment exists to remove.

# Edge Cases and Conditional Guards

• Fork pull requests: the token is read-only, so guard both write steps on github.event.pull_request.head.repo.full_name == github.repository and let the summary carry the whole report. Treating a 403 as a soft failure inside the script, as sticky.mjs does, covers the case where a repository policy narrows the token further than the trigger implies.
• Truncation on all three surfaces: a step summary over 1 MiB is dropped with a runner warning rather than clipped, an issue comment body over 65,536 characters is rejected outright, and a single Checks API request accepts at most 50 annotations. Cap rows before you hit any of them and point at the artifact for the full detail.
• Unmappable selectors: markup written by hand, injected by a third-party widget, or generated inside a shadow root will not carry the source attribute. Those findings must still reach the table with their rule ID and selector; silently skipping them produces a report that looks clean while the gate fails, which destroys trust in the reporting layer faster than a duplicate comment ever does.

# Pipeline Impact

Reporting must be independent of the merge decision in both directions. It runs if: always() so a failing scan still produces output, and it runs with continue-on-error: true so a 403 or a rate-limit response cannot turn a green gate red. The check run it creates is deliberately neutral rather than failure, because two jobs reporting a failure for the same violation gives reviewers two things to chase and gives branch protection an ambiguous signal about which check to require.

Cost is small but non-zero: the three steps together make between four and a dozen API calls, all against the 1,000-request-per-hour-per-repository limit for the automatically provisioned token, which matters only if a monorepo runs this job many times per pull request. The pagination loop in the sticky script is the main consumer, and capping it at ten pages keeps the worst case bounded. Registering the gate rather than either reporting check as the required status is covered in pull request gating and branch policies.

# Common Pitfalls

• Posting a fresh comment per run, which trains reviewers to collapse the bot and ignore the surface entirely.
• Searching only the first page of comments for the marker, so long-running pull requests quietly start accumulating duplicates.
• Storing the comment id in a cache or an output instead of in the comment body, which breaks on cache eviction and on re-runs from the UI.
• Letting the annotation step fail the job on a fork pull request, so external contributors see a red workflow they have no way to fix.
• Emitting one table row per failing node, which produces a forty-row wall for a single broken component rendered in a loop.
• Deleting the comment when the run is clean, leaving no way to distinguish a passing scan from a scan that never happened.
• Marking the annotation check run as failure, so branch protection now has two competing accessibility checks with different logic.

# FAQ

Why not use review comments anchored to diff lines instead of a check run? Review comments can only be placed on lines that appear in the diff, and accessibility violations frequently live in files the pull request never touched — a new page rendering an old, broken component is the common case. A check-run annotation can point at any line in any file at the head commit, which matches the shape of the problem. Review comments also thread and resolve, so they accumulate state that has to be cleaned up on every push.

Does the step summary work when the workflow is triggered by a fork? Yes, and that is the whole reason to make it the primary surface. $GITHUB_STEP_SUMMARY is a file path in the runner’s environment; writing to it is an ordinary shell redirect that involves no credentials and no API call, so the read-only token on fork pull requests is irrelevant. The summary renders on the run page, and the failing gate check links to that page from the pull request.

How should the comment behave when several jobs report findings? Give each reporting concern its own marker and its own comment, or merge everything into one comment written by a single job that has all the data. What fails is two jobs sharing a marker: they will each find the other’s comment, patch it with their own partial content, and produce a body that alternates between two halves of the truth on every push. One marker, one writer.

# Related

• GitHub Actions a11y Pipeline Setup — the four-job workflow this reporting job belongs to.
• Blocking Pull Requests on Critical Accessibility Violations — the separate job that owns the exit code these comments describe.
• Reporting, Dashboards & Violation Tracking — where the same JSON goes once it needs to outlive the pull request.