Requiring an Accessibility Status Check in Branch Protection

This guide is part of Pull Request Gating & Branch Policies. It deals with one narrow mechanical task: getting the string that GitHub requires to match the string that the accessibility workflow actually emits, expressing that requirement as a file rather than a settings page, and keeping it satisfiable on pull requests where the scan legitimately has nothing to do.

Root Cause

A required status check is a string comparison and nothing more. GitHub stores a list of contexts against a branch pattern, and when a pull request asks whether it can merge, it looks for a check run or commit status with each of those names on the head SHA. There is no validation at configuration time, no autocomplete against the workflows in the repository, and no warning when a required name has never been reported by anything. Type a11y-gaet into the settings field and the repository will happily accept it and then block every pull request forever.

The name itself is assembled at run time from parts that are easy to misread. For a GitHub Actions job the check run name is the job’s name: value, or the job’s key in the jobs: map if name: is absent. The workflow’s own name: does not appear anywhere in it, which surprises people who read the checks list on a pull request and see the workflow name in the grouping header. And when the job carries a strategy.matrix, Actions appends the matrix values in parentheses, comma-separated, in the order the matrix keys are declared — so a job named a11y with project: [storefront, docs-site] and ruleset: [wcag22aa] reports as a11y (storefront, wcag22aa) and a11y (docs-site, wcag22aa). Neither of those is a11y, and requiring a11y is a guaranteed deadlock.

The second half of the problem is where the requirement lives. Configured through the web interface it is invisible state: it is not in the repository, it is not reviewed, it does not appear in a diff, and it cannot be copied to the next twenty services except by a human repeating the same clicks. Repository rulesets fix that by making the whole policy a JSON document that can be created, read back and compared with a single API call — and by allowing the same document to be applied at organisation level against a repository-name pattern, so a new service inherits the gate the moment it is named. The mechanics below assume rulesets for that reason; classic branch protection accepts the same context strings through a different endpoint if a repository is not yet migrated.

The matrix suffix is generated, not chosen, which is why a matrix dimension added in a routine change silently invalidates a hand-typed requirement.

Configuration

Read the names the repository is actually producing

Never type a context from memory. Push a branch, let the workflow run once, and ask the commit what reported against it. This output is the authoritative list and the only safe source for the ruleset.

# Names reported on the tip of the current branch.
SHA=$(git rev-parse HEAD)
gh api "repos/$(gh repo view --json nameWithOwner -q .nameWithOwner)/commits/${SHA}/check-runs" \
  --paginate \
  --jq '.check_runs[] | "\(.name)\t\(.app.slug)\t\(.conclusion)"' | sort -u
# a11y (docs-site, wcag22aa)    github-actions   success
# a11y (storefront, wcag22aa)   github-actions   success
# build                         github-actions   success

The app.slug column matters as much as the name. A context is only trustworthy if it came from the app you expect, and a ruleset can pin that: integration_id restricts a required context to a single GitHub App, so a workflow or personal token cannot satisfy the accessibility gate by posting a fake success status under the same name. GitHub Actions is app id 15368.

Write the ruleset as a file in the repository

Keep the policy next to the workflow it protects. The file below requires both matrix contexts, pins them to Actions, enables the up-to-date-branch behaviour, and applies to the default branch plus every long-lived release branch.

{
  "name": "a11y-required-checks",
  "target": "branch",
  "enforcement": "active",
  "conditions": {
    "ref_name": {
      "include": ["~DEFAULT_BRANCH", "refs/heads/release/**/*"],
      "exclude": []
    }
  },
  "rules": [
    {
      "type": "required_status_checks",
      "parameters": {
        "strict_required_status_checks_policy": true,
        "do_not_enforce_on_create": true,
        "required_status_checks": [
          { "context": "a11y (storefront, wcag22aa)", "integration_id": 15368 },
          { "context": "a11y (docs-site, wcag22aa)", "integration_id": 15368 }
        ]
      }
    }
  ]
}

Three parameters carry real behaviour. strict_required_status_checks_policy is the “branch must be up to date with the base” setting: with it on, a green accessibility result recorded against an older base is not accepted, and the author must merge or rebase before merging. That is correct in a repository with modest traffic and expensive to live with in a busy one, because every push to the base invalidates every open pull request’s scan. do_not_enforce_on_create exempts the initial creation of a branch through the API from the requirement, which is what stops automation that creates release branches from tripping over a check that has not run yet. And integration_id is the only thing standing between a required context and a token that can write statuses.

Apply it, capturing the returned id so the same file can be pushed again idempotently later:

REPO="acme/storefront"
RULESET_ID=$(gh api --method POST "repos/${REPO}/rulesets" \
  --input a11y/policy/a11y-required-checks.json --jq '.id')
echo "created ruleset ${RULESET_ID} on ${REPO}"

Roll the same file out to every repository

Clicking through settings does not scale past about three repositories, and it produces drift that nobody can see. There are two reproducible options, and most organisations end up using both.

The first is an organisation ruleset, which lives once and matches repositories by name. Rules from an organisation ruleset and a repository ruleset are additive — a pull request must satisfy the union — so a team can add stricter local rules without being able to remove the accessibility requirement.

# One ruleset covering every current and future frontend service.
ORG="acme"
jq '.name = "org-a11y-required-checks"
    | .conditions += { "repository_name": { "include": ["web-*", "storefront"],
                                            "exclude": ["web-sandbox"] } }' \
  a11y/policy/a11y-required-checks.json \
  | gh api --method POST "orgs/${ORG}/rulesets" --input -

The second is a per-repository loop for cases where the contexts genuinely differ — a repository whose matrix has different project names, for instance. Make it idempotent by looking the ruleset up by name and choosing PUT over POST accordingly, so the script can be re-run from a scheduled workflow to correct drift instead of creating duplicates.

#!/usr/bin/env bash
# a11y/policy/sync-rulesets.sh — safe to run repeatedly.
set -euo pipefail
FILE="a11y/policy/a11y-required-checks.json"
NAME=$(jq -r '.name' "$FILE")

while read -r repo; do
  [ -z "$repo" ] && continue
  existing=$(gh api "repos/${repo}/rulesets" \
    --jq "[.[] | select(.name == \"${NAME}\")][0].id // empty")
  if [ -n "$existing" ]; then
    gh api --method PUT "repos/${repo}/rulesets/${existing}" --input "$FILE" >/dev/null
    echo "updated ${repo} (ruleset ${existing})"
  else
    new=$(gh api --method POST "repos/${repo}/rulesets" --input "$FILE" --jq '.id')
    echo "created ${repo} (ruleset ${new})"
  fi
done < a11y/policy/repos.txt

The pattern-matched organisation ruleset is what gates a repository that does not exist yet, which is the case a per-repository loop can never cover.

Keep the context reporting when the paths are untouched

A workflow-level paths: filter is the most common cause of a permanently pending required check. The filter is evaluated before the workflow starts, so a pull request that touches only README.md produces no workflow run, no check run, and no context — and the pull request sits at “Expected — waiting for status to be reported” with no job to inspect, no failure to fix, and no way for the author to make progress. The requirement is unsatisfiable rather than unsatisfied.

The remedy is a companion workflow with the inverse filter and a job whose name is byte-identical to the required context. Exactly one of the two workflows runs for any given push, so exactly one check run appears per context, and the guard version costs a few seconds.

# .github/workflows/a11y-skip.yml — the mirror of a11y.yml's paths filter.
name: accessibility (not required)
on:
  pull_request:
    paths-ignore:
      - 'src/**'
      - 'packages/**'
      - 'public/**'
jobs:
  a11y:
    # Must match the scanning workflow's job name exactly, matrix included.
    name: a11y
    runs-on: ubuntu-24.04
    strategy:
      matrix:
        project: [storefront, docs-site]
        ruleset: [wcag22aa]
    steps:
      - name: Report success without scanning
        run: |
          echo "No app code changed; ${{ matrix.project }} scan not required." \
            >> "$GITHUB_STEP_SUMMARY"

Keeping two mirrored filters in step is real maintenance, and forgetting one side reintroduces the deadlock. If the repository can afford it, drop the workflow-level filter entirely and put the scope decision in a job instead, as the parent guide describes — a single workflow that always starts and decides internally has no filter to keep in sync.

Both panels show the same two pull requests; only the guard workflow's existence decides whether the documentation change can ever merge.

Validation

Verify the policy from outside the repository settings, in three passes: what rules apply, what the contexts are, and whether a real pull request behaves.

REPO="acme/storefront"

# 1. Which rulesets apply to this repository, including inherited org ones.
gh ruleset list --repo "$REPO" --parents
# ID     NAME                      SOURCE               STATUS  RULES
# 41207  org-a11y-required-checks  acme (org)           active  1
# 88914  a11y-required-checks      acme/storefront      active  1

# 2. Which rules a specific branch would enforce right now.
gh ruleset check --repo "$REPO" --branch main

# 3. The exact contexts, so a typo is visible rather than inferred.
gh api "repos/${REPO}/rules/branches/main" \
  --jq '.[] | select(.type=="required_status_checks")
            | .parameters.required_status_checks[].context'
# a11y (docs-site, wcag22aa)
# a11y (storefront, wcag22aa)

Then prove it end to end on a throwaway pull request. mergeable_state is the field that tells the truth: blocked means a requirement is unmet, behind means strict mode wants a rebase, and clean means every gate is satisfied.

PR=1421
gh pr checks "$PR" --repo "$REPO"
# a11y (storefront, wcag22aa)   pass   1m42s
# a11y (docs-site, wcag22aa)    pass   0m38s

gh api "repos/${REPO}/pulls/${PR}" --jq '{state: .mergeable_state, merge: .mergeable}'
# {"state":"clean","merge":true}

The negative test matters more than the positive one. Open a pull request that changes only a Markdown file and confirm mergeable_state is clean rather than blocked — that is the assertion that the guard workflow is wired up. Then push a deliberate violation, such as an <img> with no alt, and confirm the state flips to blocked. A gate that has never been observed blocking anything has not been tested. The severity filter that decides which findings reach that point is covered in blocking pull requests on critical accessibility violations.

Edge Cases and Conditional Guards

Merge queues need the same context. A queue evaluates the branch’s required checks against the merge_group ref, so the scanning workflow must also trigger on merge_group and produce the identical job name. A workflow that only listens for pull_request leaves every queue entry waiting for a context that cannot appear, and the entry is ejected on timeout rather than reported as a configuration error.
Fork pull requests get a read-only token. The check run created by the job itself still appears and still satisfies the requirement, but any step that posts a status through the API fails, and any step needing a secret — a private registry token for npm ci, for instance — fails with it. Keep the gating job free of secrets so a fork contribution is gateable at all, and move secret-dependent reporting into a separate non-required workflow.
Matrix shape can differ between branches. The required contexts come from the base branch’s policy while the check runs come from the pull request’s workflow file. A pull request that adds a matrix dimension therefore reports contexts nobody requires, while the required ones vanish — a self-inflicted deadlock that resolves only by updating the ruleset first, merging the requirement change, then merging the workflow change. Sequence matrix changes in that order deliberately.

Pipeline Impact

Making the context required changes nothing about how long the scan takes and everything about what a failure costs. Before, a red accessibility job was a notification; after, it is the difference between shipping today and shipping tomorrow, so the job’s reliability budget tightens. Give the scan job an explicit timeout-minutes well below the workflow default so a hung browser fails fast instead of holding a pull request for six hours, and keep the retry logic inside the job so exactly one check run exists per context per commit.

Strict mode has the largest throughput effect of anything on this page. With strict_required_status_checks_policy: true, every merge into the base branch marks every open pull request as behind, and each one must be updated and re-scanned. On a repository merging thirty pull requests a day that is a re-run storm, and the accessibility scan — usually the longest check — becomes the bottleneck the whole team notices. Repositories at that volume should turn strict off and use a merge queue instead, which validates the post-merge state once per merge rather than repeatedly per pull request.

Finally, the requirement now has to be treated as part of the deployment surface. Add the ruleset file to the accessibility owners’ CODEOWNERS entries, run the synchronisation script from a scheduled workflow so drift is corrected rather than discovered, and record in the same place that tracks scan results whether each protected branch actually requires the context. Where that record lives, and how it is charted alongside violation counts, is covered in reporting, dashboards and violation tracking.

Common Pitfalls

Requiring the bare job name when the job uses a matrix, so no reported context ever matches and every pull request blocks at “Expected”.
Requiring the workflow’s name: value, which never appears in a context string even though the pull-request checks list displays it as a heading.
Adding a paths: filter to the workflow that owns the required job without adding the mirrored guard workflow, converting every documentation change into an unmergeable pull request.
Leaving integration_id off the required context, so any token with status-write permission can satisfy the accessibility gate by posting a success under the same name.
Configuring the requirement through the settings page on the first repository and then hand-repeating it, guaranteeing that repository number nine has a subtly different context list.
Enabling strict mode on a high-traffic repository and then blaming the accessibility scan for the resulting queue of stale pull requests.
Renaming the job and the ruleset entry in the same pull request, which cannot merge because the old context is still required by the base branch’s policy.

FAQ

Can a required context be a wildcard or a pattern? No. Required status checks are exact string matches, with no globbing and no partial matching, which is why a matrix suffix must be spelled out per combination. The practical workaround is to stop requiring matrix jobs at all: add one aggregate job that depends on the matrix, give it a fixed name, and require only that. The ruleset then has one entry that never changes when the matrix does.

What happens to open pull requests when the requirement is added? They are re-evaluated immediately against the head commit that already exists. If the workflow ran on that commit and the context matches, the pull request stays mergeable; if the context is absent — because the run predates the workflow, or the name differs — the pull request becomes blocked until a new push produces the context. Announce the change and expect a wave of pushes on the day it lands.

Does classic branch protection work the same way? The matching semantics are identical: the same context strings, the same exact-match rule, the same deadlock when nothing reports. The differences are administrative. Classic protection targets one branch or one pattern per entry, uses enforce_admins instead of named bypass actors, and is edited through a different endpoint. Rulesets are worth the migration mainly because they can be applied at organisation level to repositories that do not exist yet.

Pull Request Gating & Branch Policies — the surrounding policy model: aggregate check names, merge queues, waivers and audit.
Gating Only Changed Pages With a Diff-Aware Scan — reduce scan cost without letting the required context change shape.
CI/CD Integration & Automated Quality Gating — the section covering scanners, gates, thresholds and reporting end to end.

# Requiring an Accessibility Status Check in Branch Protection

# Root Cause

# Configuration

# Read the names the repository is actually producing

# Write the ruleset as a file in the repository

# Roll the same file out to every repository

# Keep the context reporting when the paths are untouched

# Validation

# Edge Cases and Conditional Guards

# Pipeline Impact

# Common Pitfalls

# FAQ

# Related