# Ratcheting Violation Budgets Down Each Sprint

A violation ceiling only ever falls when a human edits a file, and violation counts fall whenever anyone fixes anything, which means the gap between the two grows quietly and permanently. This guide is part of Progressive Threshold Management, and it covers the scheduled job that closes that gap on its own: read the counts from recent green runs on the default branch, take the lowest one that has been seen more than once, write it back as the new ceiling, and open a pull request that a human approves in ten seconds instead of computing by hand in an hour.

# Root Cause

The asymmetry is the whole problem. Fixing a link-name violation lowers the observed count immediately and silently; lowering the link-name ceiling from 412 to 390 requires somebody to notice the fix, remember the ritual, and open a config pull request nobody asked for. In one repository measured over six weeks, 63 rule ceilings had accumulated 214 findings of unused slack — 214 regressions that could have shipped without turning a single check red, all of them paid for by work that had already been done.

Slack also concentrates in exactly the wrong place. The rules that get fixed in bulk are the ones with automated remediations, and those are usually the high-count ones: contrast tokens, missing alt, form labels. A color-contrast ceiling of 2,190 sitting above an actual count of 2,038 for a month is 152 free contrast regressions in the part of the product where contrast regressions are most likely, because that is where the team is actively editing CSS.

The naive fix — set every ceiling to the last observed count on every merge — replaces one failure with a worse one. Counts are not perfectly stable: a lazily loaded gallery, a virtualised table, a runner that painted a skeleton a frame early, and one run reads 388 where the honest number is 391. Commit 388 as the ceiling and the next three pull requests fail for a defect none of them contain. The job therefore needs a window, a stability requirement, a grace factor for rules that genuinely wobble, and an escape hatch for the sprint where a legitimate feature adds findings that cannot be fixed yet.

# Configuration

The job needs a history to read. Every merge to the default branch appends one line to a11y/history.jsonl recording the commit, the run id, the route count and the per-rule counts — the append is a two-line step in the post-merge job, and JSON Lines is used precisely because appending never conflicts the way a rewritten JSON array does.

{"commit":"8f21c0d","runId":9912441,"routes":214,"axe":"4.10.2","countsByRule":{"color-contrast":2038,"link-name":390,"heading-order":318,"image-alt":71,"label":88}}

Ratchet behaviour lives in its own config so tuning it does not touch the budget file the ratchet writes.

{
  "window": 10,
  "minSamples": 6,
  "minOccurrences": 2,
  "requireRoutes": 214,
  "defaultGrace": 1.0,
  "graceByRule": {
    "color-contrast": 1.01,
    "image-alt": 1.05,
    "aria-hidden-focus": 1.1
  }
}

minOccurrences: 2 is the stability requirement from the figure above. requireRoutes is the guard that discards any historical run that scanned a different number of pages, because a run that missed nine routes recorded a floor for an application that was never fully measured. Grace is per rule and deliberately small: 1.05 on image-alt allows three findings of wobble at a count of 71, which is enough for lazy images and not enough to hide a regression.

The escape hatch is a separate file, because it holds decisions with owners and deadlines rather than tuning parameters. An allowance widens one ceiling by a stated amount, for a stated reason, until a stated date.

{
  "allowances": [
    {
      "rule": "aria-required-children",
      "extra": 12,
      "reason": "New reporting grid ships behind a flag; grid roles land in ORD-4471",
      "owner": "@team-orders",
      "expires": "2026-08-22"
    }
  ]
}

Now the proposer. It reads the history, filters it, computes a floor per rule, applies grace, refuses to lower a ceiling that an active allowance is protecting, and writes both the new budget and a markdown table for the pull-request body.

#!/usr/bin/env node
// a11y/ratchet/propose.mjs — usage: node a11y/ratchet/propose.mjs [--dry-run]
// exit 0 = a proposal was written, 3 = nothing to lower, 2 = history unusable
import { readFileSync, writeFileSync } from 'node:fs';

const dryRun = process.argv.includes('--dry-run');
const cfg = JSON.parse(readFileSync('a11y/ratchet.config.json', 'utf8'));
const budget = JSON.parse(readFileSync('a11y/budget.json', 'utf8'));
const { allowances } = JSON.parse(readFileSync('a11y/allowances.json', 'utf8'));
const today = new Date().toISOString().slice(0, 10);

const history = readFileSync('a11y/history.jsonl', 'utf8')
  .trim().split('\n').map((line) => JSON.parse(line))
  .filter((run) => run.routes === cfg.requireRoutes) // partial scans are not evidence
  .slice(-cfg.window);

if (history.length < cfg.minSamples) {
  console.error(`Only ${history.length} comparable run(s); need ${cfg.minSamples}.`);
  process.exit(2);
}

// An active allowance pins a rule: the ceiling must not move while it is open.
const pinned = new Map(
  allowances.filter((a) => a.expires >= today).map((a) => [a.rule, a]),
);

const rows = [];
for (const [rule, entry] of Object.entries(budget.rules)) {
  const observed = history
    .map((run) => run.countsByRule[rule])
    .filter(Number.isInteger);           // absent is not the same as zero

  if (observed.length < cfg.minSamples) {
    rows.push({ rule, from: entry.max, to: entry.max, why: 'too few samples' });
    continue;
  }
  if (pinned.has(rule)) {
    rows.push({ rule, from: entry.max, to: entry.max,
      why: `pinned by allowance until ${pinned.get(rule).expires}` });
    continue;
  }

  // Lowest value seen at least minOccurrences times, so a single fluke is ignored.
  const tally = new Map();
  for (const n of observed) tally.set(n, (tally.get(n) ?? 0) + 1);
  const stable = [...tally.entries()]
    .filter(([, count]) => count >= cfg.minOccurrences)
    .map(([value]) => value);
  if (stable.length === 0) {
    rows.push({ rule, from: entry.max, to: entry.max, why: 'no repeated value' });
    continue;
  }

  const floor = Math.min(...stable);
  const grace = cfg.graceByRule[rule] ?? cfg.defaultGrace;
  const proposed = Math.ceil(floor * grace);
  if (proposed < entry.max) {
    rows.push({ rule, from: entry.max, to: proposed, why: `floor ${floor} × ${grace}` });
    entry.max = proposed;
  } else {
    rows.push({ rule, from: entry.max, to: entry.max, why: 'already at floor' });
  }
}

const lowered = rows.filter((r) => r.to < r.from);
const reclaimed = lowered.reduce((sum, r) => sum + (r.from - r.to), 0);

const table = ['| rule | ceiling | proposed | basis |', '|---|---|---|---|']
  .concat(lowered.map((r) => `| \`${r.rule}\` | ${r.from} | ${r.to} | ${r.why} |`))
  .join('\n');
const body = `Ratchet over the last ${history.length} comparable runs on main.\n\n` +
  `${table}\n\nReclaims **${reclaimed}** findings of slack across ` +
  `${lowered.length} rule(s). Skipped: ` +
  `${rows.length - lowered.length} rule(s).\n`;

if (dryRun) {
  console.log(body);
  process.exit(lowered.length > 0 ? 0 : 3);
}

budget.policy.totalCeiling = Object.values(budget.rules)
  .reduce((sum, r) => sum + r.max, 0);
writeFileSync('a11y/budget.json', JSON.stringify(budget, null, 2) + '\n');
writeFileSync('a11y/ratchet/proposal.md', body);
console.log(body);
process.exit(lowered.length > 0 ? 0 : 3);

The why column is what makes review fast. A reviewer looking at | link-name | 412 | 390 | floor 390 × 1 | does not need to open the history file to know where the number came from, and a row reading pinned by allowance until 2026-08-22 is a visible reminder that a deadline exists.

# Choosing the Interval

Run the job on the sprint boundary, not on every merge and not once a quarter. Both extremes fail, in opposite and equally annoying ways.

Ratcheting on every merge sets the ceiling to whatever the last run produced, so the first slightly noisy run installs a floor the branch cannot reach and the next pull request fails for nothing. Ratcheting quarterly means twelve weeks of fixes buy no protection at all: the slack sits open, and the one week somebody regresses contrast is a week when the gate has 150 findings of headroom to absorb it. A window of about ten runs, evaluated weekly or fortnightly, keeps the floor honest while still reclaiming slack fast enough that nobody notices it accumulating.

# Validation

Validate with --dry-run against real history before the job is ever allowed to write a file, then prove idempotency, which is the property that catches most mistakes in this kind of script.

# 1. See what would change, without touching the budget.
node a11y/ratchet/propose.mjs --dry-run

# 2. Apply it, then run again: the second run must find nothing to lower.
node a11y/ratchet/propose.mjs
node a11y/ratchet/propose.mjs --dry-run; echo "exit=$?"   # expect exit=3

# 3. Confirm the new ceilings are actually reachable on the current commit.
node a11y/scan.mjs && node a11y/budget-check.mjs; echo "exit=$?"  # expect exit=0

# 4. Confirm an expired allowance is rejected rather than silently honoured.
node a11y/ratchet/check-allowances.mjs; echo "exit=$?"

Step 2 is the important one. A proposer that keeps finding something to lower on every invocation is applying grace to an already-graced number, or reading its own written ceiling as an observation. Step 3 is the honesty check: the ratchet is only correct if the budget it wrote passes against the commit it was computed from. If step 3 exits 1, the window or minOccurrences is too permissive for this codebase — raise minOccurrences to 3 before reaching for a larger grace factor, because grace hides regressions and stability requirements do not.

The allowance checker is four lines and belongs in the ordinary pull-request gate rather than in the ratchet, so an expired allowance blocks a merge instead of waiting for the next scheduled run.

// a11y/ratchet/check-allowances.mjs — runs in the PR gate, not in the ratchet
import { readFileSync } from 'node:fs';
const { allowances } = JSON.parse(readFileSync('a11y/allowances.json', 'utf8'));
const today = new Date().toISOString().slice(0, 10);
const expired = allowances.filter((a) => a.expires < today);
for (const a of expired) {
  console.error(`EXPIRED allowance: ${a.rule} +${a.extra} (${a.owner}) — ${a.reason}`);
}
process.exit(expired.length > 0 ? 1 : 0); // an allowance with no deadline is a suppression

# Edge Cases and Conditional Guards

• A rule missing from recent history. countsByRule omits rules with zero findings, so a naive read treats absence as zero and proposes a ceiling of zero for a rule that simply had no findings on a route that was temporarily removed from the scan. The Number.isInteger filter keeps absent values out of the sample entirely, and minSamples then skips the rule rather than guessing.
• A feature that legitimately adds findings mid-sprint. A grid component shipping behind a flag can add twelve aria-required-children findings that genuinely cannot be fixed this sprint. Open an allowance with an owner, a ticket and an expiry date; the ratchet pins that rule while the allowance is open, and the pull-request gate fails the day it expires. Never widen the ceiling by hand instead — a raise with no expiry is indistinguishable from a suppression six months later.
• A scanner upgrade inside the window. Half the history was recorded on axe-core 4.10.2 and half on 4.11.0, and the newer version reclassified two rules, so the observed floors mix two definitions. Record the axe version on every history line, refuse to ratchet across a version change by discarding runs whose axe value differs from the current one, and let the window refill over the following two weeks.

# Pipeline Impact

The ratchet is a scheduled workflow that writes a branch and opens a pull request; it never pushes to the default branch and it never has permission to. The pull request then runs the ordinary accessibility gate, so an unreachable floor shows up as a red check on the ratchet’s own proposal rather than on somebody’s feature work — which is the single most important property of this design.

name: a11y-ratchet
on:
  schedule:
    - cron: '0 6 * * MON'   # Monday 06:00 UTC, the sprint boundary
  workflow_dispatch:
permissions:
  contents: write
  pull-requests: write
jobs:
  propose:
    runs-on: ubuntu-24.04
    timeout-minutes: 10
    steps:
      - uses: actions/checkout@v4
      - uses: actions/setup-node@v4
        with:
          node-version: '20'
      - name: Propose lower ceilings
        id: propose
        run: node a11y/ratchet/propose.mjs
        continue-on-error: true   # exit 3 means nothing to lower; that is not a failure
      - name: Open the proposal as a pull request
        if: steps.propose.outcome == 'success'
        env:
          GH_TOKEN: ${{ secrets.GITHUB_TOKEN }}
        run: |
          BRANCH="a11y/ratchet-$(date +%Y-%m-%d)"
          git config user.name  'a11y-ratchet[bot]'
          git config user.email 'a11y-ratchet@users.noreply.github.com'
          git checkout -b "$BRANCH"
          git add a11y/budget.json
          git commit -m 'chore(a11y): ratchet violation ceilings down'
          git push origin "$BRANCH"
          gh pr create --title 'chore(a11y): ratchet violation ceilings down' \
            --body-file a11y/ratchet/proposal.md \
            --label a11y-ratchet --label automated
      - name: Close stale ratchet proposals
        env:
          GH_TOKEN: ${{ secrets.GITHUB_TOKEN }}
        run: |
          # Only the newest proposal should be open; older ones are already superseded.
          gh pr list --label a11y-ratchet --state open --json number,createdAt \
            --jq 'sort_by(.createdAt) | .[:-1][].number' \
            | xargs -r -n1 gh pr close --comment 'Superseded by a newer ratchet run.'

Three operational rules keep this from becoming noise. The proposal must be labelled so the stale-closing step can find it and so reviewers can filter it out of their queue. The ratchet pull request must be subject to the same required checks as any other change — register the accessibility gate in branch protection as described in pull request gating and branch policies — because that is the mechanism that catches an unreachable floor. And the reclaimed-slack number in the body is worth publishing to the same place as the rest of the accessibility trend data, since “reclaimed 214 findings of headroom this quarter” is the one metric that shows the gate is tightening and not just holding, as covered in tracking accessibility violation trends across sprints.

# Common Pitfalls

• Ratcheting from the last observed count instead of a repeated minimum, which installs a floor from the one run where a lazy gallery never rendered.
• Letting the job commit straight to the default branch, so an unreachable floor breaks everyone’s build and the whole mechanism gets deleted the same afternoon.
• Applying a generous global grace factor (1.2 or worse) to hide count instability, which quietly reopens 20% of every ceiling as headroom for regressions.
• Treating a rule absent from countsByRule as zero, which proposes a ceiling of zero for a rule whose route was temporarily out of the scan list.
• Allowing allowances without an expires date, turning the escape hatch into a permanent suppression that no report ever surfaces again.
• Ratcheting across an axe-core upgrade, so floors computed under two different rule definitions get mixed into one number nobody can explain.

# FAQ

Why open a pull request rather than committing the new ceilings directly? Because a ceiling is policy, and because the pull request is the test. Running the ordinary accessibility gate on the proposal proves the new numbers are reachable on real code before they can affect anyone else, and a floor that turns out to be too tight fails on the bot’s own branch where the fix is to close the pull request. A direct commit skips both the review and the test, and the first bad proposal then blocks every developer at once.

Should the ratchet ever raise a ceiling? No. Automated raising removes the friction that makes a regression visible, and the whole value of the mechanism is that ceilings move in exactly one direction. Legitimate additions go through the allowance file, which carries an owner, a reason and a deadline, and which the pull-request gate enforces. If allowances are being opened every sprint for the same rule, that is a signal to fix the underlying component rather than to relax the ratchet.

How does the window interact with a monorepo where each package is scanned separately? Each package needs its own history stream and its own requireRoutes value, otherwise a run that only scanned two of five packages contributes a floor computed from an incomplete application. Key the history lines by package, run the proposer once per package, and combine the proposals into one pull request — that path is the natural companion to per-directory accessibility budgets in legacy code, where the budget is already keyed by path.

# Related

• Progressive Threshold Management — the parent guide on baselines, fingerprints and per-rule budgets.
• Setting Up Progressive Accessibility Thresholds in CI — the manual day-one version of the ceilings this job automates.
• Reporting, Dashboards & Violation Tracking — where the history stream the ratchet reads also becomes a trend chart.