Unit-Testing Custom axe Rules With Jest Fixtures

A custom rule is code, and untested code that gates merges is a liability. The awkward part is that the obvious assertion — “the fixture produces no violations” — is satisfied by a rule that works, a rule whose selector matches nothing, and a rule that was never registered at all. This guide is part of Custom Rule Testing & Distribution, and it builds a Jest plus jsdom harness that loads an HTML fixture, registers the rule under test with axe.configure, runs axe.run scoped to the fixture, and asserts membership in all four result buckets so a silent non-application fails loudly.

Root Cause

axe-core reports on nodes, not on rules. When axe.run finishes, each rule has contributed zero or more entries to violations, passes and incomplete, and if the rule’s selector matched nothing at all the rule appears once in inapplicable with an empty nodes array. From the perspective of expect(results.violations).toEqual([]), the last case is indistinguishable from a clean pass. Both are an empty array.

That matters because the two most common defects in a new custom rule both land in inapplicable. The first is a selector typo — [data-ds="tolbar"] — which is invisible in review and silently disables the rule forever. The second is forgetting that a rule authored with enabled: false, which is the sensible default when versioning rules that gate merges, does not run unless the consumer switches it on; axe simply omits it. In both cases the fixture test goes green, the rule ships, and the first person to learn it never worked is whoever audits the component six months later.

Jest adds a second trap. Its testEnvironment: 'jsdom' gives a shared document for every test in a file, and axe.configure writes into a module-level registry inside axe-core rather than into anything scoped to a test. Register a rule in the first test and it is still registered in the fourth; leave a fixture in document.body and the next test’s scan sees it. The result is order-dependent flake that disappears when a developer runs the single failing test in isolation, which is the least useful kind of failure to debug.

Empty violations has two very different causes A scan with no violations may mean the rule ran and passed, or that the selector matched nothing and the rule landed in inapplicable; only a bucket assertion separates them. axe.run(fixture) violations = [] rule applied, passed id present in passes rule never applied id in inapplicable real signal safe to gate on false comfort typo or disabled the assertion expect(violations).toEqual([]) cannot tell these apart so assert the rule id lands in the bucket you expect
Both branches yield an empty violations array; only a bucket-membership assertion distinguishes them.

Configuration

Start with the Jest environment, because the defaults do not give axe what it needs:

// jest.config.js
module.exports = {
  testEnvironment: 'jsdom',
  testEnvironmentOptions: {
    // axe schedules work via requestAnimationFrame; without this it never resolves.
    pretendToBeVisual: true,
  },
  testMatch: ['<rootDir>/tests/**/*.test.js'],
  // Fixture HTML is data, not a module: keep the transform away from it.
  transformIgnorePatterns: ['<rootDir>/tests/fixtures/'],
  testTimeout: 10000,      // a hung axe.run should fail fast, not stall the job
};

Then two files: a helper that owns fixture loading, registration and teardown, and a spec that asserts buckets. The helper resets state on every call so no test inherits another’s registry or DOM:

// tests/support/rule-fixture.js
const fs = require('node:fs');
const path = require('node:path');

const FIXTURES = path.join(__dirname, '..', 'fixtures');

/**
 * Mount an HTML fixture, register one rule, and scan only that rule.
 * @param {string} file     fixture filename under tests/fixtures
 * @param {object} rule     { rule, checks } definition under test
 * @param {string} ruleId   the rule id to scope the run to
 */
async function scanFixture(file, { rule, checks }, ruleId) {
  // axe.configure mutates module state, so start from a clean axe every time.
  jest.resetModules();
  const axe = require('axe-core');

  const root = document.createElement('div');
  root.id = 'fixture-root';
  root.innerHTML = fs.readFileSync(path.join(FIXTURES, file), 'utf8');
  document.body.appendChild(root);

  axe.configure({ checks, rules: [{ ...rule, enabled: true }] });  // force opt-in rules on

  const results = await axe.run(root, {
    runOnly: { type: 'rule', values: [ruleId] },   // nothing else can muddy the buckets
    // jsdom reports every box as 0x0, so size-aware rules would be pure noise.
    rules: { 'color-contrast': { enabled: false } },
    resultTypes: ['violations', 'passes', 'incomplete', 'inapplicable'],
  });

  root.remove();          // leave the DOM as we found it for the next test
  return results;
}

/** Convenience: the set of rule ids present in one result bucket. */
const idsIn = (bucket) => new Set(bucket.map((entry) => entry.id));

module.exports = { scanFixture, idsIn };

resultTypes is the flag that makes this work. Left unset, axe-core prunes buckets it assumes the caller does not care about, and results.inapplicable can come back empty even when the rule genuinely did not apply — which would defeat the whole exercise. Listing all four keeps every bucket populated.

The rule itself is an ordinary definition of the kind produced in component-specific rule writing, exported so both the spec and the shipped package import the same object:

// rules/toolbar-controls-named.js
const check = {
  id: 'ds-name-present',
  evaluate: function (node) {
    const label = (node.getAttribute('aria-label') || '').trim();
    if (label) return true;
    if ((node.textContent || '').trim()) return true;
    if (node.hasAttribute('aria-labelledby')) return undefined;  // cannot resolve: review
    return false;
  },
};

const rule = {
  id: 'ds-toolbar-controls-named',
  selector: '[data-ds="toolbar"] button, [data-ds="toolbar"] [role="button"]',
  tags: ['wcag2a', 'wcag412'],       // SC 4.1.2 Name, Role, Value
  impact: 'serious',
  enabled: false,                     // opt-in until consumers adopt it
  all: ['ds-name-present'],
  metadata: { description: 'Toolbar controls must expose an accessible name' },
};

module.exports = { rule, checks: [check] };

The fixtures are plain HTML fragments with no wrapper document, because the helper supplies the container. Keep them minimal and keep the failing one plural — two bad controls, not one:

<!-- tests/fixtures/toolbar.bad.html -->
<div data-ds="toolbar">
  <button type="button"><svg aria-hidden="true" width="16" height="16"></svg></button>
  <span role="button" tabindex="0"></span>
  <button type="button" aria-label="Undo">Undo</button>
</div>

The third control is named and belongs in the fixture on purpose: it proves the rule reports two nodes rather than flagging the whole container, which is the failure mode a single-control fixture cannot detect. The good fixture is the same markup with a label on each control; the ambiguous fixture points aria-labelledby at an id that is not in the fragment; and plain-page.html contains a heading and a paragraph with no data-ds attribute anywhere.

Returning undefined when aria-labelledby is present but unresolvable is deliberate. jsdom has the referenced element, but a real page may put it in another document fragment, so the check declines to decide and the node routes to incomplete. That is a third outcome the tests have to cover explicitly.

Per-test isolation steps in the Jest helper Each test resets the axe module registry, mounts the fixture into a fresh container, configures the single rule, runs a scoped scan, then removes the container so no state leaks forward. resetModules fresh axe mount fixture #fixture-root axe.configure enabled: true axe.run(root) runOnly one id root.remove clean DOM skip any step and the suite passes alone but flakes in a full run order dependence is the usual symptom of a leaked registry
The loop back to resetModules is what makes each test independent of the ones before it.

Validation

Four fixtures, four assertions, and every one names the bucket it expects. The inapplicable test is the one that would have caught the selector typo:

// tests/toolbar-controls-named.test.js
const { scanFixture, idsIn } = require('./support/rule-fixture.js');
const definition = require('../rules/toolbar-controls-named.js');

const RULE = 'ds-toolbar-controls-named';

test('bad fixture: both unnamed controls appear in violations', async () => {
  const res = await scanFixture('toolbar.bad.html', definition, RULE);
  expect(idsIn(res.violations)).toContain(RULE);
  const entry = res.violations.find((v) => v.id === RULE);
  expect(entry.nodes).toHaveLength(2);            // two bad controls, two nodes
  expect(entry.impact).toBe('serious');
});

test('good fixture: the rule ran and landed in passes', async () => {
  const res = await scanFixture('toolbar.good.html', definition, RULE);
  expect(res.violations).toEqual([]);
  expect(idsIn(res.passes)).toContain(RULE);      // proves it actually applied
});

test('unresolvable aria-labelledby lands in incomplete, not passes', async () => {
  const res = await scanFixture('toolbar.ambiguous.html', definition, RULE);
  expect(idsIn(res.incomplete)).toContain(RULE);
  expect(idsIn(res.passes)).not.toContain(RULE);
  expect(res.violations).toEqual([]);             // undecidable is not a failure
});

test('no toolbar markup: the rule is inapplicable, not passing', async () => {
  const res = await scanFixture('plain-page.html', definition, RULE);
  expect(idsIn(res.inapplicable)).toContain(RULE);
  expect(idsIn(res.passes)).not.toContain(RULE);
});

Run it and the output distinguishes the failure modes by name rather than by violation count:

npx jest tests/toolbar-controls-named.test.js --verbose
# PASS  tests/toolbar-controls-named.test.js
#   ✓ bad fixture: both unnamed controls appear in violations (74 ms)
#   ✓ good fixture: the rule ran and landed in passes (41 ms)
#   ✓ unresolvable aria-labelledby lands in incomplete, not passes (38 ms)
#   ✓ no toolbar markup: the rule is inapplicable, not passing (33 ms)
#
# After breaking the selector to [data-ds="tolbar"]:
#   ✕ bad fixture: both unnamed controls appear in violations
#     Expected value: "ds-toolbar-controls-named"
#     Received set:   Set {}
#   ✕ good fixture: the rule ran and landed in passes
#     Expected value: "ds-toolbar-controls-named"
#     Received set:   Set {}

Two tests fail instead of zero, and the message says the rule id is absent from the bucket rather than reporting a count that happens to be right. That is the difference between a suite that guards the rule and a suite that decorates it.

Try the second sabotage as well: delete the enabled: true override in the helper and re-run. Every bucket comes back empty, all four tests fail, and the reason is visible in the diff rather than in a runtime message — which is exactly the outcome wanted, because an opt-in rule that quietly never runs in a consumer’s pipeline is the single most expensive way for a rule package to be wrong.

Edge Cases and Conditional Guards

  • aria-busy fixtures. A fixture with aria-busy="true" on the container models a component mid-hydration. Decide explicitly whether the rule should skip busy subtrees; if it should, add a fixture asserting inapplicable for the busy case so the behaviour is pinned rather than incidental.
  • Shadow roots in jsdom. jsdom implements attachShadow, but a fixture parsed from innerHTML has no declarative shadow DOM unless the fixture script attaches it. Mount those fixtures through a small setup function that calls attachShadow and assigns shadowRoot.innerHTML, then assert the same four buckets.
  • Async checks. A check with async: true must call this.async() and resolve its callback, or axe.run never settles and Jest fails on its own five-second default. Give async-rule tests an explicit third-argument timeout and treat a timeout as a rule bug, not a flaky test.

Pipeline Impact

These tests belong in the ordinary unit-test job, not the accessibility job. They need no browser, no server and no built application, so they run in the same few seconds as the rest of the suite and fail before anything is deployed to a preview environment — which is the earliest possible point to catch a broken rule.

The placement is worth being deliberate about. A broken rule discovered by the browser scan costs the full pipeline: install, build, start the server, launch Chromium, scan, and only then a red check — typically two minutes of runner time before anyone learns that a selector has a typo in it. The same defect caught by a fixture test costs under a second and reports a rule id instead of a violation count.

Feedback point for a broken rule, fixture test versus browser scan Fixture tests report a broken rule about two seconds into the pipeline, while the browser accessibility scan only reveals it after install, build and browser launch, roughly two minutes later. npm ci jest build serve + chromium browser axe scan annotate rule bug found: 2 s names the rule id same bug found: 118 s reports a violation count 0 s 140 s
Fixture tests move rule feedback from two minutes into the pipeline down to two seconds, and change the message from a count to a rule id.

The gate is a plain non-zero Jest exit code. What is worth adding is a coverage assertion: a final test that reads the rule directory, compares it against the set of rule ids referenced by the spec files, and fails when a rule has no fixture test at all. Without it, coverage silently decays as rules are added under deadline. Keep this job ahead of the browser scan in the GitHub Actions accessibility pipeline so a broken rule never consumes a browser slot. Publish the Jest JUnit XML as a CI artifact so a fixture regression appears in the test-report UI beside application failures, and keep the fixture HTML in the published package so consumers can re-run the corpus against their own axe-core version, as the configuration guide for axe-core recommends for any pinned scanner.

Common Pitfalls

  • Asserting only expect(violations).toEqual([]), which passes identically when the rule never applied.
  • Omitting resultTypes, so inapplicable comes back empty and the applicability test can never fail.
  • Sharing one axe module across tests, so a rule disabled in test one stays disabled for the rest of the file.
  • Leaving fixture markup attached to document.body, letting a later test scan a previous test’s DOM.
  • Writing a bad fixture with exactly one failing node, which cannot distinguish a per-node rule from a per-fixture one.
  • Leaving enabled: false in place inside the helper, so an opt-in rule never runs and every bucket is empty.

FAQ

Why not use jest-axe instead of a hand-rolled helper? jest-axe wraps axe.run and gives a matcher for “no violations”, which is precisely the assertion that cannot detect a rule that never applied. It is a good fit for scanning rendered components against the default rule set. For testing a custom rule’s own behaviour you need access to passes, incomplete and inapplicable, so a thin helper over axe.run is the more honest tool.

Does testing in jsdom mean the rule is verified for real browsers? It verifies the rule’s logic against a DOM, which is most of what a custom rule does. It does not verify anything depending on layout, computed style, or real focus behaviour, because jsdom reports every element as zero-sized and does not paint. Keep a short browser-based suite for those, and disable size-aware rules in the jsdom harness so they cannot produce noise.

How should a check signal “I cannot tell”? Return undefined. axe routes that node into incomplete, which means “needs human review” rather than pass or fail, and a test can assert the rule id is in incomplete while violations stays empty. Returning false instead would claim a violation the check cannot prove, and consumers would be blocked on something no one can fix.

Back to Custom Rule Testing Distribution