Verifying Live Region Announcements in Automated Tests

A status message is the only part of an interface whose correctness is defined by something that happened rather than by something that exists. “12 results found” is not accessible because a role="status" element contains that text; it is accessible because the text was written into a region that assistive technology was already tracking, with a politeness that matches how urgent the message is. This guide is part of Screen Reader Automation Testing, and it covers how to prove that in a headless browser: install a recorder before the application boots, capture every mutation to every live region with its resolved politeness, and assert what would have been spoken and in what order to satisfy WCAG 2.2 SC 4.1.3 (Status Messages).

Root Cause

Screen readers watch live regions for changes. When a region is already in the accessibility tree and its contents change, the reader speaks the change; when a node arrives in the tree with its contents already in place, most readers treat that as initial content and say nothing. This makes the most natural implementation of a toast the broken one: build a <div role="status">Saved</div>, append it to the body, remove it after three seconds. Every attribute is correct, axe reports nothing, and no user hears a word. The region has to exist first and receive its text second, which is a temporal property that no snapshot of the DOM can express.

Politeness is the second trap, and it is mostly implicit. aria-live="polite" on a container is explicit and carries aria-atomic="false" by default, so a reader announces only the changed part of the region. role="status" carries an implicit aria-live="polite" and an implicit aria-atomic="true", so the whole region is re-read on every change — which is why a status region containing both a heading and a message reads both every time. role="alert" carries implicit assertive politeness, meaning it interrupts speech in progress, and role="log" carries polite politeness with aria-relevant="additions". Teams routinely reach for role="alert" on a save confirmation, which interrupts whatever the user was reading to deliver good news, and reach for role="status" on a validation error that then waits behind a paragraph of unrelated speech.

The third problem is that politeness is inherited. aria-live applies to the region and everything inside it, so writing into a grandchild of the region announces through the region’s politeness, not through the child’s markup. A recorder that only watches nodes matching a live-region selector will miss most real announcements, because frameworks write into an inner <span>. The resolution has to walk up from the mutated node to its nearest live-region ancestor and read the politeness from there. Note the boundary with the routing-specific case: detecting detached aria-live regions in SPA navigation is about a region’s node identity surviving a route change, while this page is about what the region says and how urgently, on any page, route change or not.

Implicit semantics of four live-region declarations Rows for aria-live polite, role status, role alert and role log. Each row shows the resolved politeness, the resolved aria-atomic value and the resulting reader behaviour, from announcing only the changed part to interrupting speech in progress. As authored Resolved What the reader does aria-live="polite" politeness: polite atomic: false (default) speaks only the changed part role="status" politeness: polite atomic: true (implicit) re-reads the whole region role="alert" politeness: assertive atomic: true (implicit) interrupts speech in progress role="log" politeness: polite relevant: additions speaks appended entries only An explicit aria-live attribute always wins over the role's implicit value, and both are inherited by descendants. The recorder must therefore resolve politeness from the nearest region ancestor, not from the mutated node.
Three of these four declarations look interchangeable in markup and behave differently in speech, which is why the recorder stores the resolved values rather than the authored ones.

Configuration

The recorder is a standalone script installed with page.addInitScript, which runs in every new document before any application code. That timing is not optional: a script injected after page.goto starts observing after the application has already had its first opportunity to announce, so the first message of every page load is invisible to the test.

// tests/reader/announce-recorder.js — installed via page.addInitScript
(() => {
  const SELECTOR =
    '[aria-live], [role="status"], [role="alert"], [role="log"], [role="timer"]';
  const IMPLICIT_LIVE = { status: 'polite', alert: 'assertive', log: 'polite',
    timer: 'off' };
  const IMPLICIT_ATOMIC = { status: true, alert: true };
  const seen = new WeakSet();          // regions already in the tree
  window.__srTranscript = [];

  const roleOf = (el) => (el.getAttribute('role') || '').toLowerCase();
  // An explicit aria-live attribute always beats the role's implicit value.
  const politenessOf = (el) =>
    el.getAttribute('aria-live') || IMPLICIT_LIVE[roleOf(el)] || 'off';
  const atomicOf = (el) => {
    const explicit = el.getAttribute('aria-atomic');
    if (explicit !== null) return explicit === 'true';
    return IMPLICIT_ATOMIC[roleOf(el)] === true;
  };
  // Politeness is inherited, so walk up from the mutated node.
  const regionFor = (node) => {
    const el = node.nodeType === 1 ? node : node.parentElement;
    return el ? el.closest(SELECTOR) : null;
  };

  const record = (region, node) => {
    if (!region) return;
    const politeness = politenessOf(region);
    const atomic = atomicOf(region);
    const text = (atomic ? region.textContent : node.textContent || '').trim();
    if (!text) return;                 // clearing a region is not an utterance
    const preexisting = seen.has(region);
    seen.add(region);
    // aria-busy defers the announcement until the region is settled.
    const busy = region.closest('[aria-busy="true"]') !== null;
    window.__srTranscript.push({
      seq: window.__srTranscript.length,
      at: Math.round(performance.now()),
      politeness, atomic, text,
      wouldAnnounce: preexisting && politeness !== 'off' && !busy,
      reason: !preexisting ? 'region-inserted-with-its-text'
        : politeness === 'off' ? 'politeness-resolves-to-off'
        : busy ? 'region-aria-busy' : null,
    });
  };

  const index = () => document.querySelectorAll(SELECTOR).forEach((r) => seen.add(r));
  document.addEventListener('DOMContentLoaded', index);

  new MutationObserver((records) => {
    for (const entry of records) {
      if (entry.type === 'characterData') record(regionFor(entry.target), entry.target);
      for (const node of entry.addedNodes) {
        if (node.nodeType === 1 && node.matches(SELECTOR)) record(node, node);
        else record(regionFor(entry.target), node);
      }
    }
  }).observe(document.documentElement, {
    childList: true, subtree: true, characterData: true,
  });
})();

The wouldAnnounce flag is the whole point. An entry with wouldAnnounce: false and reason: 'region-inserted-with-its-text' is a message the code believes it delivered and a reader would never speak, which is a defect the test must fail on even though the text is demonstrably in the DOM. The spec below drives a save flow that produces one progress message and one validation error, then asserts three things: the ordered pair of announcements, the politeness of each, and that nothing was silent.

// tests/reader/announcements.spec.ts
import { test, expect } from '@playwright/test';

type Entry = { politeness: string; text: string; wouldAnnounce: boolean;
  reason: string | null };

test('save failure announces politely then assertively (SC 4.1.3)', async ({ page }) => {
  await page.addInitScript({ path: 'tests/reader/announce-recorder.js' });
  await page.goto('/settings/notifications');

  await page.getByLabel('Notification email').fill('not-an-address');
  await page.getByRole('button', { name: 'Save preferences' }).click();

  const audible = async () =>
    (await page.evaluate(() => window.__srTranscript as Entry[]))
      .filter((entry) => entry.wouldAnnounce)
      .map((entry) => `${entry.politeness}: ${entry.text}`);

  // Poll rather than wait a fixed time: the second message follows a request.
  await expect.poll(audible, { timeout: 5000 }).toEqual([
    'polite: Saving preferences…',
    'assertive: Enter a valid email address, for example name@example.com',
  ]);

  // Any recorded write that would not be spoken is a defect, not a detail.
  const silent = (await page.evaluate(() => window.__srTranscript as Entry[]))
    .filter((entry) => !entry.wouldAnnounce);
  expect(silent.map((entry) => `${entry.reason}: ${entry.text}`)).toEqual([]);
});
How the recorder classifies one mutation A mutation is checked for a live-region ancestor, then for whether that region already existed, then for resolved politeness and aria-busy. The three failing branches produce a transcript entry marked as not announced with a reason, and the passing branch produces an audible entry. mutation observed live-region ancestor? region existed before? politeness off, or busy? ignored — ordinary DOM churn logged: inserted with its text logged: off or deferred by aria-busy audible entry: politeness + text this is what the test compares yes yes no no no yes
Every branch except the first produces a transcript entry, so a message that cannot be spoken is recorded with its reason instead of disappearing.

Validation

Prove both halves of the assertion. Break the ordering first by giving the validation error role="status" instead of role="alert": the text is identical, the transcript still has two entries, and the politeness column no longer matches. Then break the attachment by rendering the progress message and its container together, which produces a silent entry with a reason.

npx playwright test tests/reader/announcements.spec.ts --reporter=list

# 1. Error region downgraded from role="alert" to role="status":
#   ✘ save failure announces politely then assertively (SC 4.1.3)
#     - "assertive: Enter a valid email address, for example name@example.com"
#     + "polite: Enter a valid email address, for example name@example.com"

# 2. Progress region created together with its text:
#   ✘ save failure announces politely then assertively (SC 4.1.3)
#     Expected: [ "region-inserted-with-its-text: Saving preferences…" ] to equal []
#     (the text is in the DOM and a screen reader says nothing)

# 3. Both fixed — region mounted at boot, error region role="alert":
#   ✓ save failure announces politely then assertively (SC 4.1.3) (1.4s)
#   1 passed, exit code 0

The order in the expectation is the delivery order, not the write order, and the two differ whenever politeness differs. A polite message written while the reader is mid-sentence waits for a pause; an assertive message written a moment later cuts in front of it. That is why the transcript stores politeness alongside sequence: comparing writes in isolation would pass a form that reports its error only after reading out an unrelated confirmation.

Write order versus spoken order Three writes occur in sequence: a polite saving message, a polite row-count update and an assertive validation error. The spoken order places the assertive error first because it interrupts, then the two polite messages in the order they were queued. Write order (recorder sequence) seq 0 · polite "Saving preferences…" seq 1 · polite "3 channels selected" seq 2 · assertive "Enter a valid email" 0 ms 120 ms 180 ms Spoken order (what the assertion encodes) 1st — interrupts "Enter a valid email" 2nd — queued "Saving preferences…" 3rd — queued "3 channels selected" Three writes in 180 ms, two different delivery positions: the assertive region jumps the queue it was written after. Asserting text alone would accept any of the six possible orderings.
Storing politeness with each entry lets the expectation describe delivery rather than authorship, which is the property a user actually experiences.

The second failure message is the reason this technique exists. Nothing else in a headless pipeline distinguishes “the text is present” from “the text was announced”, and the difference is total for anyone relying on speech. Dumping the full transcript on failure makes the diagnosis immediate:

# Attach the transcript to the test result for triage, then read it locally.
npx playwright test tests/reader/announcements.spec.ts --reporter=json \
  | node -e "let s='';process.stdin.on('data',d=>s+=d).on('end',()=>{
      const r=JSON.parse(s);
      console.log(JSON.stringify(r.suites?.[0]?.specs?.[0]?.tests?.[0]?.status));
    })"
# "failed"

Edge Cases and Conditional Guards

  • aria-atomic="false" with a multi-part region. When the region holds a label and a value, a partial update announces only the changed fragment, so the reader says “4” rather than “4 filters applied”. The recorder reproduces this by storing the changed node’s text rather than the region’s, which means the assertion sees the fragment and fails — correctly, because that is what a user would hear.
  • Regions hidden with display: none or aria-hidden="true". A hidden region is not in the accessibility tree, so writes into it announce nothing while textContent still reports the message. Visually-hidden CSS that keeps the node rendered is fine; display: none is not. Guard the recorder by checking region.offsetParent !== null or by asserting the region’s computed visibility in a separate expectation.
  • A toast removed before it can be spoken. A region that is unmounted 200 ms after the write may be gone before a polite announcement reaches the front of the queue. Assert that the region survives for a realistic minimum — three to five seconds — using a locator that stays attached, and treat any removal faster than that as a defect rather than as a timing quirk of the test.

Pipeline Impact

These tests belong on the pull-request gate. They run in headless Chromium, add one to three seconds per flow, and are deterministic once the polling replaces fixed waits — which puts them on the right side of the cost line drawn in the parent guide. Run them as a named Playwright project alongside the axe scan so one required status check covers both, and attach the transcript JSON to every failed test so a reviewer reads the recorded speech rather than reproducing the flow.

What they do not prove is that a reader spoke the words. The recorder models the DOM contract that makes an announcement possible; NVDA’s actual behaviour — its queueing, its verbosity settings, whether it truncates the phrase — is only observable from a real transcript, which is what the nightly job in automating NVDA output capture in a Windows CI runner exists for. The division is deliberate: the gate proves the contract on every merge, and the nightly run confirms the contract still produces the intended speech. When the two disagree, the recorder is the one to fix, because it is the model.

Common Pitfalls

  • Injecting the recorder with addScriptTag after navigation, which silently misses every announcement fired during boot and hydration.
  • Matching only elements that carry a live-region attribute, so writes into an inner <span> — the normal framework pattern — are never recorded.
  • Reaching for role="alert" on confirmations, which interrupts whatever the user was reading; reserve assertive politeness for errors and time-critical warnings.
  • Asserting that the region contains the right text instead of asserting the transcript, which passes for a region that was created with its message already inside it.
  • Ignoring aria-atomic, then being surprised that a role="status" region re-reads its heading with every update because atomic is implicitly true for that role.
  • Using a fixed wait before reading the transcript, which makes the test slower and still races the request that produces the second message on a loaded runner.

FAQ

Why record politeness instead of just asserting the text was written? Because politeness decides whether the message arrives at a useful moment. A validation error in a polite region queues behind everything the reader is currently saying, so a user can submit a form, hear a paragraph of unrelated content, and reach the error thirty seconds later; the same error in an assertive region arrives immediately. Recording the resolved value turns that from a code-review opinion into a test failure with a named expectation.

How does this differ from an axe rule for live regions? axe evaluates one instant of the DOM, so it can confirm that a region is well-formed, that its role is valid and that aria-live has a legal value. It cannot see the sequence — region attached, then text written — that determines whether anything is announced, and it has no notion of the order two messages arrive in. The recorder is a temporal observer, which makes it complementary rather than duplicative; a page can pass every live-region rule and announce nothing at all.

What about announcements triggered outside a live region entirely? Some announcements come from focus moves rather than from live regions: focusing a heading after a route change makes the reader read that heading, and focusing an input reads its label, role and description. Those are not recorded here and should not be — assert them with a focus expectation on the intended target plus a name assertion, so the test states both where focus landed and what that element is called. Live regions and focus moves are the two announcement channels, and mixing them into one recorder makes both harder to read.

Back to Screen Reader Automation Testing