Caching Browser Binaries for axe-core Scans in CI

This guide is part of Docker-Based Pipeline Execution, and it does exactly one thing: stop the accessibility job downloading a browser it already downloaded yesterday. On a cold run, fetching and extracting a Chromium bundle costs 25 to 40 seconds of the job’s wall clock and roughly 170 MB of egress, every run, on every branch. That is the single largest fixed cost in a containerised axe-core scan and the easiest one to remove — provided the cache key tracks the thing that actually changes the binary, and provided you can tell the difference between a cache that worked and a cache that reported a hit and then downloaded anyway.

Root Cause

Neither Playwright nor Puppeteer ships a browser inside its npm package. The package contains a driver and a version manifest; the browser itself is fetched on first use into a directory outside node_modules. Playwright uses $HOME/.cache/ms-playwright on Linux, overridable with PLAYWRIGHT_BROWSERS_PATH (and note that the special value 0 relocates browsers into node_modules, which changes what you have to cache). Puppeteer uses $HOME/.cache/puppeteer, overridable with PUPPETEER_CACHE_DIR or the cacheDirectory key in .puppeteerrc.cjs. axe-core itself is pure JavaScript and lives in node_modules, so the dependency cache already covers it — the expensive artifact is always the browser.

Inside a fresh CI container both of those directories are empty, so npx playwright install chromium re-fetches the full bundle. Teams then reach for their provider’s cache action and key it on something convenient rather than something correct. A key built from the branch name restores whatever that branch happened to write last, which means a driver upgrade quietly reuses the previous revision and the scan runs against a browser the driver was not tested against. A key built only from the lockfile hash is closer but still wrong in the other direction: the lockfile changes when any dependency moves, so an unrelated dependency bump throws away a perfectly good browser. The key has to contain both the resolved driver version and the lockfile hash, and nothing whose value is unrelated to the binary.

There is a second, quieter failure. A restored cache directory is not the same thing as a usable browser. The cache holds the browser bundle; it does not hold the system libraries that playwright install --with-deps installs through apt-get. Restore the cache into a container that never ran install-deps and Chromium fails at launch with error while loading shared libraries: libnss3.so, which reads like a broken image rather than a cache problem. And when the restored directory contains chromium-1140 while the pinned driver wants chromium-1148, playwright install simply downloads the missing revision, prints one line about it, and exits zero — the job stays green, the cache stays permanently stale, and the 30 seconds you thought you saved is still being spent every run.

Anatomy of a browser-binary cache key The upper band shows a cache key built from the branch name and explains that it restores whatever the branch last wrote. The lower row breaks a correct key into four segments: a fixed namespace, the runner architecture, the resolved driver version and the lockfile hash, each labelled with the event that changes it, and a closing band notes that a restore requires all four to match. A key that can only match when the binary matches key: pw-browsers-${{ github.ref_name }} restores whatever that branch last wrote, so a driver bump reuses a stale revision keyed on the two facts that decide the binary pw namespace linux-x64 runner arch 1.47.2 driver version a3f91c0e4b7d lockfile hash changes: never changes: runner image changes: driver bump changes: any dependency Each segment must be able to change on its own, or the key stops tracking the binary. A restore only happens when all four segments match, so a stale revision is unreachable.
The driver version and the lockfile hash are the only two inputs that can change which bytes belong in the cache; anything else in the key is either noise or a bug.

Configuration

If the pipeline already builds a container image, the browser belongs in an image layer and the cache action is redundant. A layer is content-addressed and deduplicated by the registry, it is pulled once and reused by every job in the run, and — crucially — the same layer that holds the browser can hold the apt packages the browser links against, so the two can never drift apart. A restored cache directory can only ever hold half the problem.

The ordering below is what makes the layer stick. System dependencies come first because they change least; the browser download sits directly after npm ci and before any application code, so editing a component does not re-fetch 170 MB.

# syntax=docker/dockerfile:1.7
# Dockerfile.a11y — the browser is a layer, not a restored directory.
FROM node:20.18.1-bookworm-slim AS deps

ENV DEBIAN_FRONTEND=noninteractive \
    PLAYWRIGHT_BROWSERS_PATH=/ms-playwright

# install-deps ONLY: the apt packages Chromium links against. This layer is
# invalidated by the driver's minor version, not by the lockfile, so it survives
# most dependency bumps. A restored CI cache never contains any of this.
RUN npx --yes playwright@1.47.2 install-deps chromium \
 && rm -rf /var/lib/apt/lists/*

FROM deps AS browser
WORKDIR /app
# Manifests only. npm ci and the download below are one cache generation.
COPY package.json package-lock.json ./
RUN --mount=type=cache,target=/root/.npm npm ci --omit=dev

# No --with-deps here: the apt work already happened in the deps stage, and
# repeating it would put an apt layer above the npm layer where it churns.
RUN npx playwright install chromium \
 && chmod -R a+rX /ms-playwright

FROM browser AS scan
WORKDIR /app
# Application code last: a source edit rebuilds this layer and nothing below it.
COPY scripts ./scripts
COPY dist ./dist
RUN useradd --create-home --uid 10001 scanner
USER scanner
ENTRYPOINT ["node", "scripts/axe-scan.mjs"]

When the pipeline does not build an image — a plain Node job running axe-core through Playwright directly on the runner — a cache action is the right tool, with two rules. Resolve the driver version at runtime rather than hardcoding it, and keep install-deps outside the cache guard so the system libraries are present whether or not the browser was restored.

- uses: actions/checkout@v4
- uses: actions/setup-node@v4
  with:
    node-version: '20'
    cache: npm
- run: npm ci
# Resolve the version the lockfile actually installed, so the key cannot lie.
- id: driver
  run: |
    echo "version=$(node -p "require('playwright/package.json').version")" \
      >> "$GITHUB_OUTPUT"
# Namespace, arch, driver version, lockfile hash. No branch, no run id.
- id: key
  env:
    DRIVER: ${{ steps.driver.outputs.version }}
    LOCK: ${{ hashFiles('package-lock.json') }}
  run: |
    echo "cache=pw-${RUNNER_OS}-${RUNNER_ARCH}-${DRIVER}-${LOCK}" >> "$GITHUB_OUTPUT"
- name: Restore the browser bundle
  id: browser-cache
  uses: actions/cache@v4
  with:
    path: ~/.cache/ms-playwright
    key: ${{ steps.key.outputs.cache }}
    # No restore-keys: a partial match here means the wrong revision.
- name: System libraries for Chromium
  # Always runs: these apt packages are NOT in the cached directory.
  run: npx playwright install-deps chromium
- name: Download the browser only when the cache did not supply it
  if: steps.browser-cache.outputs.cache-hit != 'true'
  run: npx playwright install chromium
- name: Fail if a reported hit did not actually supply the browser
  run: bash scripts/assert-browser-cache.sh
- name: Run the axe-core scan
  run: node scripts/axe-scan.mjs

Puppeteer needs the same treatment with different paths. Pin the cache directory into the repository so the value is identical on a laptop, in a container and on a runner, rather than depending on whatever $HOME happens to be for the CI user — a detail that bites when a job switches from the default user to a non-root one and the cache silently moves.

// .puppeteerrc.cjs — one cache location for every environment.
const { join } = require('node:path');

module.exports = {
  // Relative to the project root, so PLAYWRIGHT/HOME differences stop mattering.
  cacheDirectory: join(__dirname, '.cache', 'puppeteer'),
  // Never skip the download in CI: a missing browser must fail at install time,
  // not at scan time in the middle of a WCAG 2.2 SC 1.4.3 contrast check.
  skipDownload: false,
};
Baked image layer versus restored CI cache The upper lane shows a job using a baked image layer: a 24 second image pull followed by a 42 second scan, 66 seconds in total. The lower lane shows a job using a cache action: 19 seconds of npm ci, 22 seconds of cache restore, 30 seconds installing system packages, then the same 42 second scan, 113 seconds in total. Where the time goes: baked layer versus restored cache image layer 66 s total image pull 24 s axe-core scan 42 s cache action 113 s total npm ci 19 s restore 22 s install-deps 30 s axe-core scan 42 s 0 30 s 60 s 90 s 120 s The restored cache still pays for npm ci and for the system libraries it never contained.
Both lanes avoid the browser download; only the baked layer also avoids the install and the apt work, because those are already inside the image.

Validation

The guard below is the part most pipelines are missing. playwright install --dry-run prints the install location the driver expects for the pinned version, which is the only authoritative answer to “is the right revision on disk”. Comparing that path against the filesystem turns a silent re-download into a failed step with an actionable message.

#!/usr/bin/env bash
# scripts/assert-browser-cache.sh
# Fails when the pinned driver's expected browser directory is absent, which is
# the signal that a "cache hit" restored the wrong revision.
set -euo pipefail

# --dry-run resolves the revision from the installed driver and prints, e.g.
#   Install location: /home/runner/.cache/ms-playwright/chromium-1148
expected="$(npx playwright install --dry-run chromium \
  | awk -F': ' '/Install location/ {print $2; exit}')"

if [[ -z "$expected" ]]; then
  echo "could not resolve the expected browser path from the driver" >&2
  exit 1
fi

if [[ ! -x "$expected/chrome-linux/chrome" ]]; then
  echo "STALE CACHE: driver wants $expected but it is not on disk" >&2
  echo "present instead:" >&2
  ls -1 "$(dirname "$expected")" >&2
  exit 1
fi

echo "browser cache OK: $expected"
du -sh "$expected"   # ~170 MB for a chromium bundle; a few KB means a bad restore

Then prove the scan actually used it. A run that downloaded nothing prints no download progress, and the scan’s own report should record the browser build so a reviewer can tie a violation list to a revision.

// scripts/axe-scan.mjs — proves the cached engine renders and reports.
import { chromium } from 'playwright';
import AxeBuilder from '@axe-core/playwright';

// Throws instead of downloading if the bundle is missing, so a cache miss is a
// loud failure here rather than a silent 30-second tax.
const browser = await chromium.launch({ chromiumSandbox: true });
console.log(`engine: ${browser.version()}`); // e.g. 130.0.6723.31

const page = await browser.newPage();
await page.goto(process.env.SCAN_URL ?? 'http://app:8080/checkout');
await page.getByRole('main').waitFor({ state: 'visible' });

const results = await new AxeBuilder({ page })
  .withTags(['wcag2a', 'wcag2aa', 'wcag22aa'])
  .analyze();

const serious = results.violations.filter(
  (v) => v.impact === 'critical' || v.impact === 'serious',
);
console.log(`${serious.length} blocking of ${results.violations.length} total`);
await browser.close();
process.exit(serious.length > 0 ? 1 : 0); // non-zero fails the gate
The stale-cache failure a green job hides A cache action reports a hit and a directory is restored. The question is whether the browser revision the pinned driver expects exists in that directory. If it does, launch uses the cached binary and nothing is downloaded. If it does not, the install step downloads the bundle again, prints a single log line, exits zero and leaves the cache permanently stale, which is why an explicit guard step is needed. A cache hit is not the same as a usable browser cache action reports cache-hit: true a directory was restored does the revision the pinned driver expects exist in that directory? no yes install downloads 170 MB one log line, exit code 0 job green, cache never repaired launch uses the cached binary zero bytes downloaded about 30 s off the job guard: compare the dry-run install location against the disk and fail the step instead of downloading silently
Without the guard step, the left branch is indistinguishable from the right one in a green build log, which is why stale browser caches survive for months.

Edge Cases and Conditional Guards

  • Multiple engines in one job. Each engine is a separate directory inside the cache path, so a job that scans Chromium and WebKit needs both installed before the guard runs, and the guard needs a loop over the engines rather than a single hardcoded chromium. Restrict the engine list to what the accessibility gate actually asserts on; a second engine roughly doubles both the cache size and the restore time for a small increase in real coverage.
  • Non-root containers and $HOME. ~/.cache/ms-playwright resolves against the current user’s home, so a cache populated as root and then read as uid 10001 is simply not there. Set PLAYWRIGHT_BROWSERS_PATH to an absolute path outside any home directory and chmod -R a+rX it, which is why the Dockerfile above installs into /ms-playwright.
  • Sharded and matrix jobs. Parallel shards started at the same moment all miss the cache together on the first run after a driver bump, so the download cost is multiplied by the shard count exactly once. Warm the cache in a single upstream job — or build the image once and let the shards pull it — before fanning out, as described in sharding axe-core scans across parallel CI jobs.

Pipeline Impact

Caching changes nothing about what the gate decides. The scan’s exit code still comes from the violation impacts, the artifact is unchanged, and the rule set configured in the axe-core configuration and setup guide behaves identically against a restored binary and a freshly downloaded one — the bytes are the same bytes. What changes is the job’s position in the critical path. A scan that finishes in 70 seconds instead of 115 is a scan reviewers wait for; a scan that takes three minutes is a scan somebody eventually moves to a nightly schedule, at which point it stops gating anything.

The second effect is failure attribution. Once the guard step exists, “the browser was re-downloaded” is its own red step with its own message, instead of an invisible 30 seconds inside a step called Install browser. That matters when a driver upgrade lands: the guard fires on the first run after the bump, confirms the cache correctly invalidated, and then goes quiet — which is exactly the behaviour you want from a cache you are trusting to feed a required status check.

Common Pitfalls

  • A branch or run-id in the key. Both make the cache either unshareable or wrong. The key should be a pure function of the runner architecture, the driver version and the lockfile.
  • restore-keys as a fallback. A prefix match restores a different revision and reports a hit, which is precisely the stale-cache case the guard exists to catch. Leave it out for browser bundles.
  • Caching node_modules and expecting the browser. The bundle lives outside node_modules unless PLAYWRIGHT_BROWSERS_PATH=0 is set, and that setting has its own consequence: the browser now invalidates on every dependency change.
  • Running install --with-deps behind the cache-hit guard. On a hit, the apt packages are skipped and Chromium fails to load libnss3.so. Split install-deps out and run it unconditionally, or bake it into the image.
  • Never rotating the namespace. Old revisions accumulate in the cache backend and count against the repository’s cache quota, evicting the entry you actually wanted. Bump the pw prefix once a quarter.
  • Trusting cache-hit: true. It reports that a key matched and a tarball was extracted, not that the browser the driver wants is present and executable. Assert the path.

FAQ

Which directory does each tool actually download into? Playwright uses $HOME/.cache/ms-playwright on Linux and honours PLAYWRIGHT_BROWSERS_PATH, where the literal value 0 means “inside node_modules”. Puppeteer uses $HOME/.cache/puppeteer and honours PUPPETEER_CACHE_DIR or cacheDirectory in .puppeteerrc.cjs. axe-core needs no cache of its own beyond the normal dependency cache, because it is JavaScript that ships inside the package.

Is a layer cache always better than a cache action? When an image is already being built, yes, because the layer carries both the browser and its system libraries and the registry deduplicates it across jobs. When there is no image, a cache action is the only option and works well with a correct key. The one case worth avoiding is using both at once: two caches with different invalidation rules eventually disagree, and the resulting bug is very hard to see in a log.

Does a cached browser change what the scan reports? No, provided the cached revision matches the pinned driver — that is the whole reason the guard checks the revision rather than just the directory. A restored bundle is byte-identical to a downloaded one, so contrast ratios, accessible names and every WCAG 2.2 AA finding come out the same. A mismatched revision does change results, which is why a prefix-matching restore key is dangerous rather than merely wasteful.

Back to Docker Based Pipeline Execution