Lighthouse CI Baseline Configuration

Establishing a reproducible accessibility baseline requires deterministic execution within your deployment pipeline. Implementing a structured configuration automates regression detection. It enforces strict compliance gates before code reaches production.

This approach aligns directly with established Web Accessibility Testing Fundamentals & Tool Selection methodologies. Engineering teams can translate automated scoring into actionable workflows.

Key implementation steps include:

  • Initializing lhci autorun and lhci wizard for project scaffolding
  • Defining static vs dynamic assertion thresholds in lighthouserc.json
  • Configuring CI runners for deterministic headless Chrome execution
  • Integrating PR comment bots for automated developer feedback

Because Lighthouse gates on a single category score rather than discrete rule failures, choosing its place in the pipeline matters — axe-core vs Lighthouse CI for PR Gating weighs score budgets against rule-level blocking.

Lighthouse CI assertion-state gating Three runs are collected, the median score per audit is computed, and each assertion's state of error, warn, or off determines whether the build fails, logs, or skips. 3 runs collect median score assert per audit vs minScore error block build warn log, pass off skip audit
Lighthouse CI never fails on a single node — it fails when a median category or audit score drops below an assertion set to error, while warn and off stay non-blocking.

Project Initialization & Environment Setup

Scaffolding begins with deterministic package installation. Execute these commands in your project root:

npm install --save-dev @lhci/cli
npx lhci wizard

The wizard generates a foundational lighthouserc.json file. Immediately configure ci.collect.numberOfRuns to 3. Network variance causes significant score volatility. Running multiple iterations allows the median calculation to filter transient spikes.

Cross-reference rule coverage with axe-core Configuration & Setup. This ensures your baseline captures both automated DOM violations and performance-adjacent barriers.

Baseline Configuration & Assertion Mapping

The assertions object dictates your CI gating logic. Lighthouse CI evaluates each audit against three states. error blocks the pipeline. warn logs without failing. off disables execution.

Target a categories.accessibility score of 0.90 as a starting threshold. This accommodates minor variations while preventing regressions. Use skipAudits to omit resource-heavy audits irrelevant to your compliance target.

{
  "ci": {
    "collect": {
      "numberOfRuns": 3,
      "url": ["https://staging.app.internal"],
      "settings": {
        "formFactor": "desktop",
        "throttling": {
          "rttMs": 40,
          "throughputKbps": 10240,
          "cpuSlowdownMultiplier": 1
        }
      }
    },
    "assert": {
      "preset": "lighthouse:recommended",
      "assertions": {
        "categories:accessibility": ["error", {"minScore": 0.90}],
        "color-contrast": ["error", {"minScore": 1}],
        "aria-allowed-role": ["warn", {"minScore": 1}]
      }
    },
    "upload": {
      "target": "temporary-public-storage"
    }
  }
}

This configuration maps the accessibility category to a hard gate. It flags critical contrast failures as blocking. The throttling object replaces the string shorthand "desktop-dense-4g", which is not a valid Lighthouse CI throttling value — use the explicit object form or the preset "devtools". Align your specific metric targets with Setting Up Lighthouse CI Thresholds for WCAG 2.2 AA for regulatory compliance.

When thresholds fail, CI logs output exact audit IDs and delta scores. Review the assertionResults array in the runner output to pinpoint failing DOM nodes. The details.items array provides the specific element selectors and violation context required for rapid remediation.

CI/CD Pipeline Gating & Workflow Integration

Integrating Lighthouse CI requires a dedicated workflow step. Use ci.upload.serverBaseUrl to persist historical score tracking. This enables trend analysis across deployments.

For SPAs, pass multiple endpoints via ci.collect.url array. Lighthouse CI iterates through each route, applying the same assertion matrix consistently.

jobs:
  lhci:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v4
      - run: npm ci
      - run: npm run build
      - run: npx lhci autorun
        env:
          LHCI_GITHUB_APP_TOKEN: ${{ secrets.LHCI_GITHUB_APP_TOKEN }}
          LHCI_SERVER_BASE_URL: ${{ secrets.LHCI_SERVER_URL }}

This workflow executes lhci autorun, which reads lighthouserc.json from the repository root. It attaches assertion results directly to the pull request diff via the GitHub App token. For authenticated routes, orchestrate execution alongside Playwright Accessibility Plugin Integration to capture protected DOM snapshots before handing off to Lighthouse.

Troubleshooting Flaky Scores & Cache Management

CI environments introduce execution artifacts that skew baselines. Increase ci.collect.settings.maxWaitForLoad (default 45000ms) for heavy hydration.

Docker runners require explicit Chrome sandbox overrides:

npx lhci collect --settings.chromeFlags="--no-sandbox --disable-dev-shm-usage"

Validate environment compatibility using npx lhci healthcheck. This verifies Node.js and Chrome binary alignment. Clear ~/.config/lighthouse and node_modules/.cache between runs to prevent stale snapshots from producing false-positive violations.

Override static configuration for branch-specific testing:

npx lhci assert \
  --preset=lighthouse:recommended \
  --assertions.categories:accessibility=error \
  --assertions.categories:accessibility.minScore=0.95

This CLI override enforces stricter thresholds on feature branches while maintaining standard gates on main.

Common Implementation Pitfalls

  • Single-Run Volatility: Setting numberOfRuns: 1 guarantees false-positive CI failures due to network jitter and shared runner CPU contention.
  • Form Factor Mismatch: Running mobile audits in a desktop CI runner skews viewport-dependent accessibility audits like tap target sizing.
  • Over-Constrained Gates: Marking every audit as error blocks deployments for minor, non-blocking violations. Reserve error for critical WCAG failures.
  • Stale Cache Accumulation: Failing to purge Lighthouse cache in ephemeral CI containers leads to outdated DOM snapshots and inaccurate baseline comparisons.

Frequently Asked Questions

How do I handle dynamic content that Lighthouse CI misses during collection? Use ci.collect.startServerCommand to spin up a local dev server. Combine with Playwright to trigger JavaScript hydration before Lighthouse captures the DOM snapshot.

What is the recommended numberOfRuns for stable accessibility baselines? Set numberOfRuns to 3 or 5. Lighthouse CI calculates the median score across runs, effectively filtering out transient network and CPU spikes that cause flaky thresholds.

Can I bypass CI gates for minor accessibility regressions? Yes. Configure specific audits as warn instead of error in lighthouserc.json. Use branch-specific overrides with npx lhci assert --assertions.<audit>=warn to allow non-blocking merges while tracking regressions.

In This Section

Back to Web Accessibility Testing Fundamentals Tool Selection