Automated accessibility testing and its limits

Automated tools scan a page's code and report accessibility problems in seconds. They are excellent at finding missing labels, missing alt attributes and low contrast at scale. They cannot tell you whether alt text is accurate, whether focus order makes sense or whether a page is usable. Use them as the first step, never the only one.

What automation can and cannot check

A tool can only check what a computer can decide from the code and the rendered page.

Many WCAG success criteria need human judgement, such as 1.1.1 Non-text Content (is the alternative equivalent?), 2.4.3 Focus Order and 3.3.3 Error Suggestion. So a clean scan never means a page conforms. When the UK Government Digital Service tested popular tools against a page full of deliberate barriers, every tool missed a large share of them.

Lighthouse

Lighthouse is built into Chrome and Edge DevTools and runs a subset of the axe-core rules.

1. Open DevTools and select the Lighthouse panel.
2. Tick Accessibility, choose Mobile or Desktop, and click Analyze page load.
3. Read the failed audits. Each one names the rule, lists the failing elements and links to an explanation.

The score is a weighted average of the audits that apply to the page. Some failures, such as missing button names, weigh much more than others. Lighthouse only tests the page in its initial state, so menus, dialogs and error messages that appear later are not tested.

axe DevTools

The free axe DevTools browser extension from Deque runs the full axe-core rule set and is the industry standard engine. Other tools, including Lighthouse and many testing frameworks, use axe-core underneath.

1. Install the extension for Chrome, Edge or Firefox.
2. Open DevTools, select the axe DevTools panel and scan the page.
3. Open a result to see the element, the WCAG criterion, the impact and how to fix it. Use Highlight to find the element on the page.

Because you trigger the scan yourself, you can test different states: open the menu, show the dialog or submit the form with errors, then scan again.

Other useful tools:

• WAVE from WebAIM overlays icons on the page, which helps you see structure, alt text and errors in place.
• Accessibility Insights for Web from Microsoft combines automated checks with a guided manual assessment and a tab-stops visualiser.
• The browser's Accessibility pane shows each element's computed name and role.

Reading results

Automated tools report issues with an impact level. axe uses four: critical, serious, moderate and minor.

1. Start with critical and serious issues. These usually block people: unlabelled buttons, missing form labels, keyboard traps that a tool can see.
2. Group by cause. Fifty failures may come from one component template. Fix the template once.
3. Check "needs review" items. axe reports some results as needing review when it cannot decide, for example text over a background image. Check them by hand.
4. Expect few false positives. axe-core is designed to report only what it is confident about. If a result looks wrong, read the explanation before dismissing it.

Automating checks in your build

Running axe-core in automated tests stops regressions reaching production. With Playwright and the @axe-core/playwright package:

// tests/a11y.spec.js
import { test, expect } from '@playwright/test';
import AxeBuilder from '@axe-core/playwright';

test('home page has no detectable WCAG A or AA violations', async ({ page }) => {
  await page.goto('http://localhost:3000/');

  const results = await new AxeBuilder({ page })
    .withTags(['wcag2a', 'wcag2aa', 'wcag21a', 'wcag21aa', 'wcag22aa'])
    .analyze();

  expect(results.violations).toEqual([]);
});

Add tests for important states too, for example after opening the navigation or submitting an empty form. Treat a failing test like any other failing test: fix it before merging.

Plan the manual checks

Automated scans are fast, so run them first. Then cover what they miss. A practical manual checklist for each page or component:

1. Keyboard: tab through everything, operate every control, check focus is visible and never trapped or lost.
2. Zoom and reflow: zoom to 200% and 400%, or set the viewport to 320 CSS pixels wide, and check nothing is cut off or overlapping (1.4.4 Resize Text, 1.4.10 Reflow).
3. Text spacing: apply increased letter, word and line spacing and check text is not clipped (1.4.12 Text Spacing).
4. Screen reader: navigate by headings, landmarks and form fields, and listen to names, roles and states (lesson 13).
5. Content: check alt text, link text, headings, labels and error messages make sense.
6. Media and motion: check captions, transcripts, autoplay and animations.
7. Forms: submit with errors and check messages are clear, associated with fields and announced.

Summary

• Automated tools find many common, code-level problems quickly; run them first.
• They cannot judge meaning, usability, focus order or most interactive behaviour, so a clean scan is not conformance.
• Lighthouse runs a subset of axe-core on the initial page load; axe DevTools runs the full rules on any state you trigger.
• Prioritise critical and serious issues, group by cause and check items marked for review.
• Add axe-core to automated tests to prevent regressions.
• Always follow up with keyboard, zoom, screen reader and content checks.