Best practices for automated accessibility testing

It can be time-consuming for developers to validate that a service complies with the WCAG accessibility criteria. There are browser extensions and other tools that can scan for accessibility issues, but using these takes time and discipline. Developers might forget it, and this can introduce regressions, even critical ones that prevent some users from accessing the service. It is therefore important to set up automated accessibility tests that run in the CI pipeline for every pull request.

Automated accessibility tests should be written both as component tests and as end-to-end tests. Component tests ensure that each presentational component and every variant of that component has no automatically detectable accessibility violations. This is especially important in component libraries, but should also be the norm for components used only in a single application, at least if they have a non-trivial user interface. End-to-end tests catch violations on the page level, and can therefore also find issues with page structure and how different components work together. The drawback is that it is much more difficult to test every component variation, which is why the two levels complement each other.

Component tests

Storybook is a very useful tool for documenting UI components and developing them in isolation from the rest of the application. In my experience, it is also the best tool for component-level accessibility tests. If you have a good habit of writing stories for all component variants, it is not much extra effort to also run accessibility tests for these stories. Storybook has long supported accessibility testing in the Storybook UI using @storybook/addon-a11y. Previously, running the same tests automatically required configuring the test runner. Happily, since Storybook v9, which was released a couple of weeks ago, it is now as easy as setting a single parameter in .storybook/preview.ts:

const preview: Preview = {
parameters: {
a11y: {
test: 'error',
},
},
};

The tests can then be run for all stories with a single command, either using the traditional @storybook/test-runner or the newer @storybook/addon-vitest. If there are any violations, it will fail the test and print detailed information about the issue. For example:

npm run storybook
npm run test-storybook

Behind the scenes, Storybook uses the axe-core engine for detecting accessibility issues. This is the same engine that is used by most automatic and manual accessibility testing tools. It evaluates a set of rules, grouped into rulesets. By default, Storybook includes both WCAG rulesets and a best practices ruleset. Although potentially useful, in my experience, it can be difficult to get all components to comply with the best practices. My recommendation is to configure Storybook to include the rules for WCAG 2.1 A and AA, which are required by the Act on the Provision of Digital Services, as well as WCAG 2.2 AA for future-proofing:

const preview: Preview = {
parameters: {
a11y: {
test: 'error',
options: {
runOnly: ['wcag2a', 'wcag2aa', 'wcag21a', 'wcag21aa', 'wcag22aa'],
},
},
},
};

Most likely when you initially adopt Storybook, there will be at least some components with accessibility issues. In that case, my recommendation is to add a11y: { test: 'todo' } to the parameters either at the story level if the issue is with a single variant, or otherwise at the component level. This will cause tests to pass while still printing warnings. After that, you separately go through and fix all the stories in order of priority.

I recommend reading the Storybook accessibility testing docs for more information on how to set up and configure accessibility tests. There are also good docs on how to run Storybook in CI pipelines, e.g. in GitHub Actions.

End-to-end tests

Writing end-to-end tests is essential for preventing regressions in web applications. Some people recommend an inverted pyramid of many smaller unit and component tests, and fewer slower and less reliable E2E tests. In my experience, it is possible to get hundreds of tests running in minutes, although it requires sufficiently powerful test runners so that tests can be parallelized. With Playwright, which is my preferred E2E testing framework, I also find that it is not too difficult to minimize test flakiness. Because of this, I prefer having E2E tests covering most features, and not just critical paths in the application.

Playwright also has good support for automated accessibility testing, based on the same axe-core engine used by Storybook. Using @axe-core/playwright, you can create a helper function to scan the page and fail if there are any violations:

export async function expectNoAccessibilityIssues(page: Page) {
const builder = new AxeBuilder({ page }).withTags([…]);
const { violations } = await builder.analyze();
expect(violations).toEqual([])
}

The output here is not the most readable, so you might want to write custom code to print the violations and then only expect that there are none:

printViolations(violations)
expect(violations.length).toBe(0)

You can then run this function for every feature in the app. For simple features, write a test that loads the page and performs the scan. If the feature has an initial loading state, you first need to assert that it has loaded. For more complex features like a checkout flow, run the scan separately after each step or significant action in the flow.

test('does not have accessibility issues', async ({ page }) => {
await page.goto('/my-feature')
await expect(page.getByRole('heading', { level: 1 })).toBeVisible()
await expectNoAccessibilityIssues(page)
})

Another option with Playwright is to write tests to ensure that features are usable with keyboard navigation. For this, you can write a helper function that continuously presses the tab key until a locator is focused:

export async function tabUntilFocused(page: Page, locator: Locator) {
await test.step('Tab until focused: ' + locator, async () => {
let isFocused = false
while (!isFocused) {
try {
await expect(locator).toBeFocused({ timeout: 10 })
isFocused = true
} catch (e) {
await page.keyboard.press('Tab')
}
}
})
}

You can then write tests that use this helper and other page.keyboard methods to perform tasks. This is more work than running an axe scan, but in my view worth it at least for critical features like a checkout flow.

To learn more, read the Playwright accessibility testing and CI setup docs.

Other kinds of testing

It's also a good idea to lint for accessibility issues, e.g. using eslint-plugin-jsx-a11y or the newly released oxlint. In my experience, accessibility linting is pretty limited in usefulness compared to tests, but as it is easy to set up and provides quick feedback, it is still worth it.

This article has focused on automated testing, but it is important to remember that manual testing is also needed. Storybook, Playwright, and similar other tools can only detect violations that can be evaluated using rules and heuristics. For example, they can detect insufficient contrast, missing accessible names for buttons, missing labels for form controls, incorrectly used ARIA attributes, and invalid HTML structure. They cannot detect that components actually use the appropriate HTML elements and attributes to be usable by screen readers and other assistive technology.

It is therefore important for all frontend developers to learn how to manually assess accessibility. In particular, it is important to learn how to use a screen reader and regularly use it to test the application. My recommendation is NVDA, which in my experience works better than VoiceOver on Mac. As NVDA is Windows only, Mac users might need to use a virtual machine like Parallels Desktop. Ideally, I would also test the application with real users with disabilities at regular intervals.

Conclusion

Accessibility is not only ethical, it is also a legal requirement for many services, including new services like e-commerce sites starting on June 28, 2025, when the transition period for the Act on the Provision of Digital Services expires. I hope this guide has given you some ideas on how to improve accessibility engineering practices.