Inquiring minds want to know when axe will support the Web Content Accessibility Guidelines version 2.1. In a way, it already does: WCAG 2.1 encompasses all of the 2.0 success criteria word-for-word, meaning the current axe-core ruleset targeting WCAG 2.0 now also applies to 2.1.
There are, however, 17 new Success Criteria in WCAG 2.1 items for mobile accessibility, people with low-vision, and people with cognitive and learning disabilities. In order to discuss these new criteria in terms of axe-core–an automated accessibility testing API and ruleset–we first need to talk about accessibility testing.
Automated accessibility testing vs. manual accessibility testing
The axe-core library is an automated accessibility testing tool written in JavaScript, so it’s limited by what can be detected programmatically in web browsers. For axe-core to effectively audit web pages without human assistance, accessibility issues must be detected with code that works cross-platform and without false positives.
Generally speaking, some items that can be automated for accessibility include:
- HTML hierarchy and relationships
- ARIA roles, states and properties: validity, relationships, and best practices
- Color contrast (aside from limitations in browser APIs)
- Accessible naming of form controls, links, buttons
Items that can’t be automated by axe-core and must be tested manually:
- Error identification
- Focus order
- Keyboard support on custom controls (no cross-platform API for event detection)
- Screen reader testing
In axe-core version 2.1.7 we added Needs Review results that blur the line of programmatic pass/fail, allowing users to review issues and determine whether they are truly violations or can be ignored. However, to report something for review, axe-core has to determine with some degree of certainty that it’s an accessibility issue. Otherwise, the tool would flood you with too many false positives and irrelevant issues, and you’d likely chuck your computer out the window.
Note: The axe-core JavaScript library powers our Chrome and Firefox Developer Tools extensions, so in terms of analysis, those are also considered to be automated. In contrast, our WorldSpace Assure product leverages axe-core in its internal architecture, however its purpose is guided manual testing.
Automation for WCAG 2.1
Which brings us back to WCAG 2.1: many of the new Success Criteria in WCAG 2.1 require manual testing by humans, either because the recommended techniques can’t be detected with code alone, or they’re too nuanced for a axe-core to determine a pass/fail. Some of the trickier items in WCAG 2.1 simply cannot be automated due to lack of web platform APIs.
We are actively working on new rules for the few criteria that can be automated with code, and you can follow our progress on GitHub.
Automation in axe-core depends on the tool’s ability to audit the accessibility of web pages programmatically with code, without modifying the page under test. The mantra of axe-core to limit false positives and provide realistic results is part of why it has become a testing standard. We take our users’ time and trust very seriously, so we are intentionally very thoughtful when adding new rules. You’ll see support added for WCAG 2.1 over time, as we conduct the appropriate research and testing to give you the same kind of useful results you’ve grown to depend on.