Collaborative planning, the forgotten step of accessible development

You want to create accessible user experiences. Everyone in the development team agrees. But does everyone define success in the same way? If your team isn’t speaking the same language, you’re more likely to fail than to achieve the intended results. Collaborative planning is the key, but it’s too often overlooked. In this post I’ll introduce a way to approach collaborative planning called Accessibility Behavior Informed Development (a11yBID), which is an extension of Behavior Driven Development (BDD) originally created by Dan North. a11yBID can be used by agile development teams to discover and document quality accessibility requirements in the form of acceptance criteria before development starts. Before we examine what a11yBID is and how it works, we should first examine how we got here.

“By failing to prepare, you are preparing to fail.” – Benjamin Franklin

As an accessibility coach, I work with clients that want to teach their teams to “fish”, meaning that they want their teams to be self-sustainable with little involvement from me (or other subject matter experts) going forward. With one client, we did small experiments with alternative approaches and quickly discovered that each had their own gaps, and still had gaps even when combined. None of the approaches were optimal because accessibility was addressed too late in the development process, or placed too much burden on a single individual:

• Using accessibility checklists during development and testing often fail because they contain checkpoints that aren’t relevant to the scope of a specific story, and those that are relevant are often too abstract. This makes it difficult for the team to accurately plan, point, and estimate development work. Perhaps more importantly, when using checklists this way, teams often discover issues with designs or content. This causes the entire process to take several steps backwards while those issues are reviewed and fixed, resulting in significant delays.
• Similarly, addressing accessibility in high-level requirements documentation such as the Definition of Done is often too vague to be useful and comes with the same pitfalls as checklists.
• Designs that are annotated with development guidance require that the designer is fluent in accessible design and knowledgeable enough about code to provide comprehensive developer guidance. Additionally, they typically don’t include expert input from developers or testers prior to development.
• Accessibility automation is essential both for efficiency and to help mitigate regressions. For instance, our browser extension can catch 57% of web accessibility issues. However, not all accessibility requirements can be tested through automation. And, the earliest that it can be used is during development. Ideally, requirements should be discovered and documented before development, not during.
• Accessibility education and training for the team are essential, but by themselves can’t guarantee accessible software.

This is not to say that there isn’t a time and place for each of these, and teams would find many of them helpful. We just need to be cognizant of their potential shortcomings. Even taken all together, they miss one very important step in the process: collaborative planning. For teams to be more successful and efficient, they should take the abstract, high-level requirement of accessibility and work together to make it real. They should explicitly document how to incorporate it into the software they will be developing before development starts. In other words, teams need to collaborate to make accessibility intentional, right from design.

After being embedded in a few teams, I discovered that many of their development processes included aspects of BDD, and that those processes were well-positioned to fill the gaps that we had discovered. We experimented with how to leverage those aspects of BDD and ended up with what is described here. I don’t want to take credit for the idea – it was a collaborative effort between several of our accessibility coaches and our clients. I helped facilitate that collaboration, document our findings, and put a name to it.

What is BDD

To understand a11yBID, you must first understand what BDD is and why it’s beneficial.

From Wikipedia:

Behavior-driven development (BDD) is an agile software development process that encourages collaboration among developers, quality assurance testers, and customer representatives in a software project. It encourages teams to use conversation and concrete examples to formalize a shared understanding of how the application should behave.

A common problem in software development is that projects are delivered late, over budget, or fail to deliver all of the requested features and requirements. These issues usually stem from a lack of planning, no shared understanding of requirements, and undocumented assumptions. To mitigate this, BDD uses conversations between stakeholders and cross-functional team members to discover and formulate concrete examples of system behavior, also known as scenarios. These examples contain acceptance criteria (AC), and are typically described in a plain language format that is also machine readable, known as Gherkin.

• Scenario: A summary of the desired behavior.
• Given: Describes any preconditions and context; prepares the test environment.
• When: Describes the action that is being tested.
• Then: Describes the expected outcomes.

Writing requirements in a structured, plain language format like this enables the entire team, including non-technical stakeholders, to digest and refine the requirements. It eliminates ambiguities that would lead to assumptions about expected behaviors and outcomes, resulting in better shared understanding. The scenarios and associated AC can also serve as both living documentation for the software and as a testable and executable specification for the software. That is, the Gherkin style scenarios can usually be tested automatically as part of the project’s test suite via a testing framework like Cucumber or JBehave.

For a deep dive on BDD, read BDD in Action by John Smart.

Accessibility Behavior Informed Development (a11yBID)

Software accessibility suffers from very similar problems, but requires a slightly different approach. Accessibility is often not fully understood, not explicitly planned for before development starts, and testing occurs just before or after release—if it’s tested at all. This leads to projects being delivered late, over budget, and without the required accessibility capabilities. The end result is inaccessible software and unexpected remediation costs.

A11yBID extends aspects of BDD to help teams explicitly plan for accessibility. With a11yBID, teams use concrete examples such as designs, and conversations between cross-functional team members to discover and document how the system’s user interface should behave from an accessibility perspective. Gherkin style scenarios and AC are used to document these behaviors, and are called Accessibility acceptance criteria (a11yAC). These a11yAC are then included in the user stories, along with the rest of the AC that were discovered through the BDD (or similar) process. This helps the entire team get on the same page about expectations before development starts and enables them to more accurately plan, point, and estimate work. The team then uses the a11yAC:

• Developers read them to know what to code and how to test.
• Testers read them to know what to test and how.
• Product owners read them to understand the impact on end users and look for evidence that they have been met before accepting a story.
• New team members use them to understand what accessibility decisions were made, which helps them better understand the code base and mitigate regressions.

Here is the 50,000 foot view of the a11yBID process. The rest of this post will examine the process in more detail.

1. Input:
   1. Create concrete examples of the user interface (such as high fidelity designs).
2. Process: a11yBID
   1. Use conversations and established guidelines and tools to review the examples, then discover and formulate a11yAC based on the examples.
3. Output
   1. Revised designs that are more accessible.
   2. A11yAC.

In most cases, the concrete examples come in the form of high-fidelity designs. These designs do a good job of documenting accessibility requirements related to use of color, visual content, and interactions. During the a11yBID process, the design will be reviewed for accessibility by the team as they think through accessibility concerns, and modifications occur to make the design more accessible. In other words, most design-specific accessibility requirements are documented in the designs themselves, rather than a11yAC. After the designs are coded, testers verify that implementation matches the visual design, thus testing accessibility requirements documented by the visual design.

Where visual designs fall short is in describing the assistive technology experience such as the screen reader experience or keyboard experience. a11yAC fill this gap and are used to document the assistive technology experience of the design. a11yAC can also be used to document visual design requirements that may not be clearly represented in the design itself (like page titles), or other aspects of accessibility.

For the most part, a11yAC that describes a screen reader experience should be agnostic. They should not be written to include specific screen reader commands or output. This allows the a11yAC to be tested in many screen readers while keeping them simple and easy to understand. For example, say “when I navigate to” instead of “when I press <key>.” Similarly, state the expected semantics such as role, state, or property instead of how a specific screen reader will convey those items. For example, say “then I hear the image role” instead of “then I hear the word ‘graphic’.” Developers that are new to accessibility may see these semantics in the a11yAC and reach directly for ARIA when it is not needed, breaking the first rule of ARIA (if you can use an existing native HTML element or attribute, then do so).  If desired, information on screen reader-specific commands and output can be provided in notes. Coding suggestions, further reading, and links to example implementations can also be included in the notes to enhance learning and mitigate ARIA misuse.

Digital accessibility can be a deep, wide, and subjective topic. To help teams stay on task and deliver value, the conversations facilitated by a11yBID should center on WCAG conformance, feedback from end-users with disabilities, and industry-recognized best practices. To facilitate similar conversations in BDD, teams often use the concept of a “3 Amigos” meeting (sometimes also referred to as a pre-refinement meeting), where a product owner, developer, and tester, work together to discover and formulate AC. A similar meeting that I’ll call the a11y Amigos can be used to facilitate a11yBID conversations. This meeting should include the analyst (story writer), a developer, a tester, and the designer. It’s best to keep this meeting to only the essential people so others are not kept from working on other critical tasks. The key here is that no single person on the team is expected to know enough about all aspects of accessibility to write quality requirements. Instead, that knowledge and responsibility is shared across different team members.

A11yBID drives development in a different way than BDD. It doesn’t discover classic business requirements as BDD does. Instead, it clarifies the accessibility-specific aspects of those business requirements and informs their development. That’s why it’s called Accessibility Behavior Informed Development, instead of Accessibility Behavior Driven Development. That’s not to say that a11yBID can’t help discover business requirements, including ones focused on accessibility, but that’s not its primary intention.

Example 1 – error message

Let’s say, for example, that an Analyst is tasked with creating AC for a form validation scenario where the data is valid but the server fails to respond. The team might do something like this:

1. Request examples of the interactions and error messages in the form of high-fidelity designs.
2. Hold an a11y Amigos meeting where:
   1. The group reviews the designs, asks questions, and removes assumptions. For example, they might ask the designer if and when a loading spinner is shown as the user waits for a response from the server. The tester might bring up concerns about focus management… where should keyboard focus go after activating the submit button? The developer might suggest that keyboard focus go first to the loading spinner, then to the error message so that a screen reader user hears status updates and to ensure that keyboard focus is never lost.
   2. The group might use resources like the checklists and testing methodology in Deque University to make sure they didn’t miss anything.
   3. The analyst drafts a11yAC based on these conversations.
3. Hold a larger refinement meeting where the entire team reviews the stories (including a11yAC) to make sure everyone is on the same page and that no requirements were missed. The team estimates the level of effort needed to finish the story, and includes the a11yAC in those estimates. If there is disagreement, that can be a hint that there are undocumented requirements.

The a11yAC for this example might look something like this:

A11y AC 1 – loading spinner – informative image is conveyed – screen reader user experience

Scenario: A blind screen reader user can understand that content is loading
Given I am a blind screen reader user
And I have submitted the form
And a loading spinner is visible as I await a response from the server
When I navigate to the loading spinner image
Then I hear the image role
And I hear the alt text as “loading”

A11y AC 2 – button – move focus to loading spinner – keyboard user experience

Scenario: Keyboard focus moves to the loading spinner after activating the submit button
Given I am a sighted keyboard-only user with a motor disability
When I submit the form
And the form is replaced with a loading spinner
Then my keyboard focus moves to the loading spinner
And I can visually tell where focus is on the page

A11y AC 3 – move focus to error message – keyboard user experience

Scenario: Keyboard focus moves to the error message when it is displayed
Given I am a sighted keyboard-only user with a motor disability
And I have submitted the form
And a loading spinner is displayed
When the error message is displayed
And the loading spinner is removed
Then my keyboard focus moves to the container for the error message
And I can visually tell where focus is on the page

Realistically, there should be more a11yAC for the error message. For example, if the error message includes an image, a heading, and maybe a button to retry the submit action, each of those needs are relevant a11yAC. To keep this example brief, those AC were excluded.

In this example, the team decided to add a11yAC for both the screen reader user experience and the sighted keyboard user experience. They knew that they needed to manage keyboard focus after replacing the form with a loading spinner. Otherwise, the keyboard focus would become lost. There is some overlap in the experiences, as a screen reader will announce changes of keyboard focus, so the team decided to only include keyboard AC where there was overlap because keyboard requirements are also generally easier to test. This helped to reduce the total number of AC while still covering all requirements and making the goal clear.

Making a11yBID more scalable and efficient

“Give me six hours to chop down a tree and I will spend the first four sharpening the axe.” – Abraham Lincoln

As you may have already gathered, some aspects of a11yBID can be time-consuming, especially the process of crafting custom a11yAC. This problem is even more acute if research is required to understand how WCAG requirements apply to a specific design or how assistive technologies like screen readers should work in specific circumstances. Fortunately, there are several strategies which can make the a11yBID process more scalable and efficient.

Design system and component library

First and foremost, having an accessible design system and component library are critical. Accessible design systems enable designers to reuse components that have already been designed with accessibility in mind, e.g.: checks for text and non-text color contrast, clarity of error messages and instructions, or not conveying information by color alone have already been done. Having an accessible design system for your organization means that designers will spend less time on the accessibility implications of shared design patterns. An accessible design system does not eliminate the need to consider accessibility, but it minimizes time and effort and helps establish consistency for end-users.

Similarly, an accessible component library that’s tied to the design system makes it much easier for developers to quickly and accurately implement the designs in an accessible way. Having a library of accessible components lessens the opportunity for developers to accidentally introduce accessibility defects, and helps to provide consistent and predictable behaviors and interactions for end users. An accessible component library like Deque’s Cauldron also allows for defects to be fixed in one place and be disseminated to multiple implementations, saving individual teams the time and effort required to fix those defects one at a time.

The combination of an accessible design system and component library still doesn’t guarantee a bullet-proof accessible experience, but it does establish a strong foundation. Designers can still introduce accessibility defects by overriding the design system or using patterns not found in the design system. Similarly, developers can introduce accessibility defects by overriding shared components, not wiring them up correctly, or building their own custom components.

Build an a11yAC library

As teams work through the a11yBID process, they will quickly discover that they are frequently writing the same, or similar, a11yAC. This is where an a11yAC library can be extremely useful to:

• Templatize a11yAC for common components and patterns found at your organization.
• Guide Analysts (or whoever is writing stories) through selecting relevant a11yAC for common components and patterns.

This saves tremendous time and effort in discovering and formulating custom a11yAC, and it drives consistency across teams. In fact, this can greatly reduce the need for A11y Amigos meetings.

As part of our coaching program, we can help you create an a11yAC library.

Use AIMS

The AIMS workflow is a proven way for analysts (or whoever is writing stories) to approach a11yBID in an efficient and scalable way, assuming that you have an a11yAC library. An easy way to remember AIMS is “our team AIMS for solid requirements by always keeping accessibility in mind.” The workflow is as follows:

1. [A] Assess the design for accessibility, ask questions, and remove assumptions.
   1. Identify and fix any issues before they’re found by developers or testers later in the process. The analyst doesn’t need to be an expert in design or accessibility, but should be able to do a high-level smoke test of the design, and can use resources such as Deque University to aid in this review. The designer should be reviewing their designs for accessibility prior to delivering them to the analyst, but the analyst is well-positioned to add a layer of resilience to the process.
   2. Remove assumptions about functionality (interactions, dynamic content, etc.) or content (page titles, alt text, etc) by asking questions and having conversations with the designer and content writer.
2. [I] Identify and inventory patterns used in the design.
   1. Use a tool like Snagit to annotate the design, inventory all the patterns, and start to identify gaps (unknowns that need to be solved later).
3. [M] Map those patterns to AC using guidance from the AC library.
   1. Use the guidance to copy relevant templatized a11yAC and then refine them to better match the context of the story in which they are used.
4. [S] Solve for any remaining gaps, such as open questions, or missing AC.
   1. A gap might be found in any step of the process. Gaps are things like “I can’t find any patterns, guidance, or AC for this part of the design.” Work with your team to come up with answers for these gaps using the broader a11yBID process.
   2. Deque University is a great resource that can inform answers to the gaps.

Automate as much as possible

Deque offers a variety of tools to help automate as much accessibility testing as possible. By using these tools, teams can catch requirements that might have fallen through the cracks during planning. Additionally, because a11yAC are usually focused on development concerns, this sort of automation can add a layer of resilience by also checking for design issues like color contrast. For example, axe-core can find 57% of web accessibility issues. Our Intelligent Guided Tests can bring that number to 80% or more. Furthermore, adding this automation to the CI/CD pipeline is critical to preventing accessibility regressions in the future. An easy place to start could be to install the free axe-linter for Visual Studio Code. This checks for accessibility issues as you code your projects.

It may also be possible to automatically test some of the a11yAC themselves by programmatically inspecting a snapshot of the browser’s accessibility tree or system’s accessibility APIs. Similarly, it may be possible to automatically test most of the a11yAC that are focused on keyboard behaviors using tools like Puppeteer or Playwright. Ideally, a11yAC could be tested by automating screen readers themselves. Automating a11yAC in this way can help prevent regressions in the future, reduce the effort of manual testing, and make the development team and stakeholders more confident when making changes. Note that some screen reader automation is currently possible, but is an emerging technology and likely not robust enough to depend on at the time of writing this article.

Leverage Deque’s coaching program

Sustainable accessibility requires changes in skill, will, and process. Deque’s Accessibility Coaching program can help your teams embark on a transformative journey to achieve these changes. Every team operates in a unique way, and our coaching program can help teams integrate a11yBID and other accessibility practices seamlessly into their specific software development life cycle. Writing quality a11yAC takes a lot of practice and experience with interpreting WCAG and understanding assistive technologies in addition to knowing how to develop for the desired outcomes. Our coaching program can help bring your team up to speed quickly.

Gotchas

A11yBID is not a silver bullet. It requires intentional, consistent practice and a commitment from the entire team to learn accessibility and collaborate often. Like BDD, it tends to work better in teams that already practice Agile, and less well in waterfall or other non-agile contexts. Additionally, a11yBID isn’t a replacement for automated accessibility testing, manual audits, and external validation. It aims to mitigate the number of accessibility defects in a product, but the level to which defects are mitigated depends, for the most part, on the knowledge and experience of the team.

Requirements can and will fall through the cracks of planning, even for mature teams. The team can use the Intelligent Guided Testing (IGT) feature in the axe DevTools Browser Extension to mitigate defects from these missed requirements. IGT goes beyond automated testing and guides developers through manual testing. It has been found to catch 80% or more of accessibility issues that are commonly found in manual audits done by experts.

There could potentially be many a11yAC in a single story. Teams need to find the right balance between verbosity, clarity, and efficiency. It’s important to remember that the number of AC in a story does not necessarily reflect the level of effort required to develop the story. Instead, having the accessibility requirements clearly documented as AC helps teams accurately understand the scope of work, estimate it, develop it, test it, and deliver it on time. There are many ways to approach this balance, and our coaching program can help.

This isn’t just theory

Ben Allen, a former Digital Product Manager at PNC, details the benefits of a11yAC and an AC Library, along with how Deque helped develop the approach, in his Deque webinar Agile Accessibility Requirements at Scale: How PNC incorporates accessibility requirements into agile practice.

Some quotes from some of our clients:

Through some trial and error, we’ve learned in this space that consistency is key. Without consistent, reliable and reusable ACs, our teams were missing a pertinent piece of the puzzle. It wasn’t enough for us to have an Accessible Component Library and Design System. Building and implementing our own AC Library in partnership with our Deque Coaches has been a breakthrough, and one we can’t live without. – Kelly (Digital Product Manager)

Once you understand what it takes to have an accessible product, you will quickly realize that you must have specific, directed AC to account for that need.  The a11y library provided by the Deque team has been a tremendous resource for our team.  Not only is the library replete with solid AC’s, it also includes a wealth of additional resources.

As we began using the a11y AC library, we not only became much more efficient at including a11y AC, but we also realized that we could customize the content to adapt to very specific situations.  In summary, not only did it help us to become better at incorporating accessibility into our product, it allowed us to do so more efficiently. – Michael (Analyst)

A11y acceptance criteria are treated with the same care and the utmost seriousness with which all AC are treated. It introduced a level of humanity to our engineering that is refreshing and something to be proud of. – Daniel (Developer)

I was initially concerned that introducing additional development effort into our already dense deliverable commitments would adversely affect my team’s effectiveness. I thought that more training for myself and my teammates and stories with a11y acceptance criteria aplenty would slow us down. I’m happy to report that I was wrong.

I have seen a much more competent team that is more mindful of the user experience for people with disabilities. The coaches at Deque have made our transition to be a more inclusive development team painless. Deque has individualized training sessions for every single member of the team from Designers, to Business Analysts, to Developers, to Testing, to Product Owners to ensure that every member of the team is aware of the role they play in creating an accessible product. Accessibility coaches have partnered with other teams to create common components which are used throughout the new experience which are accessible, responsive, and easy to implement. The development of each element has corresponding updates to the a11y acceptance criteria library to make story writing simple for Business Analysts. To make matters much easier, these coaches partner with our team to ensure that all of our accessibility questions are answered and they are always available. – Chris (Product Owner)

Having access to the AC library has made our team’s push on accessibility so much easier. Both in writing the stories and knowing how to test. – Gabe (Quality Engineer)

Summary

A11yBID is a powerful agile development process that can help teams shift accessibility left and reduce future costs of remediation. Teams can use it to efficiently discover and formulate quality accessibility requirements and document them in a human-friendly format that’s also testable before development even starts. Here is what you can do next:

• If you are already following BDD practices, think about how your team can leverage those existing practices to adopt a11yBID.
• Try applying a11yBID to a single story in your next sprint. See how it works. Then, think about how you can approach efficiencies such as an a11yAC library.
• If you don’t already have an accessible design system and component library, think about how you can create those for your team. We can help with this.
• Employ Deque’s accessibility coaching program to embark on a transformative journey and integrate a11yBID into your team’s development process.