Google I/O and GenAI’s Impact on Accessibility

I have been watching Google I/O online since 2015. It’s their flagship tech conference for anyone in the tech industry, including accessibility professionals. This year, I realized a childhood dream by attending I/O 2024 in person. The opportunity to attend as an ambassador for Deque—a company committed to digital accessibility—made it all the more special.

Google has been a trailblazer in accessibility since the launch of TalkBack in 2009. With this year’s conference coinciding with Global Accessibility Awareness Day (GAAD), my expectations were high.

I was not disappointed. From improvements in TalkBack with Gemini Nano to Project Gameface and updates to Google Maps and Lookout, Google’s dedication to accessibility was evident throughout the main event.

There was one topic in particular that I was especially interested in: code generation.

Accessible code generation with Gemini

As an AI evangelist and early adopter of large language models (LLMs) such as ChatGPT, GitHub Copilot, and Gemini, it is clear to me that we have a unique opportunity to make accessibility accessible to developers. Code generation with powerful models like Gemini Pro 1.5 can significantly transform our ability to develop accessible applications from the get-go.

At I/O, Google announced the integration of Gemini into Android Studio, which will be available later this year. By leveraging Gemini’s multimodality, one can now simply upload an image of an app screen mockup and ask Gemini to automatically generate the code for that application.

It immediately got me thinking: Can Gemini generate accessible code for this screen?

At the Android demo booth, I tested this out. I prompted Gemini to ensure it generates accessible code, and I was quite surprised by how good the results were. I also had the privilege of discussing this topic with several Googlers who are passionate about accessibility. I learned about their experiences with Android accessibility and how simple errors, like missing content descriptions (contentDescription), can often be traced back to sample code.

My favorite announcements from Google I/O

The Gemini integration was just one of several significant announcements. Here are some of my other favorites:

Gemini Nano integration with TalkBack

Google announced an exciting integration of Gemini Nano with TalkBack for Android, a key development for blind or low-vision TalkBack users who may encounter as many as 90 unlabeled images daily. With Gemini Nano’s multimodal capabilities and ability to run completely locally on your device, TalkBack can now provide more detailed and clearer descriptions of images. The enhanced TalkBack update will begin rolling out later this year.

Project Gameface for Android

This one was particularly exciting for me as an avid gamer. Project Gameface, an open-source project, allows gamers to control their mouse cursor using head movements and facial gestures. Google has partnered with social enterprises like Incluzza to explore broader applications of Project Gameface and build integrations with Android to enable new ways to interact with applications in education and work settings.

New features in Lookout

Google introduced Find Mode in Lookout, currently in beta, which allows users to use their phone’s camera to get more information about their surroundings. Find Mode helps locate specific objects and provides AI-generated descriptions, aiding users in navigating their environment more effectively.

Accessibility enhancements in Google Maps

Google Maps now offers more detailed walking instructions for blind and low-vision users, with audio prompts for navigation. Additionally, Maps includes accessibility information for over 50 million places, marked with an wheelchair icon detailing parking, seating, and accessible restrooms. New support for Auracast in Maps helps venues broadcast assistive audio to visitors with Auracast-enabled devices.

Several other announcements were made, such as a Text-free mode for Look to Speak, Project Astra, Circle to Search, and new ways to interact with NotebookLM that can enable users across the board to enjoy the AI innovations on the horizon.

The future of GenAI-enabled accessibility is bright

Companies like Google are not only leading the charge but also enabling the future of accessible technology. With advancements in generative AI, there is a tremendous opportunity for accessibility to be integrated by default. Google’s innovative models like Gemini 1.5 Pro can ensure that accessibility is a built-in feature of all new code.

Like we love to say at Deque, “Friends don’t let friends ship inaccessible code!” Together, we can ensure that the future of digital innovation is inclusive and accessible to all.