Regardless of the industry you work in, COVID-19 has changed the way we work, live, and learn. Like many of you, I was caught off guard by the changes we were all experiencing last year.
Still today, many of us are experiencing blurred lines between our home and office experiences in ways that probably won’t be the same again. Remember what it was like to hang out and chat around the water cooler? Ah, yes, “those were the days…”
In higher education, as classrooms around the world began to go virtual at the start of the pandemic in 2020, many students faced additional accessibility challenges. One example of this is recorded lecture videos. Many times lecture videos and other audio-based content did not provide accurate captions or did not provide them at all. This left many students at a severe disadvantage.
Michigan State University (MSU) saw two big problems emerging for educators and students at the university: like most of us who work in higher education, MSU had to make many courses available online (and accessible!) as it shifted to remote learning. And, MSU students found that their campus employment evaporated overnight as everything moved virtually. Jobs like working in the libraries and dining halls were no longer an option. These were big challenges for us, in part because learning and employment are important aspects of student success at MSU.
Providing Access and Employment to Students
The more that we discussed this problem with educators and our colleagues in MSU colleges, the Library, Resource Center for Persons with Disabilities at MSU, and others, we realized that our unit over at the Digital Experience Team (DigitalX) could help support students by creating jobs to provide accurate closed captions, while at the same time, support our faculty by providing them free and accurate closed captions as they created hours of new lecture recordings throughout the week.
MSU’s Executive Vice President for Administration & Chief Information Officer, Dr. Melissa Woo, pitched an important question in a Microsoft Teams chat in March 2020: “What could it look like for us to contract student employees to provide closed captioning on content used for teaching and learning, especially for those who cannot report to their in-person jobs anymore?”
As a response, in April 2020 we started a closed captioning crowdsourcing program at Michigan State University to meet these needs. Simply put, MSU CrowdSourced Closed Captioning is a closed captioning startup within the university with a mission to support educators and students through the pandemic.
“I loved being able to help make lectures more accessible, especially in such a challenging year. It’s also been great to collaborate with other students!” – MSU Student Captioner
How Crowd Sourced Closed Captioning Works
Captions are text-based content that is presented simultaneously with audio-based content. Captions ensure that the same information that is presented audibly can be accessed by those who may not be able to consume the information in an audio-only format, particularly those who are deaf or hard of hearing. However, providing captions not only benefits people who are deaf or hard of hearing, but also people whose first language isn’t the language being spoken aloud, who consume information better visually, or to those who simply want the convenience of consuming the information without sound.
To understand how our team produces captions, think about the popular ride-sharing models like Lyft or Uber: The premise is that there is a pool of “jobs” in the queue–demand from folks who want rides– and any qualified driver can take a job when they want to make some money. Many popular captioning services use a similar “gig” approach.
Our Crowd Sourced Closed Captioning program works in the same way: educators who want to participate can submit a request or simply order machine captions on their course media. The captions are generated by AI in Kaltura MediaSpace, and usually require human corrections—that’s where the student captioners come in.
Student Captioning Freelancers choose from a queue of “jobs”: these are pre-recorded lectures, used for course work. The MSU IT department contracts students, many of whom had lost their jobs when in-person operations on campus were suspended, to help with the university’s closed captioning efforts.
Once students correct the AI-generated captions, they submit them for peer review. Once reviewed and approved by their team lead, the corrected captions automatically appear on the videos that students watch for classes. What makes these “closed” captions is how they include a coded text file and can be then toggled on and off by the viewer.
“I think my favorite thing about this work is that I get to learn super interesting things about many disciplines that I wouldn’t have the opportunity to! All the while, I feel like I’m helping enhance the accessibility of the lectures for all students.” – MSU Student Captioner
Hiring and Onboarding Student Captioners with Deque University
To launch the program, we posted a “CrowdSourcing Freelancer” student position on our student employment website and within three weeks had over 230 student applicants! Staff members reviewed applications and resumes and interviewed qualified candidates over the phone. Given the type of work and the fact that this was a remote-work position, the two most important characteristics that we were seeking among our applicants were “clear communication” and “attention to detail.”
Next, we would meet to discuss hiring recommendations weekly, which usually included somewhere between 15-30 students, then these were sent in batches to our HR department as recommended hires by Friday each week. We have a great HR department to work with, which helped to streamline the hiring process, and we can’t say enough about how important that was for us.
After being hired, students completed an online onboarding and closed captioning training through Deque University. Our goal was to help the captioners understand the requirements of the position related to workload, communication, and professional expectations, as well as an understanding of closed captioning best practices and how closed captions are essential to providing an accessible experience. Deque University provided in-depth lessons on captioning and accessibility through the Multimedia, Animations, and Motion course. In the course, students learn the differences between “open” and “closed” captions, as well as what to include in the captions as they are created. Another benefit was that lessons were available online so the students could work through the material at their own pace.
Student employees were assigned to “tiger teams”– teams of around 15 students, led by a full-time staff member (later, these were student-led positions) where they made corrections to automated captions to improve the quality and accuracy.
The Digital Experience Team (DigitalX) led the strategy development with operational support from the MSU Scoring Office and other staff from Academic Technology in MSU IT. I can’t say enough about the staff and students who contributed to this effort, many of whom continue working on this project today.
In Summer 2020, we hired 100 students and had incredible support from Scoring Office staff, and Academic Technology team staff to provide help with student hiring, management, and oversight.
The Captioning Program by the numbers, after one year
- 56 students working on the Crowd Sourcing Team (including volunteers)
- 16,077 closed-caption media files corrected to date
- 144,531 minutes of MSU video corrected (as of April 1, 2021)
- Effective rate per minute (including staff overhead): $2.59/minute.
- Single-day high of 253 videos corrected by MSU students for students
- 3,355 hours invested by full-time staff
“I think the crowdsourced captioning project is so important to MSU because it shows our commitment to making education accessible for all learners. I definitely feel good doing this work knowing the positive impact it has on our community!” – MSU Student Captioner
Innovation that leads to access, employment and better learning experiences
You don’t have to work in the digital accessibility space long to realize that many of the challenges we face are both relational and technical, and this crowdsourcing effort was no exception. Before this, I hadn’t had the opportunity to collaborate at scale with my colleagues in a virtual-only environment, but I am so proud to participate and be a part of this effort. The challenges of the past year have brought an exciting new program to life, which, in turn, helps foster an accessible learning environment while supporting students.
I would like to thank our student employees, IT and HR staff, and educators for bringing their best ideas and efforts to this project. I have been humbled to partner with such great humans and to discover what we can accomplish together.