DCMP Develops AI Scene Description Tool for Expanded Accessibility of Educational Videos

News provided by

Described and Captioned Media Program

Apr 04, 2024, 09:08 ET

SPARTANBURG, S.C., April 4, 2024 /PRNewswire/ -- The Described and Captioned Media Program (DCMP) operates the nation's largest fully-accessible educational video platform, with a library of 18,000 videos designed for use by early learner through Grade 12 students with disabilities. Our development team has been working on ways to use artificial intelligence (AI) to help with accessibility challenges, including those that might be considered "impossible." We're excited to present "AI Scene Description," a new tool for students who are blind and have low vision that can describe the visual elements of any frame within a paused video.

Through funding from the U.S. Department of Education, DCMP creates audio description (a secondary audio track that describes important visual information) for educational videos. Audio description can sometimes be limited by the amount of "quiet time" in a video, where the additional narration is inserted. There may be times when teachers and their students need a little more information about the contents of a scene. With DCMP's AI Scene Description tool, any selected scene can now be described.

DCMP's method of generating scene descriptions relies strongly on human-created metadata for each video. This additional context guides the AI in creating more accurate scene descriptions that are grade appropriate. The AI Scene Description tool is integrated into DCMP's video player and available to registered families and educators.

When users pause a video and select the AI Scene Description button, a description of the scene is spoken via text-to-speech and transcribed. The feature also allows for users to ask more detailed questions about the scene. We've added guardrails to keep the AI responses contextual to the video. Descriptions and answers use vocabulary based on the video's grade level, and take into account things like the applied educational standards, captions, and audio description.

DCMP's AI Scene Description tool is a supplement, not a replacement, for audio description. DCMP will continue to produce audio description with talented writers and voicers.

Contact:

Ted Dawson
[email protected]

SOURCE Described and Captioned Media Program