Facebook AI Gets Better at Describing Photos for Visually Impaired Users

(Photo by Rafael Henrique/SOPA Images/LightRocket via Getty Images)

In an effort to better accommodate users who are blind or visually impaired, Facebook this week updated its automatic alternative text (AAT) technology.

The feature, introduced in 2016 (and granted the Helen Keller Achievement Award from the American Foundation for the Blind in 2018), relies on object recognition to generate descriptions of photos on demand.

Blind and visually impaired (BVI) users have long relied on individuals to tag images with alternative text, or screen readers to mechanically describe pictures on their News Feed. The next generation of Facebook’s AAT, however, makes scrolling through social media much more enjoyable.

“The latest iteration … represents multiple technological advances that improve the photo experience for our users,” according to a Facebook AI blog post. The team expanded tenfold the number of concepts AAT can reliably detect and identify, promising more photos with more detailed descriptions, including activities, landmarks, types of animals, and more.

If someone navigating their feed, for instance, stops at a photo of friends posing in front of a famous Italian tourist attraction, the audio caption might say something like “May be a selfie of two people, outdoors, the Leaning Tower of Pisa.”

Facebook uses AI to improve photo descriptions for people who are blind or visually impairedtook steps to become more accessible, embracing object recognition technology that automatically identifies items in a photo and creates an audible description. Users are also encouraged to write up to 100 characters of alt text detailing what’s in their images.

January 20, 2021
To Top