eWEEK content and product recommendations are editorially independent. We may make money when you click on links to our partners. Learn More.
Facebook users who are blind or visually impaired will now be able to “see” photos posted by their friends on Facebook by hearing audio descriptions of the images read to them through screen reader applications on iOS devices.
The process, which Facebook calls automatic alternative text, uses object recognition technology to analyze the photos and then describe what is seen in them (pictured), which is read aloud to the user by the technology, according to an April 4 announcement by Facebook. Each day, more than 2 billion photos are posted by users on Facebook, Instagram, Messenger and WhatsApp, which are essentially inaccessible to users with sight impairments or blindness.
“While visual content provides a fun and expressive way for people to communicate online, consuming and creating it poses challenges for people who are blind or severely visually impaired,” according to Facebook. “With more than 39 million people who are blind, and over 246 million who have a severe visual impairment, many people may feel excluded from the conversation around photos on Facebook.”
That’s where Facebook’s automatic alternative text innovation comes in, according to the social media company.
Automatic alternative text “generates a description of a photo using advancements in object recognition technology” which are then read aloud through screen reads apps on iOS devices, according to Facebook. Blind and visually impaired users “will hear a list of items a photo may contain as they swipe past photos on Facebook,” which provides enough detail so that users can “see” what is in the images that they can’t literally see with their eyes.
“Before today, people using screen readers would only hear the name of the person who shared the photo, followed by the term ‘photo’ when they came upon an image in News Feed,” according to Facebook. “Now we can offer a richer description of what’s in a photo thanks to automatic alt text. For instance, someone could now hear, ‘Image may contain three people, smiling, outdoors.'”
The new development comes through Facebook’s object recognition technology, the company stated. “Each advancement in object recognition technology means that the Facebook Accessibility team will be able to make technology even more accessible for more people. When people are connected, they can achieve extraordinary things as individuals and as a community—and when everyone is connected, we all benefit.”
Facebook is launching the new automatic alt text on iOS screen readers in English for users in the United States, United Kingdom, Canada, Australia and New Zealand to start, but other languages and platforms are planned for inclusion in the future, the company announced. “While this technology is still nascent, tapping its current capabilities to describe photos is an important step toward providing our visually impaired community the same benefits and enjoyment that everyone else gets from photos.”
The use of this technology by Facebook to aid blind and visually impaired users is significant, said Andrew Johnson, a Gartner analyst who covers artificial intelligence. “A tremendous amount of the content on Facebook are images, and all this time they have been inaccessible to people with low vision and blindness,” Johnson told eWEEK. “What Facebook is doing is it’s now using an automated process, which is wonderful, to help them see what is being posted. It’s a hard thing to do. I hope that bleeds over to the rest of the Web.”
Such developments could be a sign of a new trend toward improved accessibility, he said.
An existing iOS app, Aipoly Vision, is an object-recognition app that uses the camera on an iOS smartphone to capture and identify images of objects and read them aloud to users, said Johnson.
Such applications could also be used by people who don’t have visual impairments, but who are driving or busy doing other things when they can’t turn their eyes away to look at an image on a screen at that moment, said Johnson. “It is the concept of giving everybody this multi-modal choice, even if you are situationally not able to look,” he said.
In March, Toyota announced it is working to develop a wearable device that could one day help blind and visually impaired people “see” their surroundings through the help of sensors, cameras and audible devices that could improve their mobility. The BLAID device, under development in Toyota’s partner robotics division, will be equipped with cameras that detect the user’s surroundings and communicate information to him or her through speakers and vibration motors, according to an earlier eWEEK story. Wearers of the device will be able to interact with it through voice recognition and buttons, and eventually Toyota engineers hope to integrate mapping, object identification and facial-recognition technologies into the BLAID. The word BLAID incorporates the words blindness and aid to give the project its early name.
The aim of the high-tech devices is to give blind and visually impaired users greater freedom, independence and confidence as they make their way around their communities and inside their workplaces and homes.