Scientists from IBM Research and Carnegie Mellon University (CMU) recently announced a new open platform to support the creation of smartphone apps that can enable the blind to better navigate their surroundings.
IBM partnered with CMU’s famed Robotics Institute to create a pilot app called NavCog that draws on existing sensors and cognitive technologies to inform blind people on the CMU campus about their surroundings by “whispering” into their ears through ear buds or by creating vibrations on smartphones.
The app analyzes signals from Bluetooth beacons located along walkways and from smartphone sensors to help users move without human assistance, whether inside campus buildings or outdoors. Researchers are exploring additional capabilities for future versions of the app to detect who is approaching and what is their mood. The NavCog app will soon be available at no cost on the App Store.
“While visually impaired people like myself have become independent online, we are still challenged in the real world,” said IBM Fellow Chieko Asakawa, visiting faculty member at Carnegie Mellon. “To gain further independence and help improve the quality of life, ubiquitous connectivity across indoor and outdoor environments is necessary. I’m excited that this open platform will help accelerate the advancement of cognitive assistance research by giving developers opportunities to build various accessibility applications and test non-traditional technologies such as ultrasonic and advanced inertial sensors to assist navigation.”
IBM has made the first set of cognitive assistance tools for developers available via the cloud through its Bluemix platform as a service here. The open toolkit consists of an app for navigation, a map editing tool and localization algorithms that can help the blind identify in near real time where they are, which direction they are facing and additional surrounding environmental information. The computer vision navigation application tool turns smartphone images of the surrounding environment into a 3D space model to help improve localization and navigation for the visually impaired.
The combination of these multiple technologies is known as “cognitive assistance,” an accessibility research field dedicated to helping the blind regain information by augmenting missing or weakened abilities. Researchers plan to add various localization technologies, including sensor fusion, which integrates data from multiple environmental sensors for sophisticated cognitive functioning, such as facial recognition in public places. Researchers also are exploring the use of computer vision to characterize the activities of people in the vicinity and ultrasonic technology to help identify locations more accurately.
“From localization information to understanding of objects, we have been creating technologies to make the real-world environment more accessible for everyone,” said Martial Hebert, director of the Robotics Institute at Carnegie Mellon, in a statement. “With our long history of developing technologies for humans and robots that will complement humans’ missing abilities to sense the surrounding world, this open platform will help expand the horizon for global collaboration to open up the new real-world accessibility era for the blind in the near future.”
IBM has been committed to technology innovation and accessibility for people with disabilities for more than 100 years, helping to ensure that employees, customers and citizens have equal access to information they need for work and life. Some early innovations for the blind include a Braille printer, a talking typewriter and the first commercially viable screen reader, IBM officials said.
IBM has a history of collaborating with CMU. In March, IBM announced a smarter buildings partnership with Carnegie Mellon, where the university uses a cloud-based analytics system for reducing energy and facility operating costs.
The university expects to save approximately 10 percent on utilities, nearly $2 million annually, from the IBM system deployed across 36 buildings on its Pittsburgh campus.
“On its own, the deployment of this technology will drive significant energy and operational savings with a very attractive return on investment,” said Donald Coffelt, associate vice president for Carnegie Mellon University’s Facilities Management Services, in a statement. “Just as important, improved building performance enhances the occupant experience and provides a much more effective education and research environment. IBM is a clear leader in the field of advanced building analytics and facilities systems integration. This technology offers us important gains in initiatives related to advanced infrastructure systems research, the Pittsburgh 2030 initiative and a more proactive building and infrastructure management model.”