Smartphone Apps Bring Sights Alive for Visually Impaired and Blind People

By: Lee Rickwood

July 31, 2019

Millions of people downloaded the face-aging FaceApp, apparently because they wanted to see what their face is facing, many years from now.

For many other folks, seeing the future means needing to know what they are facing in just the next few steps.

woman walks along city street with a white cane and smartphone in her hands

New wayfinding and location information apps can bring a person’s surroundings to life with audible descriptions of the immediate environment. Microsoft image.

Whether inside or out, our physical environment can present obstacles to blind or partially sighted people. Navigating their way to a specific destination can be a real challenge, and maintaining a sense of independence while doing so is equally difficult.

But smartphone apps can bring a kind of sight to visually impaired and blind people. By using GPS, audio and sound recordings and other digital tools, these wayfinding and location information apps can bring surroundings to life with audible descriptions of the immediate environment. Visually challenged people gain the ability to find people, places and things they need with more independence and self-reliance.

One of the newest such apps available in Canada is called Soundscape, from Microsoft. While some navigation apps give almost turn-by-turn directions to a specific location, Soundscape uses 3D audio – like surround sound, 3D audio seems to come from a specific location – to describe the environment so users can build their own mental map of the surroundings and navigational choices.

A person with low or no vision would use Soundscape in combination with traditional mobility aides, like a cane or an assistive dog. The app calls out roads, intersections, buildings or other landmarks as the user passes by, and it also calls out specific places or addresses that a user has marked in advance, based on mapping data culled from Open Street Map and Bing Maps.

man exits building, earbuds clearly visible in his ears

The Soundscape smartphone app uses 3D audio to describe the environment so users can build a mental map of their surroundings and navigational choices. Microsoft image.

Soundscape uses Azure to manage the data it work with; the developer says the app requires no user identification or account to use, so therefore no user-identifiable data is captured.

Users place beacons and other location buttons in the app that relate to their destination(s). The beacon emits signals that the app responds to, and the app then gives audible feedback to the user that confirms location or direction.

The call-outs sound as if they are coming from the actual location, in relation to the user. A series of audible clicks and reassuring bells are emitted by the app to inform the user they are “on the right track” to reach their intended destination. Users wear stereo headsets, ear buds or another listening accessory.

As part of Microsoft Canada’s roll-out of the product, it is working with the City of Brampton on piloting the Soundscape app, and the company worked with the Canadian Institute for the Blind (CNIB) Foundation and Vision Loss Rehabilitation Canada (VLRC) during beta testing.

“Creating a truly accessible city for everyone is a major priority for the City of Brampton and we believe technology will play a big role in helping us achieve our goals,” said Joseph Pittari, Interim Chief Administrative Officer, City of Brampton, in a release about the city’s use of Soundscape.

soundscape user interface on smartphone screen

Soundscape users can place beacons and other location buttons in the app that relate to their destination(s).

“Technology is a game changer for people with sight loss and exciting apps, like Soundscape, are helping to transform communities into beacons of accessibility and inclusion where people can live, work and play without barriers,” added Angela Bonfanti, Vice President, CNIB Foundation. “[W]e’re thrilled that it’s finally available in Canada.”

The CNIB has also been using another smartphone app for visually challenged, called BlindSquare, which also describes the immediate environment, announces points of interest and calls out street intersections, based on data from free, third-party map apps.

grey-haired man walks by building while holding smartphone out in front of him

Assistive technologies like wayfinding and location information apps let visually challenged people use their smartphone to navigate. CNIB/BlindSquare

In fact, the app is helping people find their way inside the CNIB national office. The BlindSquare app responds to signals from its companion Beacon Positioning System, a series of location-specific low-energy Bluetooth transmitters that are placed strategically to allow users to locate and get to important places like the cafeteria, a meeting room or a washroom.

Of course, BlindSquare works outdoors, too, using a device’s GPS capabilities to find information about the user’s surroundings (from Foursquare and Open Street Maps). Users can recall locations from their maps for future use with stored personal favourites and digital map markers.

BlindSquare will then speak out the relevant information using a synthetic voice, and in a meaningful way, tell them something useful about the future.

-30-

 

 

 

 

 

 


3 comments

  1. Lee Rickwood says:

    Yes, there are other apps from which to choose that are designed to help blind or visually challenged people.

    The CNIB lists a few on its website (https://cnib.ca/en/sight-loss-info/living-with-blindness/technology/apps?region=on); there is also an app developed at McGill (https://autour.mcgill.ca/en/) you might be interested in.

    Also, the big companies are all involved in developing new assistance or access apps: Microsoft (https://www.microsoft.com/en-us/ai/seeing-ai) and Google (https://www.blog.google/outreach-initiatives/accessibility/lookout-discover-your-surroundings-help-ai/) among them.

  2. Kenneth says:

    Is there no simple audio, capable G.P.S available for the blind? We are rural.

  3. Deepak Sahani says:

    Nice app and good post

Leave a Reply

Your email address will not be published. Required fields are marked *