Audio technology helps blind people follow Australian Open Tennis by providing a live commentary of the match.

To hear more audio stories, download Audm for iPhone or Android.
Michael Marshall hears rapid, echoing pops when he listens to an Australian Open tennis match, followed closely by high- and low-pitched clinks. Three pops on the left signal that the ball landed close to the line; a low-pitched clink means that the player returned it with a backhand stroke.
These noises might sound like arcade sound effects or some new version of Morse code to people without context, but each one is a message meant to help people who are blind or have limited vision follow the game. A new technology, called Action Audio, is being tested on a large scale for the first time at this year’s Australian Open, where every match in the Rod Laver Arena is available on a livestream with this accessibility feature.
Marshall, 35, of Melbourne, is an avid tennis fan who listens to the tournament on the radio every year. He said that Action Audio had added a layer to his experience, allowing him to more clearly track the ball during a point.
Marshall said that everything is like, "oh, yes, this is the final piece that I'm missing." He later added that "it gives you those cues that you never really had before."
He and other blind sports fans often listen to radio descriptions of games instead of watching them on TV. But with serves flying at over 100 miles an hour and groundstrokes routinely exceeding 80, even the most descriptive radio announcer can’t talk fast enough to capture every nuance of the action on a tennis court. Details like how close the ball landed to a line, how fast it traveled and what direction it moved are not always described, according to Machar Reid, head of innovation at Tennis Australia. This can be frustrating for blind fans, who often have to rely on their own knowledge of the game to fill in the gaps.
He said of radio announcers that they might do it on one shot, but it's hard to do on every single shot on the rally.
Tennis Australia partnered with Monash University in Melbourne, Australia, and the digital design and communications agency AKQA to develop Action Audio in 2019. Reid said that tennis was the ideal sport to begin with because of its existing ball-tracking technology. In general, a fully equipped tennis court has 10 or 12 cameras collecting data around 50 times a second. That data is typically used to make quick calls about whether a shot landed in or out of play. Action Audio will use that data to provide real-time audio feedback to players about their shots.
Action Audio's technology converts the data into 3-D sound - a process that takes less than a second - allowing it to be broadcast alongside live radio commentary.
The sounds include one, two or three blips to indicate how close the ball is to the sideline or baseline. If the ball is close to the line, three blips will go off. If the ball lands farther inside the line, two blips sound. One blip means the ball was hit toward the middle of the court.
If the ball is played to the left side of the court, the sounds come through the left speaker. If the ball is played to the right side of the court, the sounds come through the right speaker.
As the ball moves around, it makes a sound. The sound is loudest when the player strikes the ball, and it fades away gradually as the ball travels. High-pitched clinks indicate a forehand, while low-pitched clinks indicate a backhand.
"In the future," Reid said, "I would love to integrate Action Audio into TV broadcasts and develop it for other sports that use optical tracking systems, such as baseball."
Developing sport soundscapes can be difficult because it is hard to determine how many actions can be captured and translated to sound without the audio becoming overwhelming or distracting, according to Tim Devine, the executive innovation director at AKQA.
He asked if I wanted to know where the ball was or follow a player.
AKQA and Tennis Australia reached out to fans to ask what they found most interesting about tennis and then surveyed blind people to see which sounds worked best. Developers tried to use sounds that were already familiar to users, such as the sound of a bell ball used in blind tennis, Reid said.
The developers are also collecting feedback from people who watch this year’s Action Audio livestream, which has drawn listeners from about 70 countries.
Karl Belanger, an accessibility analyst for the National Federation of the Blind in the United States, is among those listeners. He was listening to the audio stream provided by Tennis Australia last week when he heard an advertisement for Action Audio. He was excited to tune in but became frustrated because the design of the site was not fully compatible with the screen-reader technology that blind people use to decipher web pages. He was happy to hear that the site has been updated and is now fully compatible.
"That one layer of it just kind of mars the whole experience," Belanger said. "It's frustrating when you have to fight with the player to even experience it."
AKQA and Tennis Australia said that the Action Audio player would be improved in the future.
Belanger said that to get the full benefit of Action Audio, listeners need headphones or speakers that can separate sounds for the left and right ears. Without this, the audio is not as helpful.
Belanger suggested other improvements — such as a sound for when the ball hits the net — but overall, he said, the audio soundscape was well designed. As a sports fan, he now wants the same concept used for every sport. Belanger suggested other improvements — such as a sound for when the ball hits the net — but overall, he said, the audio soundscape was well designed. He now wants the same concept used for every sport.
For some blind and low-vision fans, Action Audio makes it easier to enjoy the game. Kala Petronijevic, 11, who is from Melbourne, is blind in her right eye and has limited vision in her left eye.
Kala has been playing blind tennis since she was 5. She is a big fan of the sport, but she didn't always enjoy playing it. She used to have to constantly ask her father who had hit the ball and what was going on.
Kala said it was hard following the game. "I wouldn't be interested because I wouldn't know what was going on." But with Action Audio, she can follow the game promptly.
Adrienne Hurst produced the audio.