Imagine that you are on a family meeting or you are with friends. You are sitting at the table, everyone is talking and the topics are changing quickly. You see many lips moving very fast, you’re able to catch some words occasionally, but you can’t get involved in this conversation because for it is impossible for you to understand what they’re currently talking about. The next moment everyone is laughing. Sure, they’ve thrown some joke. Or maybe they’re laughing at you, who knows? You laugh along with them, but your smile is not sincere. You can’t ask anybody in this moment what is so funny, because you don’t want to interrupt the conversation or draw the attention of everyone towards yourself.
And you feel ashamed that you can’t understand any word by yourself. Your friends one day make the decision to go to one of those places, where it’s too noisy and dark for you to see their faces and lips. In some time, you tend to avoid going there with them. You feel too tired to pretend to be involved in the matters while you’re indeed distanced from your friends. You can’t attempt to read their lips nonstop either. Then, you are forced to stay at home, with a book or movie with captions, but this is very lonely. As a social person, you can't live in full isolation.
Does it sound familiar to you? This is the everyday life of millions of people in the world, who are hard-of-hearing. This situation is a big barrier between them and their families and friends. The deaf and hard-of-hearing people enter in a state of self-imposed isolation, where communication for most of them is an unpleasant part, straining and stressful. For the people with hearing impairments, it is impossible to follow the process thoroughly. Whether the gatherings are formal or informal, it is practically impossible for people who rely on lip reading to follow up on the conversations. The simple reason why they cannot do this is it is impossible for them to turn around in all directions in a second and get what the speakers are saying since the latter are exchanging words with incredible speed.
The AVA team is aware of this. They know how frustrating and profound the communication gap with friends and family could be, and decided to do something about it, because they believe that all deaf and hard-of-hearing people around the world should be able to lead a life without communication barriers. So, those three young men created the application AVA, that breaks the barriers, breaks the wall of silence and makes the communication easy and possible anywhere, anytime, at anyplace.
Meet Thibault Duchemin, Pieter Doevendans, and Skinner Cheng. Thibault is the child of deaf parents, Pieter is self-educated in communication via signs, and Skinner is the only deaf person on the team. In 2014, they started developing a mobile phone application, focused on the conversion of speech into text for a group of speakers. There are other applications that convert voice commands into text, but the great challenge for the AVA team is to create a smart application that can accurately catch and textualize conversations in real time and among large groups of people.
The team believes in combining audio and visual technologies. When a person is deaf, they have developed their visual attention by reading and lip reading. Most of the accessibility efforts are aimed at compensating hearing loss through hearing aids. People with natural hearing are accustomed to believing that it is enough for one deaf person to put a hearing aid, and they will automatically understand everything. Not at all! During group conversations, lip readability is not good. The distance between the participants as well as the background noise around them makes it harder for the hearing aid to transmit the surrounding sounds. What’s more, when the conversation is particularly dynamic, the hearing aid transmits a chaos of sounds and lip reading is already hampered to the maximum extent. Visual contact becomes much more difficult, hence tracking oral articulation during such a dynamic dialogue is hindered. That's why live captions or sign language are an opportunity to focus the attention of deaf peoples in just one direction. This is called visual support. But not everyone understands sign language.
The goal of the application’s developers is for it to become absolutely accessible. Although it is sometimes used by companies and in classrooms, subtitle service is costly with an average price of 120 dollars an hour. This is not realistic when talking about common accessibility. Not everyone could afford to set aside 100 dollars an hour for a lunch meeting with friends, right? The cost of accessibility should be one hundred times lower, according to the team, but it has hardly declined over the past 25 years. Young people developing the AVA application believe that accessibility largely remains a social experience. Every conversation includes people. There are barriers that technology can not break - prejudices, ignorance, laziness. Awareness on the matters is the way forward, the team said.
Today, it's hard to imagine everyday life without mobile phones with their many applications. Despite their availability, for millions of deaf and hard-of-hearing people, technical assistance is limited to the services of a professional interpreter. A radical solution is needed, and it comes about now, in the year 2016, on the eve of the holidays.
The application, called AVA, is already life-saving, as its creators claim. A wonderful application that allows people who prefer to read lips, to monitor dynamic communication on their smart phones. The application connects all smart phones of people who’ve downloaded it and merges them into a synchronized system of intelligent microphones that turn speech into text in seconds in real time. This allows the app to be used in almost any case and makes it easier for the readers, giving them 24/7 access to any conversation. On the screen of every phone AVA is a chat room, where the speech of many people speaking simultaneously is turned in text in real time.
The way of this challenge is long and hard. This is the first version of the AVA app that will be further developed, modified and improved. It needs to be multilingual and be able to recognize spoken words in foreign languages well. The ideal goal is communicating without barriers. It is indeed possible! :)
The app is currently available to download for Android and iOS.
Article and EN Translation by: Christina Tchoparova
Photo and video: AVA - Introducing AVA
Alliance NCAC “We Hear You” is the holder of publishing rights on this article