We’re developing an AI based platform to allow the hard of hearing to stay connected to the world around them


What is lipreading?

Lipreading is a communication technique used by the hard-of-hearing.  Unlike sign language it doesn’t require both parties to be trained in the technique. From an accuracy perspective however, human lipreading is generally poor. Indeed, it requires intense levels of concentration for the lipreader. As such, it is not a favoured communication technique for the hard-of-hearing.  

The challenges

The optimal scenario for the lipreader is a face-to-face engagement with someone they know – ideally whose lip movements are familiar. Lipreading strangers is much more challenging. More often than not, interactions and environments are not ideal. Lipreading multiple speakers in a group is virtually impossible. People do not turn to face the lipreader, or speak one at a time in an orderly fashion!  Additionally, different mouth shapes, facial hair, rate of speech and distance from speaker all create problems for even the best trained lipreaders.  

Low accuracy

As a result, the accuracy of human lipreading is unfortunately very low.  Most lipreaders actually try to pick out keywords and ‘fill in the blanks’ given the context of the conversation.  In fact, lipreading is said to be 80% guesswork! Studies have shown that the best performing lipreaders struggle to achieve greater than 50% accuracy in ideal conditions.  These accuracy levels tail off markedly in longer tests, as the lipreader tires. Amongst the standard hearing population lipreading accuracy is about 10% – 1 word in 10.    

Why being lipreader “friendly” is important

Effective communication techniques allow the hard of hearing to stay connected to the world around them. They build confidence and develop social and communication skills. Not being able to understand what is being said can be frustrating and lead to a sense of isolation. Communication is part of human contact and is vital for mental well-being.  It is important that the hearing population are aware of the difficulties the hard of hearing have in using techniques such as lipreading. They should ensure, where possible, that they communicate in such a way that is best for the lipreader.  

You can read some suggestions on how to more lipreader “friendly”:  Healthy Hearing: Resolve to Improve Your Lipreading Skills and this Deaf Expressions blog: Five Tips to make Lipreading Easier. 

What we’re doing to help

Liopa is developing an automated lipreading platform – LipRead. We use videos of people speaking to train our AI-based LipRead platform to recognise speech from lip movements. LipRead is initially targeted at constrained vocabularies. This includes the command set for an in-vehicle voice-activation unit, for instance. It can be used to  improve the accuracy of other current speech recognition technologies, especially which analyse audio and are susceptible to background noise. 

Over time, LipRead will support larger vocabularies and more languages with increasing accuracy. With these additional capabilities, we plan to provide a smartphone application that can assist the hard-of-hearing in the difficult task of lipreading.

Contact us to find out more! 

 

 



Share this entry

Share on twitter
Share on linkedin
Share on reddit
Share on facebook
Share on whatsapp
Share on email

Check out related content

WordPress Lightbox Plugin
Scroll to Top