Will be working with the Defence and Security Accelerator (DASA) which funds innovative, exploitable ideas that could lead to a cost-effective advantage for UK armed forces and national security.

DASA has selected Liopa to take part in a new initiative. It will be investigating how behavioural analytics can improve understanding and measurement, help make confident and ethical predictions, and guide better judgements on interventions for defence and security.

DASA, a cross-Government organisation, finds and funds exploitable innovation to support UK defence and security quickly and effectively, and support UK prosperity. Its vision is for the UK to maintain its strategic advantage over its adversaries through the most innovative defence and security capabilities in the world.

Liopa will leverage its existing Visual Speech Recognition (VSR) technology which deciphers speech from analysis of Lip Movements for activities such as key word spotting. The existing VSR engine takes, as input, video of a subject(s) speaking. It uses advanced AI-Based techniques to predict most likely utterances. Liopa will adapt its technology to identify utterances of specified words in uploaded video content, where audio is either not present or of very poor quality.

“This competition set out to find and fund a wide range of exciting and diverse proposals to advance Behaviour Analytics capabilities for the Defence and Security sector. A large number of high quality proposals were received and we are delighted to offer Liopa this contract through the Defence and Security Accelerator,” commented Richard Leigh, Influence Programme Manager, Defence Science and Technology Laboratory.

Liam McQuillan, Founder and CEO, Liopa, said, “This represents a considerable stamp of approval. We were able to show how our idea will work, and how it fits in with a larger ecosystem and other data analytics feeds. We’ve the relevant Artificial Intelligence expertise and capability inhouse, and we’ll also be looking to grow our team of experts in Belfast.”

Since I remember, philosophy of life and science, curiosity about the inner workings of things, and how to make this world a better place, were my main concerns.

 

Some background

At the moment, I am an undergraduate student at the Technological Educational Institute of Peloponnese, Dept. of Computer Engineering, based in Peloponnese, Greece. During my time there, I got heavily interested in Artificial Intelligence (AI). I got hooked on the science behind Machine Learning (ML), as well as its real world applications. This led me to wonder whether or not there exists any internship out there combining both the academic and the industrial aspects of my interests and at the same time align with my concerns as much as possible.

So, what’s a better place than a top-world university spin-out company; leveraging cutting-edge technologies and creating state-of-the-art Deep Learning Visual Speech Recognition engines that aim to improve voice-impaired people’s lives?

Seems like a combination of – if not all, most of my concerns, right?

Why working here is so rewarding

I’ve been part of Liopa’s Research and Development team for almost two months now. I could safely say – already, that the working and learning experience have far exceeded my expectations.

One of my favorite projects is SRAVI. This is an artificial intelligence-based app which is being used to enhance tracheostomy patients’ ability to vocally express themselves. Which is – for sure, a contribution in making this world a better place to live in. A very serious reason and motivation to work for.

Here, I’m constantly being exposed to a broad set of interesting problems, challenges and technologies. One important aspect of my everyday life here is to stay updated on the latest advances in the fields of AI and ML, as well as a lot of research and experimentation with new technologies. My learning and working experience ranges from pure mathematical problems, to computer vision, image processing, data visualization, GPU parallel computing and projective geometry to even web crawlers. These days, for instance, I’m developing a custom neural network video classifier – so exciting!

People here come from both academic and business worlds as well – with tons of years of experience. This academic-industry mixture, in combination with the from-nature challenging artificial intelligence problems the company tries to solve, makes working here even more fun, interesting and a “never-get-bored” environment. In addition, small relaxing walks with the teammates around the beautiful area of ECIT’s building is an everyday ritual. Not to mention the flexibility and freedom employees are given – in terms of their “working style”.  As well, the company encourages an attitude of constantly inclining employees towards new skill acquisition and – ultimately their overall evolution. All in all, Liopa has simply built a great environment, with a powerful team, awesome and extremely smart people to work with!

Would I recommend an internship at Liopa?

Hands down this company has the most unique, challenging, cutting-edge and meaningful projects for an intern to work on.
Hands down one of the best places for an internship!

LIOPA secures Innovate UK Funding, along with partners Lancashire Teaching Hospitals NHS Foundation Trust and Queen’s University Belfast. They will deploy a communications aid for tracheostomy patients, aimed at improving patient engagement and autonomy.

Liopa, a spin out of the Centre for Secure Information Technologies (CSIT) at Queen’s University Belfast (QUB) has announced that it is to deliver a prototype patient/carer communications aid. Tracheostomy patients will use it in critical care environments.

Working along with Lancashire Teaching Hospitals NHS Foundation Trust and Queen’s University Belfast, Liopa will develop SRAVI (Speech Recognition App for the Voice Impaired). Compared to the limited alternatives available, SRAVI will provide an easy-to-use, accurate and cost effective method for communication between these patients, their family members and healthcare staff. SRAVI will integrate with LipRead, Liopa’s artificial intelligence engine for Visual Speech Recognition.

The initial project will focus on a select group of tracheostomy patients (approximately 10,000 tracheostomies are performed annually in the UK). They will struggle to vocalise but be able to move their lips normally. Whilst the initial prototype will support a limited vocabulary in English, the application can be further developed to support larger vocabularies across multiple languages.

Clinical Professor Danny McAuley at QUB’s Wellcome-Wolfson Institute for Experimental Medicine and Consultant at the Belfast Trust commented, “The inability to communicate during an ICU stay is a major source of morbidity for patients, family and staff. A patient’s non-verbal attempts to communicate are often difficult to understand. This can be frustrating for patients and carers. This novel approach may allow better communication between the patient, staff and family from an early stage.”

“This is an innovative application of our proven AI-based Visual Speech Recognition (VSR) system LipRead. LipRead analyses and translates lip movements into recognisable words. The technology allows the translation of lip movement to text using a mobile app on a mobile device. It requires very little training and is inexpensive,” said Liam McQuillan, Co-founder and CEO, Liopa. He continued, “SRAVI can be deployed on commodity smartphones and tablets, that can be used by multiple patients.  Alternative technologies, such as ‘eye-gaze’ systems, require bespoke hardware and are generally much more expensive.”

Shondipon Laha is a Consultant in Critical Care and Anaesthesia at Lancashire Teaching Hospital. He explained further, “This project will address a government priority to implement new digital solutions in the NHS. SRAVI will deliver improved patient-carer communications for patients with tracheostomies. It therefore reduces rehabilitation times in expensive ICU settings.”

The project will run for 9 months. It will include an evaluation phase, carried out in hospital critical care environments in Lancashire and Belfast. It has been funded by UK Research and Innovation. This new organisation brings together the UK Research Councils, Innovate UK and Research England. It creates the best environment for research and innovation to flourish, to ensure the UK maintains its world-leading position in research and innovation.

Find out more about what lipreading is from this short video….



Belfast AI Startup Liopa Raises Seed funding on the Syndicate Room crowdfunding platform

A spin-out from Queen’s University Belfast and the Centre for Secure IT, Liopa is developing lipreading technology to enable visual speech recognition. 

AI-based technology startup Liopa has completed a very successful fund-raising campaign on the SyndicateRoom crowdfunding website. The company raised 2.5 times the target amount in a 4 week period. It received strong backing from a number of angel investors and the Fund Twenty8 EIS fund. 

Founded in 2015, Liopa is commercialising over 10 years of research in the field of Speech and Image processing. The company’s technology can determine speech by analysing the movement of a user’s lips as they speak into a camera. In addition, the technology can be used to prevent “spoofing” and security issues in facial recognition systems. 

Liopa’s primary focus is improving the accuracy of voice driven applications which have risen in popularity. Virtual assistants such as Apple’s Siri and Amazon’s Alexa have brought voice interaction into the mainstream. And now corporations such as Google and Sonos are following their lead. These voice driven systems, however, rely on audio speech recognition (ASR) to determine speech. This means their accuracy deteriorates with the increase of real-world audio noise – for example, in a busy restaurant or outside on a windy day. 

Liopa’s visual speech recognition technology, LipRead, is designed to decipher speech from lip movements. It is therefore agnostic to audio noise. Liopa hopes to augment existing voice driven systems in real-world environments, improving accuracy when background noise is present. The company calls this usage of LipRead “ASR-Assist”. 

Speaking about the funding round, Liopa’s CEO Liam McQuillan said: “This investment will allow us to grow our engineering capability and AI talent.” He continued, “We’ll be able to accelerate the exciting developments we have planned in our roadmap, and protect our valuable IP.”

SyndicateRoom’s co-founder, Tom Britton commented, “It’s no wonder the technology being developed by Liopa is so incredible. The team behind it have spent a combined 50+ man years researching or developing the technology. Their backgrounds range from Senior Academics to C- suite Commercial ranging from startups to the likes of Intel. The applications for their platform are wide ranging, everything from helping law enforcement decipher what’s been said on CCTV footage to giving those who have lost their ability to vocalise a new way to easily communicate. We’re delighted to play a role in such an innovative technology that is applying machine learning and AI for an ultimate good.” 

Launched in September 2013, SyndicateRoom is an online investment platform. It has helped 170+ early-stage UK businesses secure more than £215 million in funding through its investor-led equity crowdfunding model.  

Watch the Liopa company overview video …..




What is lipreading?

Lipreading is a communication technique used by the hard-of-hearing.  Unlike sign language it doesn’t require both parties to be trained in the technique. From an accuracy perspective however, human lipreading is generally poor. Indeed, it requires intense levels of concentration for the lipreader. As such, it is not a favoured communication technique for the hard-of-hearing.  

The challenges

The optimal scenario for the lipreader is a face-to-face engagement with someone they know – ideally whose lip movements are familiar. Lipreading strangers is much more challenging. More often than not, interactions and environments are not ideal. Lipreading multiple speakers in a group is virtually impossible. People do not turn to face the lipreader, or speak one at a time in an orderly fashion!  Additionally, different mouth shapes, facial hair, rate of speech and distance from speaker all create problems for even the best trained lipreaders.  

Low accuracy

As a result, the accuracy of human lipreading is unfortunately very low.  Most lipreaders actually try to pick out keywords and ‘fill in the blanks’ given the context of the conversation.  In fact, lipreading is said to be 80% guesswork! Studies have shown that the best performing lipreaders struggle to achieve greater than 50% accuracy in ideal conditions.  These accuracy levels tail off markedly in longer tests, as the lipreader tires. Amongst the standard hearing population lipreading accuracy is about 10% – 1 word in 10.    

Why being lipreader “friendly” is important

Effective communication techniques allow the hard of hearing to stay connected to the world around them. They build confidence and develop social and communication skills. Not being able to understand what is being said can be frustrating and lead to a sense of isolation. Communication is part of human contact and is vital for mental well-being.  It is important that the hearing population are aware of the difficulties the hard of hearing have in using techniques such as lipreading. They should ensure, where possible, that they communicate in such a way that is best for the lipreader.  

You can read some suggestions on how to more lipreader “friendly”:  Healthy Hearing: Resolve to Improve Your Lipreading Skills and this Deaf Expressions blog: Five Tips to make Lipreading Easier. 

What we’re doing to help

Liopa is developing an automated lipreading platform – LipRead. We use videos of people speaking to train our AI-based LipRead platform to recognise speech from lip movements. LipRead is initially targeted at constrained vocabularies. This includes the command set for an in-vehicle voice-activation unit, for instance. It can be used to  improve the accuracy of other current speech recognition technologies, especially which analyse audio and are susceptible to background noise. 

Over time, LipRead will support larger vocabularies and more languages with increasing accuracy. With these additional capabilities, we plan to provide a smartphone application that can assist the hard-of-hearing in the difficult task of lipreading.

Contact us to find out more! 

 

 



I was intrigued by an internship vacancy from Liopa, a company that, at its core, uses machine learning to effectively read peoples’ lips from videos. So I made an application for the role – and, after a successful interview, I happily accepted the post.

I have now been working at Liopa for a little over nine months. From creating custom neural network animations in Matlab, web servers using flask, making use of the Google Speech API, complete Web SDKs in JavaScript to creating custom data augmentation algorithms for videos in python, it is safe to say that every day is different.

I’m based in the Queen’s University Institute of Electronics, Communications and Information Technology  (ECIT) building. There are a number of options for lunch and coffee in the building or a mere stones throw away. Buying your own coffee can prove difficult as the CEO and COO are always buying coffee for the team, just one of many company perks.

Working at Liopa has been and continues to be a challenging yet rewarding experience. Employees are given a lot of flexibility and allowed to independently to work through issues for themselves without being micromanaged. Additionally, the company also has a strong desire to let employees grow by learning new languages, tools and frameworks. This is in order to achieve a company objective whilst simultaneously helping the team gain valuable knowledge and expertise.

Would I recommend an internship at Liopa? Absolutely!



Visual Speech Recognition - Read My Lips photo

Liopa commences trials of AI-based LipSecure with established User Authentication and Identity Verification providers to provide enhanced anti-spoofing capabilities.

Read more

EU ERDF

Liopa has received an Invest NI Grant for Research and Development, targeted towards a project to develop LipRead, a Visual Speech Recognition (VSR) product.

Read more

Belfast-based technology specialist Liopa has commercially launched the world’s first automated Lip Reader. The service will initially be used to prevent ‘spoofing’ in Facial Recognition systems where there is a threat of compromise from images or videos of the subject being presented by an imposter.

Read more