Study reveals that SRAVI reads lips with 86% accuracy.
The paper, entitled, “A user evaluation of speech/phrase recognition software in critically ill patients: a DECIDE-AI feasibility study” was published in June 2023, in Critical Care Journal. The paper, as evidence of the accuracy of SRAVI’s lipreading capabilities, is available at this link. Critical Care is an international, peer-reviewed, clinical medical journal.
The stated objectives of the study were: “Evaluating effectiveness of speech/phrase recognition software in critically ill patients with speech.” The subject of the study was Liopa’s SRAVI software, which uses DNN (deep neural networks) to recognise speech from lip movements alone.
The study was undertaken at Royal Preston Hospital, part of the Lancashire Teaching Hospitals NHS Trust, under the leadership of Dr. Shondipon Laha.
During the trial, 14 patients with tracheostomies utilised Liopa’s SRAVI lipreading app, a communications aid designed to assist critical care patients in hospital. These patients recorded a total of 616 patient recordings on the app. Phrases were identified via lip movements by SRAVI and returned with rankings for the 1st, 2nd and 3rd most likely phrases that the patient had said. As published in the study: “The overall results revealed a total recognition accuracy across all three ranks of 86%. The rank 1 recognition accuracy of the DNN method was 75%.”
This means that SRAVI returned the correct phrase as the first option three-quarters of the time, and the correct phrase was in the top three options 86 per cent of the time.
More information about the study
The full report can be downloaded at this link.
Authors: M. Musalia1, S. Laha1,5*, J. Cazalilla‑Chica1, J. Allan2, L. Roach2, J. Twamley1, S. Nanda1, M. Verlander1, A. Williams1, I. Kempe1, I. I. Patel1, F. Campbell‑West3, B. Blackwood4 and D. F. McAuley4
Results: A total of 616 patient recordings were taken with 516 phrase identifiable recordings. The overall results
revealed a total recognition accuracy across all three ranks of 86% using the DNN method. The rank 1 recognition
accuracy of the DNN method was 75%. The DTW method had a total recognition accuracy of 74%, with a rank 1 accu‑
racy of 48%.
Conclusion: This feasibility evaluation of a novel speech/phrase recognition app using SRAVI demonstrated a good
correlation between spoken phrases and app recognition. This suggests that speech/phrase recognition technology could be a therapeutic option to bridge the gap in communication in critically ill patients.
What is already known about this topic: Communication can be attempted using visual charts, eye gaze boards, alphabet boards, speech/phrase reading, gestures and speaking valves in critically ill patients with speech impairments.
What this study adds: Deep neural networks and dynamic time warping methods can be used to analyse lip movements and identify intended phrases.
How this study might affect research, practice and policy: Traditional speech recognition can fail for people with speech impairments. Our study shows that lip-reading software, using video instead of speech, has a role to play in bridging the communication gap in speech impairment.
Correspondence about the study should be directed to S. Laha and the Liopa team via email at the address here.
