Silent Speech Interfaces for all
Recognising speech from ultrasound images of the tongue
Silent speech interfaces perform speech recognition and synthesis from articulatory data in order to restore spoken communication for users with voice impairments (for example, after laryngectomy) or to allow silent communication in situations where audible speech is undesirable. Much of the previous work in this area has focused on models learned on data from single speakers (called speaker-dependent models), which do not generalize to unknown speakers. This project proposes to investigate the first speaker-independent silent speech interface for continuous speech recognition from ultrasound images of the tongue. This interface will be benchmarked against a system trained on high-quality data from a single speaker (speaker-dependent model). Additionally, this project will investigate speaker adaptation techniques, which use small amounts of speaker-specific data to bridge the gap between speaker-dependent and independent systems.
Funded by the Carnegie Trust for the Universities of Scotland Research Incentive Grant – grant number RIG008585 (“Silent speech interfaces for all – recognising speech from ultrasound images of the tongue”).
- Robust Word Alignment of Child Speech Therapy Sessions using Data Augmentation on Audio and Ultrasound Tongue Imaging. Wang Yau Li. MSc dissertation. School of Philosophy, Psychology & Language Sciences. The University of Edinburgh. 2019.
- Weakly-Supervised Keyword Recognition Applied to Child Speech Therapy Data. Carlos Mocholí Calvo. MSc dissertation. School of Informatics, The University of Edinburgh, 2019.
- Ultrasound-based Audio-Visual Speech Recognition for Children with Speech Disorders. Alexandra Antonides. MSc dissertation. School of Philosophy, Psychology & Language Sciences. The University of Edinburgh. 2018.
- Phone recognition from Ultrasound Data in Child Speech. Jie Chi. MSc dissertation. School of Informatics, The University of Edinburgh, 2018.
- Speech, Gross, and Fine Motor Skills in Children with Autism. Louise McKeever. School of Psychological Sciences and Health, University of Strathclyde, 2020
- Using Ultrasound to Identify Inconsistency in Children’s Speech. Tom Starr-Marshall. School of Psychological Sciences and Health, University of Strathclyde, ongoing
Using Ultrasound Visual Biofeedback to Diagnose and Treat Speech Disorders in Children with Cleft Lip and Palate
Children with cleft lip and palate (CLP) often continue to have problems producing clear speech long after the clefts have been surgically repaired, leading to educational and social disadvantage. For these children, speech and language therapy is required to diagnose the specific type of speech disorder present and provide intervention. Currently, this is undertaken by listening to speech and transcribing it. This is problematic because clinicians disagree on transcriptions and using listening methods only can miss imperceptible but important errors (e.g., instead of producing “t” with the tip of the tongue, producing it incorrectly with both the tip and back of the tongue). This project developed a technical solution for improving diagnosis of speech disorders in children with CLP. Ultrasound Visual Biofeedback (U-VBF) uses ultrasound placed under the chin to provide a real-time view of the tongue during speech. By recording and analysing these movements during speech we observe errors which are impossible to hear, leading to better diagnosis and treatment planning.
Funded by Action Medical Research, 17/04/17 → 31/07/18
This project showed that it is possible to use ultrasound visual biofeedback to remediate previously intractable SSDs.
Funded by the Chief Scientist Office, 01/01/15 – 30/06/2016