Related Projects

Silent Speech Interfaces for all

Recognising speech from ultrasound images of the tongue

Silent speech interfaces perform speech recognition and synthesis from articulatory data in order to restore spoken communication for users with voice impairments (for example, after laryngectomy) or to allow silent communication in situations where audible speech is undesirable. Much of the previous work in this area has focused on models learned on data from single speakers (called speaker-dependent models), which do not generalize to unknown speakers. This project proposes to investigate the first speaker-independent silent speech interface for continuous speech recognition from ultrasound images of the tongue. This interface will be benchmarked against a system trained on high-quality data from a single speaker (speaker-dependent model). Additionally, this project will investigate speaker adaptation techniques, which use small amounts of speaker-specific data to bridge the gap between speaker-dependent and independent systems.

See some samples of the data that we are currently collecting: single-speaker dataset and multi-speaker dataset.

Funded by the Carnegie Trust for the Universities of Scotland Research Incentive Grant – grant number RIG008585 (“Silent speech interfaces for all – recognising speech from ultrasound images of the tongue”).

Student projects

Visualising Speech

Using Ultrasound Visual Biofeedback to Diagnose and Treat Speech Disorders in Children with Cleft Lip and Palate

Children with cleft lip and palate (CLP) often continue to have problems producing clear speech long after the clefts have been surgically repaired, leading to educational and social disadvantage. For these children, speech and language therapy is required to diagnose the specific type of speech disorder present and provide intervention. Currently, this is undertaken by listening to speech and transcribing it. This is problematic because clinicians disagree on transcriptions and using listening methods only can miss imperceptible but important errors (e.g., instead of producing “t” with the tip of the tongue, producing it incorrectly with both the tip and back of the tongue). This project developed a technical solution for improving diagnosis of speech disorders in children with CLP. Ultrasound Visual Biofeedback (U-VBF) uses ultrasound placed under the chin to provide a real-time view of the tongue during speech. By recording and analysing these movements during speech we observe errors which are impossible to hear, leading to better diagnosis and treatment planning.

Funded by Action Medical Research, 17/04/17 → 31/07/18

UltraPhonix

Ultrasound Visual Biofeedback Treatment for Speech Sound Disorders in Children

This project showed that it is possible to use ultrasound visual biofeedback to remediate previously intractable SSDs.

Funded by the Chief Scientist Office, 01/01/15 – 30/06/2016