Improved tactile speech perception using audio-to-tactile sensory substitution with formant frequency focusing. 2024

Mark D Fletcher, and Esma Akis, and Carl A Verschuur, and Samuel W Perry
University of Southampton Auditory Implant Service, University of Southampton, University Road, Southampton, SO17 1BJ, UK. M.D.Fletcher@soton.ac.uk.

Haptic hearing aids, which provide speech information through tactile stimulation, could substantially improve outcomes for both cochlear implant users and for those unable to access cochlear implants. Recent advances in wide-band haptic actuator technology have made new audio-to-tactile conversion strategies viable for wearable devices. One such strategy filters the audio into eight frequency bands, which are evenly distributed across the speech frequency range. The amplitude envelopes from the eight bands modulate the amplitudes of eight low-frequency tones, which are delivered through vibration to a single site on the wrist. This tactile vocoder strategy effectively transfers some phonemic information, but vowels and obstruent consonants are poorly portrayed. In 20 participants with normal touch perception, we tested (1) whether focusing the audio filters of the tactile vocoder more densely around the first and second formant frequencies improved tactile vowel discrimination, and (2) whether focusing filters at mid-to-high frequencies improved obstruent consonant discrimination. The obstruent-focused approach was found to be ineffective. However, the formant-focused approach improved vowel discrimination by 8%, without changing overall consonant discrimination. The formant-focused tactile vocoder strategy, which can readily be implemented in real time on a compact device, could substantially improve speech perception for haptic hearing aid users.

UI MeSH Term Description Entries

Related Publications

Mark D Fletcher, and Esma Akis, and Carl A Verschuur, and Samuel W Perry
August 2023, Scientific reports,
Mark D Fletcher, and Esma Akis, and Carl A Verschuur, and Samuel W Perry
January 2024, PloS one,
Mark D Fletcher, and Esma Akis, and Carl A Verschuur, and Samuel W Perry
February 2022, Scientific reports,
Mark D Fletcher, and Esma Akis, and Carl A Verschuur, and Samuel W Perry
November 2019, The Journal of the Acoustical Society of America,
Mark D Fletcher, and Esma Akis, and Carl A Verschuur, and Samuel W Perry
January 1974, The Journal of the Acoustical Society of America,
Mark D Fletcher, and Esma Akis, and Carl A Verschuur, and Samuel W Perry
November 2010, The Journal of the Acoustical Society of America,
Mark D Fletcher, and Esma Akis, and Carl A Verschuur, and Samuel W Perry
January 2013, Advances in experimental medicine and biology,
Mark D Fletcher, and Esma Akis, and Carl A Verschuur, and Samuel W Perry
January 2019, Restorative neurology and neuroscience,
Mark D Fletcher, and Esma Akis, and Carl A Verschuur, and Samuel W Perry
September 2022, Journal of visualized experiments : JoVE,
Mark D Fletcher, and Esma Akis, and Carl A Verschuur, and Samuel W Perry
May 2004, Annals of the New York Academy of Sciences,
Copied contents to your clipboard!