Speechreading supplemented by single-channel and multichannel tactile displays of voice fundamental frequency. 1995

R S Waldstein, and A Boothroyd
Center for Research in Speech and Hearing Sciences, Graduate School, City University of New York, USA.

The benefits of two tactile codes of voice fundamental frequency (F0) were evaluated as supplements to the speechreading of sentences in two short-term training studies, each using 12 adults with normal hearing. In Experiment 1, a multichannel spatiotemporal display of F0, known as Portapitch, was used to stimulate the index finger. In an attempt to improve on past performance with this display, the coding scheme was modified to better cover the F0 range of the talker in the training materials. For Experiment 2, to engage kinesthetic/proprioceptive pathways, a novel single-channel positional display was built, in which F0 was coded as the vertical displacement of a small finger-rest. Input to both displays consisted of synthesized replicas of the F0 contours of the sentences, prepared and perfected off-line. Training with the two tactile F0 displays included auditory presentation of the synthesized F0 contours in conjunction with the tactile patterns on alternate trials. Speechreading enhancement by the two tactile F0 displays was compared to the enhancement provided when auditory F0 information was available in conjunction with the tactile patterns, by auditory presentation of a sinusoidal indication of the presence or absence of voicing, and by a single-channel tactile display of the speech waveform presented to the index finger. Despite the modified coding strategy, the multichannel Portapitch provided a mean tactile speechreading enhancement of 7 percentage points, which was no greater than that found in previous studies. The novel positional F0 display provided only a 4 percentage point enhancement. Neither F0 display was better than the simple single-channel tactile transform of the full speech waveform, which gave a 7 percentage point enhancement effect. Auditory speechreading enhancement effects were 17 percentage points with the voicing indicator and approximately 35 percentage points when the auditory F0 contour was provided in conjunction with the tactile displays. The findings are consistent with the hypothesis that subjects were not taking full advantage of the F0 variation information available in the outputs of the two experimental tactile displays.

UI MeSH Term Description Entries
D008085 Lipreading The process by which an observer comprehends speech by watching the movements of the speaker's lips without hearing the speaker's voice. Speechreading,Lip Reading,Lip Readings,Lipreadings,Reading, Lip,Readings, Lip,Speechreadings
D008297 Male Males
D008875 Middle Aged An adult aged 45 - 64 years. Middle Age
D004867 Equipment Design Methods and patterns of fabricating machines and related hardware. Design, Equipment,Device Design,Medical Device Design,Design, Medical Device,Designs, Medical Device,Device Design, Medical,Device Designs, Medical,Medical Device Designs,Design, Device,Designs, Device,Designs, Equipment,Device Designs,Equipment Designs
D005260 Female Females
D006801 Humans Members of the species Homo sapiens. Homo sapiens,Man (Taxonomy),Human,Man, Modern,Modern Man
D000328 Adult A person having attained full growth or maturity. Adults are of 19 through 44 years of age. For a person between 19 and 24 years of age, YOUNG ADULT is available. Adults
D013067 Speech Perception The process whereby an utterance is decoded into a representation in terms of linguistic units (sequences of phonetic segments which combine to form lexical and grammatical morphemes). Speech Discrimination,Discrimination, Speech,Perception, Speech
D014110 Touch Sensation of making physical contact with objects, animate or inanimate. Tactile stimuli are detected by MECHANORECEPTORS in the skin and mucous membranes. Tactile Sense,Sense of Touch,Taction,Sense, Tactile,Senses, Tactile,Tactile Senses,Tactions,Touch Sense,Touch Senses

Related Publications

R S Waldstein, and A Boothroyd
September 1990, The Journal of the Acoustical Society of America,
R S Waldstein, and A Boothroyd
December 1988, Ear and hearing,
R S Waldstein, and A Boothroyd
July 1996, The Journal of the Acoustical Society of America,
R S Waldstein, and A Boothroyd
October 1992, Journal of speech and hearing research,
R S Waldstein, and A Boothroyd
December 1988, Ear and hearing,
R S Waldstein, and A Boothroyd
September 1984, The Journal of the Acoustical Society of America,
R S Waldstein, and A Boothroyd
January 1991, Journal of rehabilitation research and development,
R S Waldstein, and A Boothroyd
January 1985, The Journal of the Acoustical Society of America,
R S Waldstein, and A Boothroyd
January 1988, Journal of rehabilitation research and development,
Copied contents to your clipboard!