Eye movements while viewing narrated, captioned, and silent videos. 2013

Nicholas M Ross, and Eileen Kowler
Department of Psychology, Rutgers University, Piscataway, NJ, USA. nickross@rci.rutgers.edu

Videos are often accompanied by narration delivered either by an audio stream or by captions, yet little is known about saccadic patterns while viewing narrated video displays. Eye movements were recorded while viewing video clips with (a) audio narration, (b) captions, (c) no narration, or (d) concurrent captions and audio. A surprisingly large proportion of time (>40%) was spent reading captions even in the presence of a redundant audio stream. Redundant audio did not affect the saccadic reading patterns but did lead to skipping of some portions of the captions and to delays of saccades made into the caption region. In the absence of captions, fixations were drawn to regions with a high density of information, such as the central region of the display, and to regions with high levels of temporal change (actions and events), regardless of the presence of narration. The strong attraction to captions, with or without redundant audio, raises the question of what determines how time is apportioned between captions and video regions so as to minimize information loss. The strategies of apportioning time may be based on several factors, including the inherent attraction of the line of sight to any available text, the moment by moment impressions of the relative importance of the information in the caption and the video, and the drive to integrate visual text accompanied by audio into a single narrative stream.

UI MeSH Term Description Entries
D009040 Motion Pictures The art, technique, or business of producing motion pictures for entertainment, propaganda, or instruction. Film,Motion Picture,Movie,Cinema,Films,Films as Topic,Movies,Movies as Topic,Cinemas,Films as Topics,Movies as Topics,Picture, Motion,Pictures, Motion
D011932 Reading Acquiring information from text.
D005133 Eye Movements Voluntary or reflex-controlled movements of the eye. Eye Movement,Movement, Eye,Movements, Eye
D006801 Humans Members of the species Homo sapiens. Homo sapiens,Man (Taxonomy),Human,Man, Modern,Modern Man
D000704 Analysis of Variance A statistical technique that isolates and assesses the contributions of categorical independent variables to variation in the mean of a continuous dependent variable. ANOVA,Analysis, Variance,Variance Analysis,Analyses, Variance,Variance Analyses
D001288 Attention Focusing on certain aspects of current experience to the exclusion of others. It is the act of heeding or taking notice or concentrating. Focus of Attention,Selective Attention,Social Attention,Attention Focus,Attention, Selective,Attention, Social,Selective Attentions
D014741 Video Recording The storing or preserving of video signals to be played back later via a transmitter or receiver. Audiovisual Recording,Videorecording,Audiovisual Recordings,Recording, Audiovisual,Recording, Video,Recordings, Audiovisual,Recordings, Video,Video Recordings,Videorecordings

Related Publications

Nicholas M Ross, and Eileen Kowler
April 1977, Perceptual and motor skills,
Nicholas M Ross, and Eileen Kowler
September 1971, Vision research,
Nicholas M Ross, and Eileen Kowler
August 2020, PsyCh journal,
Nicholas M Ross, and Eileen Kowler
May 2013, Journal of abnormal psychology,
Nicholas M Ross, and Eileen Kowler
December 2012, Archives of sexual behavior,
Nicholas M Ross, and Eileen Kowler
January 2014, Frontiers in psychology,
Nicholas M Ross, and Eileen Kowler
September 2018, IEEE transactions on pattern analysis and machine intelligence,
Nicholas M Ross, and Eileen Kowler
January 2019, International journal of psychophysiology : official journal of the International Organization of Psychophysiology,
Nicholas M Ross, and Eileen Kowler
November 1963, The American journal of psychiatry,
Copied contents to your clipboard!