The predictive potential of hand gestures during conversation: An investigation of the timing of gestures in relation to speech
Stockholm : GESPIN
InGesture and Speech in Interaction (GESPIN2020), pp. 1-6
7th Gesture and Speech in Interaction Conference (GESPIN2020) (Stockholm, Sweden, 7-9 September 2020)
Article in monograph or in proceedings
Display more detailsDisplay less details
SW OZ DCC PL
PI Group Neurobiology of Language
Gesture and Speech in Interaction (GESPIN2020)
In face-to-face conversation, recipients might use the bodily movements of the speaker (e.g. gestures) to facilitate language processing. It has been suggested that one way through which this facilitation may happen is prediction. However, for this to be possible, gestures would need to precede speech, and it is unclear whether this is true during natural conversation. In a corpus of Dutch conversations, we annotated hand gestures that represent semantic information and occurred during questions, and the word(s) which corresponded most closely to the gesturally depicted meaning. Thus, we tested whether representational gestures temporally precede their lexical affiliates. Further, to see whether preceding gestures may indeed facilitate language processing, we asked whether the gesture-speech asynchrony predicts the response time to the question the gesture is part of. Gestures and their strokes (most meaningful movement component) indeed preceded the corresponding lexical information, thus demonstrating their predictive potential. However, while questions with gestures got faster responses than questions without, there was no evidence that questions with larger gesture-speech asynchronies get faster responses. These results suggest that gestures indeed have the potential to facilitate predictive language processing, but further analyses on larger datasets are needed to test for links between asynchrony and processing advantages.
This item appears in the following Collection(s)
- Academic publications 
- Donders Centre for Cognitive Neuroimaging 
- Faculty of Social Sciences 
Upload full text
Use your RU credentials (u/z-number and password) to log in with SURFconext to upload a file for processing by the repository team.