Tracking naturalistic linguistic predictions with deep neural language models
S.l. : s.n.
In2019 Conference on Cognitive Computational Neuroscience, pp. 424-427
2019 Conference on Cognitive Computational Neuroscience (13-16 September 2019, Berlin, Germany)
Article in monograph or in proceedings
Display more detailsDisplay less details
PI Group Predictive Brain
SW OZ DCC PL
PI Group Neurobiology of Language
SW OZ DCC CO
2019 Conference on Cognitive Computational Neuroscience
Subject110 000 Neurocognition of Language; 180 000 Predictive Brain; Action, intention, and motor control; Psycholinguistics; Language in Interaction
Prediction in language has traditionally been studied using simple designs in which neural responses to expected and unexpected words are compared in a categorical fashion. However, these designs have been contested as being `prediction encouraging', potentially exaggerating the importance of prediction in language understanding. A few recent studies have begun to address these worries by using model-based approaches to probe the effects of linguistic predictability in naturalistic stimuli (e.g. continuous narrative). However, these studies so far only looked at very local forms of prediction, using models that take no more than the prior two words into account when computing a word's predictability. Here, we extend this approach using a state-of-the-art neural language model that can take roughly 500 times longer linguistic contexts into account. Predictability estimates from the neural network offer a much better fit to EEG data from subjects listening to naturalistic narrative than simpler models, and reveal strong surprise responses akin to the P200 and N400. These results show that predictability effects in language are not a side-effect of simple designs, and demonstrate the practical use of recent advances in AI for the cognitive neuroscience of language.
NWO (Grant code:info:eu-repo/grantAgreement/NWO/Gravitation/024.001.006)
Upload full text
Use your RU credentials (u/z-number and password) to log in with SURFconext to upload a file for processing by the repository team.