Modeling performance differences on cognitive tests using LSTMs and skip-thought vectors trained on reported media consumption.

Maury Courtland, Aida Davani, Melissa Reyes, Leigh Yeh, Jun Leung, Brendan Kennedy, Morteza Dehghani, Jason Zevin


Abstract
Cognitive tests have traditionally resorted to standardizing testing materials in the name of equality and because of the onerous nature of creating test items. This approach ignores participants’ diverse language experiences that potentially significantly affect testing outcomes. Here, we seek to explain our prior finding of significant performance differences on two cognitive tests (reading span and SPiN) between clusters of participants based on their media consumption. Here, we model the language contained in these media sources using an LSTM trained on corpora of each cluster’s media sources to predict target words. We also model semantic similarity of test items with each cluster’s corpus using skip-thought vectors. We find robust, significant correlations between performance on the SPiN test and the LSTMs and skip-thought models we present here, but not the reading span test.
Anthology ID:
W19-2106
Volume:
Proceedings of the Third Workshop on Natural Language Processing and Computational Social Science
Month:
June
Year:
2019
Address:
Minneapolis, Minnesota
Venue:
NLP+CSS
SIG:
Publisher:
Association for Computational Linguistics
Note:
Pages:
47–53
Language:
URL:
https://aclanthology.org/W19-2106
DOI:
10.18653/v1/W19-2106
Bibkey:
Cite (ACL):
Maury Courtland, Aida Davani, Melissa Reyes, Leigh Yeh, Jun Leung, Brendan Kennedy, Morteza Dehghani, and Jason Zevin. 2019. Modeling performance differences on cognitive tests using LSTMs and skip-thought vectors trained on reported media consumption.. In Proceedings of the Third Workshop on Natural Language Processing and Computational Social Science, pages 47–53, Minneapolis, Minnesota. Association for Computational Linguistics.
Cite (Informal):
Modeling performance differences on cognitive tests using LSTMs and skip-thought vectors trained on reported media consumption. (Courtland et al., NLP+CSS 2019)
Copy Citation:
PDF:
https://preview.aclanthology.org/ingestion-script-update/W19-2106.pdf