CogNLP-Sheffield at CMCL 2021 Shared Task: Blending Cognitively Inspired Features with Transformer-based Language Models for Predicting Eye Tracking Patterns
Peter Vickers, Rosa Wainwright, Harish Tayyar Madabushi, Aline Villavicencio
Abstract
The CogNLP-Sheffield submissions to the CMCL 2021 Shared Task examine the value of a variety of cognitively and linguistically inspired features for predicting eye tracking patterns, as both standalone model inputs and as supplements to contextual word embeddings (XLNet). Surprisingly, the smaller pre-trained model (XLNet-base) outperforms the larger (XLNet-large), and despite evidence that multi-word expressions (MWEs) provide cognitive processing advantages, MWE features provide little benefit to either model.- Anthology ID:
- 2021.cmcl-1.16
- Volume:
- Proceedings of the Workshop on Cognitive Modeling and Computational Linguistics
- Month:
- June
- Year:
- 2021
- Address:
- Online
- Venue:
- CMCL
- SIG:
- Publisher:
- Association for Computational Linguistics
- Note:
- Pages:
- 125–133
- Language:
- URL:
- https://aclanthology.org/2021.cmcl-1.16
- DOI:
- 10.18653/v1/2021.cmcl-1.16
- Cite (ACL):
- Peter Vickers, Rosa Wainwright, Harish Tayyar Madabushi, and Aline Villavicencio. 2021. CogNLP-Sheffield at CMCL 2021 Shared Task: Blending Cognitively Inspired Features with Transformer-based Language Models for Predicting Eye Tracking Patterns. In Proceedings of the Workshop on Cognitive Modeling and Computational Linguistics, pages 125–133, Online. Association for Computational Linguistics.
- Cite (Informal):
- CogNLP-Sheffield at CMCL 2021 Shared Task: Blending Cognitively Inspired Features with Transformer-based Language Models for Predicting Eye Tracking Patterns (Vickers et al., CMCL 2021)
- PDF:
- https://preview.aclanthology.org/auto-file-uploads/2021.cmcl-1.16.pdf
- Data
- WikiText-103, WikiText-2