Abstract
In this paper, we present the details of our approaches that attained the second place in the shared task of the ACL 2022 Cognitive Modeling and Computational Linguistics Workshop. The shared task is focused on multi- and cross-lingual prediction of eye movement features in human reading behavior, which could provide valuable information regarding language processing. To this end, we train ‘adapters’ inserted into the layers of frozen transformer-based pretrained language models. We find that multilingual models equipped with adapters perform well in predicting eye-tracking features. Our results suggest that utilizing language- and task-specific adapters is beneficial and translating test sets into similar languages that exist in the training set could help with zero-shot transferability in the prediction of human reading behavior.- Anthology ID:
- 2022.cmcl-1.16
- Volume:
- Proceedings of the Workshop on Cognitive Modeling and Computational Linguistics
- Month:
- May
- Year:
- 2022
- Address:
- Dublin, Ireland
- Editors:
- Emmanuele Chersoni, Nora Hollenstein, Cassandra Jacobs, Yohei Oseki, Laurent Prévot, Enrico Santus
- Venue:
- CMCL
- SIG:
- Publisher:
- Association for Computational Linguistics
- Note:
- Pages:
- 136–144
- Language:
- URL:
- https://aclanthology.org/2022.cmcl-1.16
- DOI:
- 10.18653/v1/2022.cmcl-1.16
- Cite (ACL):
- Ece Takmaz. 2022. Team DMG at CMCL 2022 Shared Task: Transformer Adapters for the Multi- and Cross-Lingual Prediction of Human Reading Behavior. In Proceedings of the Workshop on Cognitive Modeling and Computational Linguistics, pages 136–144, Dublin, Ireland. Association for Computational Linguistics.
- Cite (Informal):
- Team DMG at CMCL 2022 Shared Task: Transformer Adapters for the Multi- and Cross-Lingual Prediction of Human Reading Behavior (Takmaz, CMCL 2022)
- PDF:
- https://preview.aclanthology.org/nschneid-patch-5/2022.cmcl-1.16.pdf