Abstract
It can be difficult to separate abstract linguistic knowledge in recurrent neural networks (RNNs) from surface heuristics. In this work, we probe for highly abstract syntactic constraints that have been claimed to govern the behavior of filler-gap dependencies across different surface constructions. For models to generalize abstract patterns in expected ways to unseen data, they must share representational features in predictable ways. We use cumulative priming to test for representational overlap between disparate filler-gap constructions in English and find evidence that the models learn a general representation for the existence of filler-gap dependencies. However, we find no evidence that the models learn any of the shared underlying grammatical constraints we tested. Our work raises questions about the degree to which RNN language models learn abstract linguistic representations.- Anthology ID:
- 2020.conll-1.39
- Volume:
- Proceedings of the 24th Conference on Computational Natural Language Learning
- Month:
- November
- Year:
- 2020
- Address:
- Online
- Editors:
- Raquel Fernández, Tal Linzen
- Venue:
- CoNLL
- SIG:
- SIGNLL
- Publisher:
- Association for Computational Linguistics
- Note:
- Pages:
- 486–495
- Language:
- URL:
- https://aclanthology.org/2020.conll-1.39
- DOI:
- 10.18653/v1/2020.conll-1.39
- Cite (ACL):
- Debasmita Bhattacharya and Marten van Schijndel. 2020. Filler-gaps that neural networks fail to generalize. In Proceedings of the 24th Conference on Computational Natural Language Learning, pages 486–495, Online. Association for Computational Linguistics.
- Cite (Informal):
- Filler-gaps that neural networks fail to generalize (Bhattacharya & van Schijndel, CoNLL 2020)
- PDF:
- https://preview.aclanthology.org/emnlp-22-attachments/2020.conll-1.39.pdf