Abstract
We study whether linguistic information in pre-trained multilingual language models can be accessed by human language: So far, there is no easy method to directly obtain linguistic information and gain insights into the linguistic principles encoded in such models. We use the technique of prompting and formulate linguistic tasks to test the LM’s access to explicit grammatical principles and study how effective this method is at providing access to linguistic features. Our experiments on German, Icelandic and Spanish show that some linguistic properties can in fact be accessed through prompting, whereas others are harder to capture.- Anthology ID:
- 2023.emnlp-main.454
- Volume:
- Proceedings of the 2023 Conference on Empirical Methods in Natural Language Processing
- Month:
- December
- Year:
- 2023
- Address:
- Singapore
- Editors:
- Houda Bouamor, Juan Pino, Kalika Bali
- Venue:
- EMNLP
- SIG:
- Publisher:
- Association for Computational Linguistics
- Note:
- Pages:
- 7328–7336
- Language:
- URL:
- https://aclanthology.org/2023.emnlp-main.454
- DOI:
- 10.18653/v1/2023.emnlp-main.454
- Cite (ACL):
- Marion Di Marco, Katharina Hämmerl, and Alexander Fraser. 2023. A Study on Accessing Linguistic Information in Pre-Trained Language Models by Using Prompts. In Proceedings of the 2023 Conference on Empirical Methods in Natural Language Processing, pages 7328–7336, Singapore. Association for Computational Linguistics.
- Cite (Informal):
- A Study on Accessing Linguistic Information in Pre-Trained Language Models by Using Prompts (Di Marco et al., EMNLP 2023)
- PDF:
- https://preview.aclanthology.org/emnlp22-frontmatter/2023.emnlp-main.454.pdf