Abstract
Previous studies have examined the syntactic capabilities of large pre-trained language models, such as BERT, by using stimuli from psycholinguistic studies. Studying well-known processing errors, such as NPI illusive effects can reveal whether a model prioritizes linear or hierarchical information when processing language. Recent experiments have found that BERT is mildly susceptible to Negative Polarity Item (NPI) illusion effects (Shin et al., 2023; Vu and Lee, 2022). We expand on these results by examining the effect of distance on the illusive effect, using and modifying stimuli from Parker and Phillips (2016). We also further tease apart whether the model is more affected by hierarchical distance or linear distance. We find that BERT is highly sensitive to syntactic hierarchical information: added hierarchical layers affected its processing capabilities compared to added linear distance.- Anthology ID:
- 2024.emnlp-main.530
- Volume:
- Proceedings of the 2024 Conference on Empirical Methods in Natural Language Processing
- Month:
- November
- Year:
- 2024
- Address:
- Miami, Florida, USA
- Editors:
- Yaser Al-Onaizan, Mohit Bansal, Yun-Nung Chen
- Venue:
- EMNLP
- SIG:
- Publisher:
- Association for Computational Linguistics
- Note:
- Pages:
- 9443–9457
- Language:
- URL:
- https://preview.aclanthology.org/add_missing_videos/2024.emnlp-main.530/
- DOI:
- 10.18653/v1/2024.emnlp-main.530
- Cite (ACL):
- So Young Lee and Mai Ha Vu. 2024. The effects of distance on NPI illusive effects in BERT. In Proceedings of the 2024 Conference on Empirical Methods in Natural Language Processing, pages 9443–9457, Miami, Florida, USA. Association for Computational Linguistics.
- Cite (Informal):
- The effects of distance on NPI illusive effects in BERT (Lee & Vu, EMNLP 2024)
- PDF:
- https://preview.aclanthology.org/add_missing_videos/2024.emnlp-main.530.pdf