Abstract
Recent work in cross-topic argument mining attempts to learn models that generalise across topics rather than merely relying on within-topic spurious correlations. We examine the effectiveness of this approach by analysing the output of single-task and multi-task models for cross-topic argument mining, through a combination of linear approximations of their decision boundaries, manual feature grouping, challenge examples, and ablations across the input vocabulary. Surprisingly, we show that cross-topic models still rely mostly on spurious correlations and only generalise within closely related topics, e.g., a model trained only on closed-class words and a few common open-class words outperforms a state-of-the-art cross-topic model on distant target topics.- Anthology ID:
- 2021.starsem-1.25
- Volume:
- Proceedings of *SEM 2021: The Tenth Joint Conference on Lexical and Computational Semantics
- Month:
- August
- Year:
- 2021
- Address:
- Online
- Editors:
- Lun-Wei Ku, Vivi Nastase, Ivan Vulić
- Venue:
- *SEM
- SIG:
- SIGLEX
- Publisher:
- Association for Computational Linguistics
- Note:
- Pages:
- 263–277
- Language:
- URL:
- https://aclanthology.org/2021.starsem-1.25
- DOI:
- 10.18653/v1/2021.starsem-1.25
- Cite (ACL):
- Terne Sasha Thorn Jakobsen, Maria Barrett, and Anders Søgaard. 2021. Spurious Correlations in Cross-Topic Argument Mining. In Proceedings of *SEM 2021: The Tenth Joint Conference on Lexical and Computational Semantics, pages 263–277, Online. Association for Computational Linguistics.
- Cite (Informal):
- Spurious Correlations in Cross-Topic Argument Mining (Thorn Jakobsen et al., *SEM 2021)
- PDF:
- https://preview.aclanthology.org/nschneid-patch-3/2021.starsem-1.25.pdf
- Code
- terne/spurious_correlations_in_argmin