Causal Augmentation for Causal Sentence Classification
Fiona Anting Tan, Devamanyu Hazarika, See-Kiong Ng, Soujanya Poria, Roger Zimmermann
Abstract
Scarcity of annotated causal texts leads to poor robustness when training state-of-the-art language models for causal sentence classification. In particular, we found that models misclassify on augmented sentences that have been negated or strengthened with respect to its causal meaning. This is worrying since minor linguistic differences in causal sentences can have disparate meanings. Therefore, we propose the generation of counterfactual causal sentences by creating contrast sets (Gardner et al., 2020) to be included during model training. We experimented on two model architectures and predicted on two out-of-domain corpora. While our strengthening schemes proved useful in improving model performance, for negation, regular edits were insufficient. Thus, we also introduce heuristics like shortening or multiplying root words of a sentence. By including a mixture of edits when training, we achieved performance improvements beyond the baseline across both models, and within and out of corpus’ domain, suggesting that our proposed augmentation can also help models generalize.- Anthology ID:
- 2021.cinlp-1.1
- Volume:
- Proceedings of the First Workshop on Causal Inference and NLP
- Month:
- November
- Year:
- 2021
- Address:
- Punta Cana, Dominican Republic
- Editors:
- Amir Feder, Katherine Keith, Emaad Manzoor, Reid Pryzant, Dhanya Sridhar, Zach Wood-Doughty, Jacob Eisenstein, Justin Grimmer, Roi Reichart, Molly Roberts, Uri Shalit, Brandon Stewart, Victor Veitch, Diyi Yang
- Venue:
- CINLP
- SIG:
- Publisher:
- Association for Computational Linguistics
- Note:
- Pages:
- 1–20
- Language:
- URL:
- https://aclanthology.org/2021.cinlp-1.1
- DOI:
- 10.18653/v1/2021.cinlp-1.1
- Cite (ACL):
- Fiona Anting Tan, Devamanyu Hazarika, See-Kiong Ng, Soujanya Poria, and Roger Zimmermann. 2021. Causal Augmentation for Causal Sentence Classification. In Proceedings of the First Workshop on Causal Inference and NLP, pages 1–20, Punta Cana, Dominican Republic. Association for Computational Linguistics.
- Cite (Informal):
- Causal Augmentation for Causal Sentence Classification (Tan et al., CINLP 2021)
- PDF:
- https://preview.aclanthology.org/proper-vol2-ingestion/2021.cinlp-1.1.pdf
- Code
- tanfiona/causalaugment