Causal Augmentation for Causal Sentence Classification

Fiona Anting Tan, Devamanyu Hazarika, See-Kiong Ng, Soujanya Poria, Roger Zimmermann


Abstract
Scarcity of annotated causal texts leads to poor robustness when training state-of-the-art language models for causal sentence classification. In particular, we found that models misclassify on augmented sentences that have been negated or strengthened with respect to its causal meaning. This is worrying since minor linguistic differences in causal sentences can have disparate meanings. Therefore, we propose the generation of counterfactual causal sentences by creating contrast sets (Gardner et al., 2020) to be included during model training. We experimented on two model architectures and predicted on two out-of-domain corpora. While our strengthening schemes proved useful in improving model performance, for negation, regular edits were insufficient. Thus, we also introduce heuristics like shortening or multiplying root words of a sentence. By including a mixture of edits when training, we achieved performance improvements beyond the baseline across both models, and within and out of corpus’ domain, suggesting that our proposed augmentation can also help models generalize.
Anthology ID:
2021.cinlp-1.1
Volume:
Proceedings of the First Workshop on Causal Inference and NLP
Month:
November
Year:
2021
Address:
Punta Cana, Dominican Republic
Venue:
CINLP
SIG:
Publisher:
Association for Computational Linguistics
Note:
Pages:
1–20
Language:
URL:
https://aclanthology.org/2021.cinlp-1.1
DOI:
10.18653/v1/2021.cinlp-1.1
Bibkey:
Cite (ACL):
Fiona Anting Tan, Devamanyu Hazarika, See-Kiong Ng, Soujanya Poria, and Roger Zimmermann. 2021. Causal Augmentation for Causal Sentence Classification. In Proceedings of the First Workshop on Causal Inference and NLP, pages 1–20, Punta Cana, Dominican Republic. Association for Computational Linguistics.
Cite (Informal):
Causal Augmentation for Causal Sentence Classification (Tan et al., CINLP 2021)
Copy Citation:
PDF:
https://preview.aclanthology.org/ingestion-script-update/2021.cinlp-1.1.pdf
Code
 tanfiona/causalaugment