CASSI: Contextual and Semantic Structure-based Interpolation Augmentation for Low-Resource NER

Tanmay Surana, Thi-Nga Ho, Kyaw Tun, Eng Siong Chng

Abstract
While text augmentation methods have been successful in improving performance in the low-resource setting, they suffer from annotation corruption for a token-level task like NER. Moreover, existing methods cannot reliably add context diversity to the dataset, which has been shown to be crucial for low-resource NER. In this work, we propose Contextual and Semantic Structure-based Interpolation (CASSI), a novel augmentation scheme that generates high-quality contextually diverse augmentations while avoiding annotation corruption by structurally combining a pair of semantically similar sentences to generate a new sentence while maintaining semantic correctness and fluency. To accomplish this, we generate candidate augmentations by performing multiple dependency parsing-based exchanges in a pair of semantically similar sentences that are filtered via scoring with a pretrained Masked Language Model and a metric to promote specificity. Experiments show that CASSI consistently outperforms existing methods at multiple low resource levels, in multiple languages, and for noisy and clean text.
Anthology ID:
2023.findings-emnlp.651
Volume:
Findings of the Association for Computational Linguistics: EMNLP 2023
Month:
December
Year:
2023
Address:
Singapore
Editors:
Houda Bouamor, Juan Pino, Kalika Bali
Venue:
Findings
SIG:
Publisher:
Association for Computational Linguistics
Note:
Pages:
9729–9742
Language:
URL:
https://aclanthology.org/2023.findings-emnlp.651
DOI:
10.18653/v1/2023.findings-emnlp.651
Bibkey:
Cite (ACL):
Tanmay Surana, Thi-Nga Ho, Kyaw Tun, and Eng Siong Chng. 2023. CASSI: Contextual and Semantic Structure-based Interpolation Augmentation for Low-Resource NER. In Findings of the Association for Computational Linguistics: EMNLP 2023, pages 9729–9742, Singapore. Association for Computational Linguistics.
Cite (Informal):
CASSI: Contextual and Semantic Structure-based Interpolation Augmentation for Low-Resource NER (Surana et al., Findings 2023)
Copy Citation:
PDF:
https://preview.aclanthology.org/nschneid-patch-5/2023.findings-emnlp.651.pdf
PDF Search