Abstract
We explore the connection between presupposition, discourse and sarcasm and propose to leverage that connection in a transfer learning scenario with the goal of improving the performance of NLI models on cases involving presupposition. We exploit advances in training transformer-based models that show that pre-finetuning—–i.e., finetuning the model on an additional task or dataset before the actual finetuning phase—–can help these models, in some cases, achieve a higher performance on a given downstream task. Building on those advances and that aforementioned connection, we propose pre-finetuning NLI models on carefully chosen tasks in an attempt to improve their performance on NLI cases involving presupposition. We notice that, indeed, pre-finetuning on those tasks leads to performance improvements. Furthermore, we run several diagnostic tests to understand whether these gains are merely a byproduct of additional training data. The results show that, while additional training data seems to be helping on its own in some cases, the choice of the tasks plays a role in the performance improvements.- Anthology ID:
- 2023.findings-emnlp.703
- Volume:
- Findings of the Association for Computational Linguistics: EMNLP 2023
- Month:
- December
- Year:
- 2023
- Address:
- Singapore
- Editors:
- Houda Bouamor, Juan Pino, Kalika Bali
- Venue:
- Findings
- SIG:
- Publisher:
- Association for Computational Linguistics
- Note:
- Pages:
- 10482–10494
- Language:
- URL:
- https://aclanthology.org/2023.findings-emnlp.703
- DOI:
- 10.18653/v1/2023.findings-emnlp.703
- Cite (ACL):
- Jad Kabbara and Jackie Cheung. 2023. Investigating the Effect of Pre-finetuning BERT Models on NLI Involving Presuppositions. In Findings of the Association for Computational Linguistics: EMNLP 2023, pages 10482–10494, Singapore. Association for Computational Linguistics.
- Cite (Informal):
- Investigating the Effect of Pre-finetuning BERT Models on NLI Involving Presuppositions (Kabbara & Cheung, Findings 2023)
- PDF:
- https://preview.aclanthology.org/emnlp-22-attachments/2023.findings-emnlp.703.pdf