Abstract
To grasp the true reasoning ability, the Natural Language Inference model should be evaluated on counterfactual data. TabPert facilitates this by generation of such counterfactual data for assessing model tabular reasoning issues. TabPert allows the user to update a table, change the hypothesis, change the labels, and highlight rows that are important for hypothesis classification. TabPert also details the technique used to automatically produce the table, as well as the strategies employed to generate the challenging hypothesis. These counterfactual tables and hypotheses, as well as the metadata, is then used to explore the existing model’s shortcomings methodically and quantitatively.- Anthology ID:
- 2021.emnlp-demo.39
- Volume:
- Proceedings of the 2021 Conference on Empirical Methods in Natural Language Processing: System Demonstrations
- Month:
- November
- Year:
- 2021
- Address:
- Online and Punta Cana, Dominican Republic
- Editors:
- Heike Adel, Shuming Shi
- Venue:
- EMNLP
- SIG:
- Publisher:
- Association for Computational Linguistics
- Note:
- Pages:
- 350–360
- Language:
- URL:
- https://aclanthology.org/2021.emnlp-demo.39
- DOI:
- 10.18653/v1/2021.emnlp-demo.39
- Cite (ACL):
- Nupur Jain, Vivek Gupta, Anshul Rai, and Gaurav Kumar. 2021. TabPert : An Effective Platform for Tabular Perturbation. In Proceedings of the 2021 Conference on Empirical Methods in Natural Language Processing: System Demonstrations, pages 350–360, Online and Punta Cana, Dominican Republic. Association for Computational Linguistics.
- Cite (Informal):
- TabPert : An Effective Platform for Tabular Perturbation (Jain et al., EMNLP 2021)
- PDF:
- https://preview.aclanthology.org/ingest-acl-2023-videos/2021.emnlp-demo.39.pdf
- Code
- utahnlp/tabpert