Extracting or Guessing? Improving Faithfulness of Event Temporal Relation Extraction
Haoyu Wang, Hongming Zhang, Yuqian Deng, Jacob Gardner, Dan Roth, Muhao Chen
Abstract
In this paper, we seek to improve the faithfulness of TempRel extraction models from two perspectives. The first perspective is to extract genuinely based on contextual description. To achieve this, we propose to conduct counterfactual analysis to attenuate the effects of two significant types of training biases: the event trigger bias and the frequent label bias. We also add tense information into event representations to explicitly place an emphasis on the contextual description. The second perspective is to provide proper uncertainty estimation and abstain from extraction when no relation is described in the text. By parameterization of Dirichlet Prior over the model-predicted categorical distribution, we improve the model estimates of the correctness likelihood and make TempRel predictions more selective. We also employ temperature scaling to recalibrate the model confidence measure after bias mitigation. Through experimental analysis on MATRES, MATRES-DS, and TDDiscourse, we demonstrate that our model extracts TempRel and timelines more faithfully compared to SOTA methods, especially under distribution shifts.- Anthology ID:
- 2023.eacl-main.39
- Volume:
- Proceedings of the 17th Conference of the European Chapter of the Association for Computational Linguistics
- Month:
- May
- Year:
- 2023
- Address:
- Dubrovnik, Croatia
- Editors:
- Andreas Vlachos, Isabelle Augenstein
- Venue:
- EACL
- SIG:
- Publisher:
- Association for Computational Linguistics
- Note:
- Pages:
- 541–553
- Language:
- URL:
- https://aclanthology.org/2023.eacl-main.39
- DOI:
- 10.18653/v1/2023.eacl-main.39
- Cite (ACL):
- Haoyu Wang, Hongming Zhang, Yuqian Deng, Jacob Gardner, Dan Roth, and Muhao Chen. 2023. Extracting or Guessing? Improving Faithfulness of Event Temporal Relation Extraction. In Proceedings of the 17th Conference of the European Chapter of the Association for Computational Linguistics, pages 541–553, Dubrovnik, Croatia. Association for Computational Linguistics.
- Cite (Informal):
- Extracting or Guessing? Improving Faithfulness of Event Temporal Relation Extraction (Wang et al., EACL 2023)
- PDF:
- https://preview.aclanthology.org/naacl-24-ws-corrections/2023.eacl-main.39.pdf