Probing Language Models for Understanding of Temporal Expressions

Shivin Thukral, Kunal Kukreja, Christian Kavouras


Abstract
We present three Natural Language Inference (NLI) challenge sets that can evaluate NLI models on their understanding of temporal expressions. More specifically, we probe these models for three temporal properties: (a) the order between points in time, (b) the duration between two points in time, (c) the relation between the magnitude of times specified in different units. We find that although large language models fine-tuned on MNLI have some basic perception of the order between points in time, at large, these models do not have a thorough understanding of the relation between temporal expressions.
Anthology ID:
2021.blackboxnlp-1.31
Volume:
Proceedings of the Fourth BlackboxNLP Workshop on Analyzing and Interpreting Neural Networks for NLP
Month:
November
Year:
2021
Address:
Punta Cana, Dominican Republic
Venue:
BlackboxNLP
SIG:
Publisher:
Association for Computational Linguistics
Note:
Pages:
396–406
Language:
URL:
https://aclanthology.org/2021.blackboxnlp-1.31
DOI:
10.18653/v1/2021.blackboxnlp-1.31
Bibkey:
Cite (ACL):
Shivin Thukral, Kunal Kukreja, and Christian Kavouras. 2021. Probing Language Models for Understanding of Temporal Expressions. In Proceedings of the Fourth BlackboxNLP Workshop on Analyzing and Interpreting Neural Networks for NLP, pages 396–406, Punta Cana, Dominican Republic. Association for Computational Linguistics.
Cite (Informal):
Probing Language Models for Understanding of Temporal Expressions (Thukral et al., BlackboxNLP 2021)
Copy Citation:
PDF:
https://preview.aclanthology.org/emnlp-22-attachments/2021.blackboxnlp-1.31.pdf
Code
 kunalkukreja21/temporal-expressions-evaluation-lm
Data
MultiNLI