LLMs Struggle with NLI for Perfect Aspect: A Cross-Linguistic Study in Chinese and Japanese

Lu Jie, Du Jin, Hitomi Yanaka


Abstract
Unlike English, which uses distinct forms (e.g., had, has, will have) to mark the perfect aspect across tenses, Chinese and Japanese lack sep- arate grammatical forms for tense within the perfect aspect, which complicates Natural Lan- guage Inference (NLI). Focusing on the per- fect aspect in these languages, we construct a linguistically motivated, template-based NLI dataset (1,350 pairs per language). Experi- ments reveal that even advanced LLMs strug- gle with temporal inference, particularly in de- tecting subtle tense and reference-time shifts. These findings highlight model limitations and underscore the need for cross-linguistic evalua- tion in temporal semantics. Our dataset is avail- able at https://github.com/Lujie2001/ CrossNLI.
Anthology ID:
2025.iwcs-1.9
Volume:
Proceedings of the 16th International Conference on Computational Semantics
Month:
September
Year:
2025
Address:
Düsseldorf, Germany
Editors:
Kilian Evang, Laura Kallmeyer, Sylvain Pogodalla
Venues:
IWCS | WS
SIG:
Publisher:
Association for Computational Linguistics
Note:
Pages:
99–107
Language:
URL:
https://preview.aclanthology.org/iwcs-25-ingestion/2025.iwcs-1.9/
DOI:
Bibkey:
Cite (ACL):
Lu Jie, Du Jin, and Hitomi Yanaka. 2025. LLMs Struggle with NLI for Perfect Aspect: A Cross-Linguistic Study in Chinese and Japanese. In Proceedings of the 16th International Conference on Computational Semantics, pages 99–107, Düsseldorf, Germany. Association for Computational Linguistics.
Cite (Informal):
LLMs Struggle with NLI for Perfect Aspect: A Cross-Linguistic Study in Chinese and Japanese (Jie et al., IWCS 2025)
Copy Citation:
PDF:
https://preview.aclanthology.org/iwcs-25-ingestion/2025.iwcs-1.9.pdf