Abstract
Automatic text dating(ATD) is a challenging task since explicit temporal mentions usually do not appear in texts. Existing state-of-the-art approaches learn word representations via language models, whereas most of them ignore diachronic change of words, which may affect the efforts of text modeling. Meanwhile, few of them consider text modeling for long diachronic documents. In this paper, we present a time-aware language model named TALM, to learn temporal word representations by transferring language models of general domains to those of time-specific ones. We also build a hierarchical modeling approach to represent diachronic documents by encoding them with temporal word representations. Experiments on a Chinese diachronic corpus show that our model effectively captures implicit temporal information of words, and outperforms state-of-the-art approaches in historical text dating as well.- Anthology ID:
- 2023.findings-emnlp.911
- Volume:
- Findings of the Association for Computational Linguistics: EMNLP 2023
- Month:
- December
- Year:
- 2023
- Address:
- Singapore
- Editors:
- Houda Bouamor, Juan Pino, Kalika Bali
- Venue:
- Findings
- SIG:
- Publisher:
- Association for Computational Linguistics
- Note:
- Pages:
- 13646–13656
- Language:
- URL:
- https://aclanthology.org/2023.findings-emnlp.911
- DOI:
- 10.18653/v1/2023.findings-emnlp.911
- Cite (ACL):
- Han Ren, Hai Wang, Yajie Zhao, and Yafeng Ren. 2023. Time-Aware Language Modeling for Historical Text Dating. In Findings of the Association for Computational Linguistics: EMNLP 2023, pages 13646–13656, Singapore. Association for Computational Linguistics.
- Cite (Informal):
- Time-Aware Language Modeling for Historical Text Dating (Ren et al., Findings 2023)
- PDF:
- https://preview.aclanthology.org/proper-vol2-ingestion/2023.findings-emnlp.911.pdf