Classical Chinese Historical Event Detection Evaluation

Feng Zhenbing, Li Wei, Shao Yanqiu


Abstract
“Event detection involves identifying and extracting event information from natural language texts. The complex syntax and semantics of Classical Chinese, coupled with its limited usage, pose significant challenges for information extraction tasks on classical Chinese texts. At the 23rd China National Conference on Computational Linguistics (CCL 2024), we launched an evaluation task focused on the extraction of historical events from Classical Chinese. We used our constructed Classical Chinese Historical Event Logical Schema to identify event triggers and classify event types. The evaluation utilized the Classical Chinese Historical Event Detection Dataset (CHED), annotated from The Twenty-Four Histories corpus, with the aim of enhancing event extraction technologies and advancing the digital study of classical Chinese historical texts. The evaluation included two subtasks and attracted 28 teams, with 15 teams submitting valid results. In the subtask of trigger identification, the best-performing system achieved an Exact match score of 63.6%. In the subtasks of coarse-grained and fine-grained event type classification, the top systems achieved F1-scores of 84.5% and 81.4%, respectively.”
Anthology ID:
2024.ccl-3.22
Volume:
Proceedings of the 23rd Chinese National Conference on Computational Linguistics (Volume 3: Evaluations)
Month:
July
Year:
2024
Address:
Taiyuan, China
Editors:
Hongfei Lin, Hongye Tan, Bin Li
Venue:
CCL
SIG:
Publisher:
Chinese Information Processing Society of China
Note:
Pages:
201–209
Language:
English
URL:
https://preview.aclanthology.org/fix-sig-urls/2024.ccl-3.22/
DOI:
Bibkey:
Cite (ACL):
Feng Zhenbing, Li Wei, and Shao Yanqiu. 2024. Classical Chinese Historical Event Detection Evaluation. In Proceedings of the 23rd Chinese National Conference on Computational Linguistics (Volume 3: Evaluations), pages 201–209, Taiyuan, China. Chinese Information Processing Society of China.
Cite (Informal):
Classical Chinese Historical Event Detection Evaluation (Zhenbing et al., CCL 2024)
Copy Citation:
PDF:
https://preview.aclanthology.org/fix-sig-urls/2024.ccl-3.22.pdf