Overview of CCL23-Eval Task 2: The Third Chinese Abstract Meaning Representation Parsing Evaluation
Zhixing Xu, Yixuan Zhang, Bin Li, Zhou Junsheng, Weiguang Qu
Abstract
“Abstract Meaning Representation has emerged as a prominent area of research in sentence-levelsemantic parsing within the field of natural language processing in recent years. Substantialprogress has been made in various NLP subtasks through the application of AMR. This paperpresents the third Chinese Abstract Meaning Representation Parsing Evaluation, held as part ofthe Technical Evaluation Task Workshop at the 22nd Chinese Computational Linguistics Confer-ence. The evaluation was specifically tailored for the Chinese and utilized the Align-smatch met-ric as the standard evaluation criterion. Building upon high-quality semantic annotation schemesand annotated corpora, this evaluation introduced a new test set comprising interrogative sen-tences for comprehensive evaluation. The results of the evaluation, as measured by the F-score,indicate notable performance achievements. The top-performing team attained a score of 0.8137in the closed test and 0.8261 in the open test, respectively, using the Align-smatch metric. No-tably, the leading result surpassed the SOTA performance at CoNLL 2020 by 3.64 percentagepoints when evaluated using the MRP metric. Further analysis revealed that this significantprogress primarily stemmed from improved relation prediction between concepts. However, thechallenge of effectively utilizing semantic relation alignments remains an area that requires fur-ther enhancement.”- Anthology ID:
- 2023.ccl-3.7
- Volume:
- Proceedings of the 22nd Chinese National Conference on Computational Linguistics (Volume 3: Evaluations)
- Month:
- August
- Year:
- 2023
- Address:
- Harbin, China
- Editors:
- Maosong Sun, Bing Qin, Xipeng Qiu, Jing Jiang, Xianpei Han
- Venue:
- CCL
- SIG:
- Publisher:
- Chinese Information Processing Society of China
- Note:
- Pages:
- 70–83
- Language:
- English
- URL:
- https://aclanthology.org/2023.ccl-3.7
- DOI:
- Cite (ACL):
- Zhixing Xu, Yixuan Zhang, Bin Li, Zhou Junsheng, and Weiguang Qu. 2023. Overview of CCL23-Eval Task 2: The Third Chinese Abstract Meaning Representation Parsing Evaluation. In Proceedings of the 22nd Chinese National Conference on Computational Linguistics (Volume 3: Evaluations), pages 70–83, Harbin, China. Chinese Information Processing Society of China.
- Cite (Informal):
- Overview of CCL23-Eval Task 2: The Third Chinese Abstract Meaning Representation Parsing Evaluation (Xu et al., CCL 2023)
- PDF:
- https://preview.aclanthology.org/naacl24-info/2023.ccl-3.7.pdf