Abstract
We propose a novel method for calculating PARSEVAL measures to evaluate constituent parsing results. Previous constituent parsing evaluation techniques were constrained by the requirement for consistent sentence boundaries and tokenization results, proving to be stringent and inconvenient. Our new approach handles constituent parsing results obtained from raw text, even when sentence boundaries and tokenization differ from the preprocessed gold sentence. Implementing this measure is our evaluation by alignment approach. The algorithm enables the alignment of tokens and sentences in the gold and system parse trees. Our proposed algorithm draws on the analogy of sentence and word alignment commonly used in machine translation (MT). To demonstrate the intricacy of calculations and clarify any integration of configurations, we explain the implementations in detailed pseudo-code and provide empirical proof for how sentence and word alignment can improve evaluation reliability.- Anthology ID:
- 2024.cl-3.10
- Volume:
- Computational Linguistics, Volume 50, Issue 3 - September 2024
- Month:
- September
- Year:
- 2024
- Address:
- Cambridge, MA
- Venue:
- CL
- SIG:
- Publisher:
- MIT Press
- Note:
- Pages:
- 1181–1190
- Language:
- URL:
- https://preview.aclanthology.org/build-pipeline-with-new-library/2024.cl-3.10/
- DOI:
- 10.1162/coli_a_00512
- Cite (ACL):
- Eunkyul Leah Jo, Angela Yoonseo Park, and Jungyeul Park. 2024. A Novel Alignment-based Approach for PARSEVAL Measuress. Computational Linguistics, 50(3):1181–1190.
- Cite (Informal):
- A Novel Alignment-based Approach for PARSEVAL Measuress (Jo et al., CL 2024)
- PDF:
- https://preview.aclanthology.org/build-pipeline-with-new-library/2024.cl-3.10.pdf