Abstract
In this paper, we present AnCast, an intuitive and efficient tool for evaluating graph-based meaning representations (MR). AnCast implements evaluation metrics that are well understood in the NLP community, and they include concept F1, unlabeled relation F1, labeled relation F1, and weighted relation F1. The efficiency of the tool comes from a novel anchor broadcast alignment algorithm that is not subject to the trappings of local maxima. We show through experimental results that the AnCast score is highly correlated with the widely used Smatch score, but its computation takes only about 40% the time.- Anthology ID:
- 2024.lrec-main.94
- Volume:
- Proceedings of the 2024 Joint International Conference on Computational Linguistics, Language Resources and Evaluation (LREC-COLING 2024)
- Month:
- May
- Year:
- 2024
- Address:
- Torino, Italia
- Editors:
- Nicoletta Calzolari, Min-Yen Kan, Veronique Hoste, Alessandro Lenci, Sakriani Sakti, Nianwen Xue
- Venues:
- LREC | COLING
- SIG:
- Publisher:
- ELRA and ICCL
- Note:
- Pages:
- 1052–1062
- Language:
- URL:
- https://aclanthology.org/2024.lrec-main.94
- DOI:
- Cite (ACL):
- Haibo Sun and Nianwen Xue. 2024. Anchor and Broadcast: An Efficient Concept Alignment Approach for Evaluation of Semantic Graphs. In Proceedings of the 2024 Joint International Conference on Computational Linguistics, Language Resources and Evaluation (LREC-COLING 2024), pages 1052–1062, Torino, Italia. ELRA and ICCL.
- Cite (Informal):
- Anchor and Broadcast: An Efficient Concept Alignment Approach for Evaluation of Semantic Graphs (Sun & Xue, LREC-COLING 2024)
- PDF:
- https://preview.aclanthology.org/nschneid-patch-4/2024.lrec-main.94.pdf