Enhancing and Evaluating the Grammatical Framework Approach to Logic-to-Text Generation
Eduardo Calò, Elze van der Werf, Albert Gatt, Kees van Deemter
Abstract
Logic-to-text generation is an important yet underrepresented area of natural language generation (NLG). In particular, most previous works on this topic lack sound evaluation. We address this limitation by building and evaluating a system that generates high-quality English text given a first-order logic (FOL) formula as input. We start by analyzing the performance of Ranta (2011)’s system. Based on this analysis, we develop an extended version of the system, which we name LoLa, that performs formula simplification based on logical equivalences and syntactic transformations. We carry out an extensive evaluation of LoLa using standard automatic metrics and human evaluation. We compare the results against a baseline and Ranta (2011)’s system. The results show that LoLa outperforms the other two systems in most aspects.- Anthology ID:
- 2022.gem-1.13
- Volume:
- Proceedings of the 2nd Workshop on Natural Language Generation, Evaluation, and Metrics (GEM)
- Month:
- December
- Year:
- 2022
- Address:
- Abu Dhabi, United Arab Emirates (Hybrid)
- Editors:
- Antoine Bosselut, Khyathi Chandu, Kaustubh Dhole, Varun Gangal, Sebastian Gehrmann, Yacine Jernite, Jekaterina Novikova, Laura Perez-Beltrachini
- Venue:
- GEM
- SIG:
- SIGGEN
- Publisher:
- Association for Computational Linguistics
- Note:
- Pages:
- 148–171
- Language:
- URL:
- https://aclanthology.org/2022.gem-1.13
- DOI:
- 10.18653/v1/2022.gem-1.13
- Cite (ACL):
- Eduardo Calò, Elze van der Werf, Albert Gatt, and Kees van Deemter. 2022. Enhancing and Evaluating the Grammatical Framework Approach to Logic-to-Text Generation. In Proceedings of the 2nd Workshop on Natural Language Generation, Evaluation, and Metrics (GEM), pages 148–171, Abu Dhabi, United Arab Emirates (Hybrid). Association for Computational Linguistics.
- Cite (Informal):
- Enhancing and Evaluating the Grammatical Framework Approach to Logic-to-Text Generation (Calò et al., GEM 2022)
- PDF:
- https://preview.aclanthology.org/improve-issue-templates/2022.gem-1.13.pdf