Fact Checking Machine Generated Text with Dependency Trees
Alex Estes, Nikhita Vedula, Marcus Collins, Matt Cecil, Oleg Rokhlenko
Abstract
Factual and logical errors made by Natural Language Generation (NLG) systems limit their applicability in many settings. We study this problem in a conversational search and recommendation setting, and observe that we can often make two simplifying assumptions in this domain: (i) there exists a body of structured knowledge we can use for verifying factuality of generated text; and (ii) the text to be factually assessed typically has a well-defined structure and style. Grounded in these assumptions, we propose a fast, unsupervised and explainable technique, DepChecker, that assesses factuality of input text based on rules derived from structured knowledge patterns and dependency relations with respect to the input text. We show that DepChecker outperforms state-of-the-art, general purpose fact-checking techniques in this special, but important case.- Anthology ID:
- 2022.emnlp-industry.46
- Volume:
- Proceedings of the 2022 Conference on Empirical Methods in Natural Language Processing: Industry Track
- Month:
- December
- Year:
- 2022
- Address:
- Abu Dhabi, UAE
- Venue:
- EMNLP
- SIG:
- Publisher:
- Association for Computational Linguistics
- Note:
- Pages:
- 458–466
- Language:
- URL:
- https://aclanthology.org/2022.emnlp-industry.46
- DOI:
- Cite (ACL):
- Alex Estes, Nikhita Vedula, Marcus Collins, Matt Cecil, and Oleg Rokhlenko. 2022. Fact Checking Machine Generated Text with Dependency Trees. In Proceedings of the 2022 Conference on Empirical Methods in Natural Language Processing: Industry Track, pages 458–466, Abu Dhabi, UAE. Association for Computational Linguistics.
- Cite (Informal):
- Fact Checking Machine Generated Text with Dependency Trees (Estes et al., EMNLP 2022)
- PDF:
- https://preview.aclanthology.org/paclic-22-ingestion/2022.emnlp-industry.46.pdf