2023
pdf
abs
Hallucination Mitigation in Natural Language Generation from Large-Scale Open-Domain Knowledge Graphs
Xiao Shi
|
Zhengyuan Zhu
|
Zeyu Zhang
|
Chengkai Li
Proceedings of the 2023 Conference on Empirical Methods in Natural Language Processing
In generating natural language descriptions for knowledge graph triples, prior works used either small-scale, human-annotated datasets or datasets with limited variety of graph shapes, e.g., those having mostly star graphs. Graph-to-text models trained and evaluated on such datasets are largely not assessed for more realistic large-scale, open-domain settings. We introduce a new dataset, GraphNarrative, to fill this gap. Fine-tuning transformer-based pre-trained language models has achieved state-of-the-art performance among graph-to-text models. However, this method suffers from information hallucination—the generated text may contain fabricated facts not present in input graphs. We propose a novel approach that, given a graph-sentence pair in GraphNarrative, trims the sentence to eliminate portions that are not present in the corresponding graph, by utilizing the sentence’s dependency parse tree. Our experiment results verify this approach using models trained on GraphNarrative and existing datasets. The dataset, source code, and trained models are released at https://github.com/idirlab/graphnarrator.
2021
pdf
abs
A Dashboard for Mitigating the COVID-19 Misinfodemic
Zhengyuan Zhu
|
Kevin Meng
|
Josue Caraballo
|
Israa Jaradat
|
Xiao Shi
|
Zeyu Zhang
|
Farahnaz Akrami
|
Haojin Liao
|
Fatma Arslan
|
Damian Jimenez
|
Mohanmmed Samiul Saeef
|
Paras Pathak
|
Chengkai Li
Proceedings of the 16th Conference of the European Chapter of the Association for Computational Linguistics: System Demonstrations
This paper describes the current milestones achieved in our ongoing project that aims to understand the surveillance of, impact of and intervention on COVID-19 misinfodemic on Twitter. Specifically, it introduces a public dashboard which, in addition to displaying case counts in an interactive map and a navigational panel, also provides some unique features not found in other places. Particularly, the dashboard uses a curated catalog of COVID-19 related facts and debunks of misinformation, and it displays the most prevalent information from the catalog among Twitter users in user-selected U.S. geographic regions. The paper explains how to use BERT models to match tweets with the facts and misinformation and to detect their stance towards such information. The paper also discusses the results of preliminary experiments on analyzing the spatio-temporal spread of misinformation.
2020
pdf
abs
Jennifer for COVID-19: An NLP-Powered Chatbot Built for the People and by the People to Combat Misinformation
Yunyao Li
|
Tyrone Grandison
|
Patricia Silveyra
|
Ali Douraghy
|
Xinyu Guan
|
Thomas Kieselbach
|
Chengkai Li
|
Haiqi Zhang
Proceedings of the 1st Workshop on NLP for COVID-19 at ACL 2020
Just as SARS-CoV-2, a new form of coronavirus continues to infect a growing number of people around the world, harmful misinformation about the outbreak also continues to spread. With the goal of combating misinformation, we designed and built Jennifer–a chatbot maintained by a global group of volunteers. With Jennifer, we hope to learn whether public information from reputable sources could be more effectively organized and shared in the wake of a crisis as well as to understand issues that the public were most immediately curious about. In this paper, we introduce Jennifer and describe the design of this proof-of-principle system. We also present lessons learned and discuss open challenges. Finally, to facilitate future research, we release COVID-19 Question Bank, a dataset of 3,924 COVID-19-related questions in 944 groups, gathered from our users and volunteers.
pdf
abs
Modeling Factual Claims with Semantic Frames
Fatma Arslan
|
Josue Caraballo
|
Damian Jimenez
|
Chengkai Li
Proceedings of the Twelfth Language Resources and Evaluation Conference
In this paper, we introduce an extension of the Berkeley FrameNet for the structured and semantic modeling of factual claims. Modeling is a robust tool that can be leveraged in many different tasks such as matching claims to existing fact-checks and translating claims to structured queries. Our work introduces 11 new manually crafted frames along with 9 existing FrameNet frames, all of which have been selected with fact-checking in mind. Along with these frames, we are also providing 2,540 fully annotated sentences, which can be used to understand how these frames are intended to work and to train machine learning models. Finally, we are also releasing our annotation tool to facilitate other researchers to make their own local extensions to FrameNet.
2019
pdf
abs
ClaimPortal: Integrated Monitoring, Searching, Checking, and Analytics of Factual Claims on Twitter
Sarthak Majithia
|
Fatma Arslan
|
Sumeet Lubal
|
Damian Jimenez
|
Priyank Arora
|
Josue Caraballo
|
Chengkai Li
Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics: System Demonstrations
We present ClaimPortal, a web-based platform for monitoring, searching, checking, and analyzing English factual claims on Twitter from the American political domain. We explain the architecture of ClaimPortal, its components and functions, and the user interface. While the last several years have witnessed a substantial growth in interests and efforts in the area of computational fact-checking, ClaimPortal is a novel infrastructure in that fact-checkers have largely skipped factual claims in tweets. It can be a highly powerful tool to both general web users and fact-checkers. It will also be an educational resource in helping cultivate a society that is less susceptible to falsehoods.