Exploring Health Misinformation Detection with Multi-Agent Debate

Chih-Han Chen, Chen-Han Tsai, Yu-Shao Peng


Abstract
Fact-checking health-related claims has become increasingly critical as misinformation proliferates online. Effective verification requires both the retrieval of high-quality evidence and rigorous reasoning processes. In this paper, we propose a two-stage framework for health misinformation detection: Agreement Score Prediction followed by Multi-Agent Debate. In the first stage, we employ large language models (LLMs) to independently evaluate retrieved articles and compute an aggregated agreement score that reflects the overall evidence stance. When this score indicates insufficient consensus—falling below a predefined threshold—the system proceeds to a second stage. Multiple agents engage in structured debate to synthesize conflicting evidence and generate well-reasoned verdicts with explicit justifications. Experimental results demonstrate that our two-stage approach achieves superior performance compared to baseline methods, highlighting the value of combining automated scoring with collaborative reasoning for complex verification tasks.
Anthology ID:
2025.wasp-main.3
Volume:
Proceedings of the Third Workshop for Artificial Intelligence for Scientific Publications
Month:
December
Year:
2025
Address:
Mumbai, India and virtual
Editors:
Alberto Accomazzi, Tirthankar Ghosal, Felix Grezes, Kelly Lockhart
Venues:
WASP | WS
SIG:
Publisher:
Association for Computational Linguistics
Note:
Pages:
16–21
Language:
URL:
https://preview.aclanthology.org/ingest-ijcnlp-aacl/2025.wasp-main.3/
DOI:
Bibkey:
Cite (ACL):
Chih-Han Chen, Chen-Han Tsai, and Yu-Shao Peng. 2025. Exploring Health Misinformation Detection with Multi-Agent Debate. In Proceedings of the Third Workshop for Artificial Intelligence for Scientific Publications, pages 16–21, Mumbai, India and virtual. Association for Computational Linguistics.
Cite (Informal):
Exploring Health Misinformation Detection with Multi-Agent Debate (Chen et al., WASP 2025)
Copy Citation:
PDF:
https://preview.aclanthology.org/ingest-ijcnlp-aacl/2025.wasp-main.3.pdf