Decide less, communicate more: On the construct validity of end-to-end fact-checking in medicine

Sebastian Antony Joseph, Lily Chen, Barry Wei, Michael Mackert, Iain James Marshall, Paul Pu Liang, Ramez Kouzy, Byron C Wallace, Junyi Jessy Li


Abstract
Technological progress has led to concrete advancements in tasks that were regarded as challenging, such as automatic fact-checking. Interest in adopting these systems for public health and medicine has grown due to the high-stakes nature of medical decisions and challenges in critically appraising a vast and diverse medical literature. Evidence-based medicine connects to every individual, and yet the nature of it is highly technical, rendering the medical literacy of majority users inadequate to sufficiently navigate the domain. Such problems with medical communication ripen the ground for end-to-end fact-checking agents: check a claim against current medical literature and return with an evidence-backed verdict. And yet, such systems remain largely unused.In this position paper, developed with expert input, we present the first study examining how clinical experts verify real claims from social media by synthesizing medical evidence. In searching for this upper-bound, we reveal fundamental challenges in end-to-end fact-checking when applied to medicine: Difficulties connecting claims in the wild to scientific evidence in the form of clinical trials; ambiguities in underspecified claims mixed with mismatched intentions; and inherently subjective veracity labels. We argue that fact-checking should be approached as an interactive communication problem, rather than an end-to-end process.
Anthology ID:
2026.findings-acl.496
Volume:
Findings of the Association for Computational Linguistics: ACL 2026
Month:
July
Year:
2026
Address:
San Diego, California, United States
Editors:
Maria Liakata, Viviane P. Moreira, Jiajun Zhang, David Jurgens
Venue:
Findings
SIG:
Publisher:
Association for Computational Linguistics
Note:
Pages:
10196–10221
Language:
URL:
https://preview.aclanthology.org/ingest-acl/2026.findings-acl.496/
DOI:
Bibkey:
Cite (ACL):
Sebastian Antony Joseph, Lily Chen, Barry Wei, Michael Mackert, Iain James Marshall, Paul Pu Liang, Ramez Kouzy, Byron C Wallace, and Junyi Jessy Li. 2026. Decide less, communicate more: On the construct validity of end-to-end fact-checking in medicine. In Findings of the Association for Computational Linguistics: ACL 2026, pages 10196–10221, San Diego, California, United States. Association for Computational Linguistics.
Cite (Informal):
Decide less, communicate more: On the construct validity of end-to-end fact-checking in medicine (Joseph et al., Findings 2026)
Copy Citation:
PDF:
https://preview.aclanthology.org/ingest-acl/2026.findings-acl.496.pdf
Checklist:
 2026.findings-acl.496.checklist.pdf