COMMUNITYNOTES: A Dataset for Exploring the Helpfulness of Fact-Checking Explanations

Rui Xing, Preslav Nakov, Timothy Baldwin, Jey Han Lau


Abstract
Fact-checking on major platforms, such as X, Meta, and TikTok, is shifting from expert-driven verification to a community-based setup, where users contribute explanatory notes to clarify why a post might be misleading. An important challenge here is determining whether an explanation is helpful for understanding real-world claims and the reasons why, which remains largely underexplored in prior research. In practice, most community notes remain unpublished due to slow community annotation, and the reasons for helpfulness lack clear definitions. To bridge these gaps, we introduce the task of predicting both the helpfulness of explanatory notes and the reason for this. We present COMMUNTYNOTES, a large-scale multilingual dataset of 104k posts with user-provided notes and helpfulness labels. We further propose a framework that automatically generates and improves reason definitions via automatic prompt optimization, and integrate them into prediction. Our experiments show that the optimized definitions can improve both helpfulness and reason prediction. Finally, we show that the helpfulness information is beneficial for existing fact-checking systems. The code and the data are available at https://github.com/ruixing76/Helpfulness-FCExp.
Anthology ID:
2026.findings-eacl.71
Volume:
Findings of the Association for Computational Linguistics: EACL 2026
Month:
March
Year:
2026
Address:
Rabat, Morocco
Editors:
Vera Demberg, Kentaro Inui, Lluís Marquez
Venue:
Findings
SIG:
Publisher:
Association for Computational Linguistics
Note:
Pages:
1390–1411
Language:
URL:
https://preview.aclanthology.org/ingest-eacl/2026.findings-eacl.71/
DOI:
Bibkey:
Cite (ACL):
Rui Xing, Preslav Nakov, Timothy Baldwin, and Jey Han Lau. 2026. COMMUNITYNOTES: A Dataset for Exploring the Helpfulness of Fact-Checking Explanations. In Findings of the Association for Computational Linguistics: EACL 2026, pages 1390–1411, Rabat, Morocco. Association for Computational Linguistics.
Cite (Informal):
COMMUNITYNOTES: A Dataset for Exploring the Helpfulness of Fact-Checking Explanations (Xing et al., Findings 2026)
Copy Citation:
PDF:
https://preview.aclanthology.org/ingest-eacl/2026.findings-eacl.71.pdf
Checklist:
 2026.findings-eacl.71.checklist.pdf