Truth Gradient at SemEval-2026 Task 10:Conspiracy Belief Detection via Narrative Density and Mean Pooling

Ekansh Goyal


Abstract
Conspiracy believers use significantly more psycholinguistic markers per post than nonbelievers (Cohen’s d = 0.56, p 10⁻⁸⁰), a pattern we term narrative density, suggesting that belief manifests as structurally denser conspiratorial frames distributed across the full text rather than concentrated in specific lexical cues.We present Truth Gradient’s system for SemEval-2026 Task 10 Subtask 2 (Samory et al., 2026): a DeBERTaV3-large model with mean pooling and a 5-seed probability-averaging ensemble achieving macro F1 = 0.829 on the 77-sample development set and 0.75 on the official test set. The 5-fold CV estimate (0.734 ± 0.007) proves the more reliable predictor of test performance, and we recommend it as standard practice for low-resource shared tasks.Two convergent tests support the narrative density account: masking annotated marker spans drops F1 by 5.3 pp, and direct marker-count fusion recovers +0.9 pp, though we note these are not conclusive given the small dev set. Cross-validated ablation identifies encoder fine-tuning as the dominant design factor (−7.2 pts), and layer-wise probing confirms belief information peaks at mid-stack layers (layer 16/24).
Anthology ID:
2026.semeval-1.424
Volume:
Proceedings of the 20th International Workshop on Semantic Evaluation (2026)
Month:
July
Year:
2026
Address:
San Diego, California, USA
Editors:
Ekaterina Kochmar, Debanjan Ghosh, Kai North, Mamoru Komachi
Venues:
SemEval | WS
SIG:
Publisher:
Association for Computational Linguistics
Note:
Pages:
3422–3431
Language:
URL:
https://preview.aclanthology.org/ingest-acl-workshops/2026.semeval-1.424/
DOI:
Bibkey:
Cite (ACL):
Ekansh Goyal. 2026. Truth Gradient at SemEval-2026 Task 10:Conspiracy Belief Detection via Narrative Density and Mean Pooling. In Proceedings of the 20th International Workshop on Semantic Evaluation (2026), pages 3422–3431, San Diego, California, USA. Association for Computational Linguistics.
Cite (Informal):
Truth Gradient at SemEval-2026 Task 10:Conspiracy Belief Detection via Narrative Density and Mean Pooling (Goyal, SemEval 2026)
Copy Citation:
PDF:
https://preview.aclanthology.org/ingest-acl-workshops/2026.semeval-1.424.pdf
Supplementarymaterial:
 2026.semeval-1.424.SupplementaryMaterial.zip
Supplementarymaterial:
 2026.semeval-1.424.SupplementaryMaterial.zip