Hybrid Intelligence for Logical Fallacy Detection

Mariia Kutepova, Khalid Al Khatib


Abstract
This study investigates the impact of Hybrid Intelligence (HI) on improving the detection of logical fallacies, addressing the pressing challenge of misinformation prevalent across communication platforms. Employing a between-subjects experimental design, the research compares the performance of two groups: one relying exclusively on human judgment and another supported by an AI assistant. Participants evaluated a series of statements, with the AI-assisted group utilizing a custom ChatGPT-based chatbot that provided real-time hints and clarifications. The findings reveal a significant improvement in fallacy detection with AI support, increasing from an F1-score of 0.76 in the human-only group to 0.90 in the AI-assisted group. Despite this enhancement, both groups struggled to accurately identify non-fallacious statements, highlighting the need to further refine how AI assistance is leveraged.
Anthology ID:
2025.hcinlp-1.16
Volume:
Proceedings of the Fourth Workshop on Bridging Human-Computer Interaction and Natural Language Processing (HCI+NLP)
Month:
November
Year:
2025
Address:
Suzhou, China
Editors:
Su Lin Blodgett, Amanda Cercas Curry, Sunipa Dev, Siyan Li, Michael Madaio, Jack Wang, Sherry Tongshuang Wu, Ziang Xiao, Diyi Yang
Venues:
HCINLP | WS
SIG:
Publisher:
Association for Computational Linguistics
Note:
Pages:
197–208
Language:
URL:
https://preview.aclanthology.org/author-page-yu-wang-polytechnic/2025.hcinlp-1.16/
DOI:
10.18653/v1/2025.hcinlp-1.16
Bibkey:
Cite (ACL):
Mariia Kutepova and Khalid Al Khatib. 2025. Hybrid Intelligence for Logical Fallacy Detection. In Proceedings of the Fourth Workshop on Bridging Human-Computer Interaction and Natural Language Processing (HCI+NLP), pages 197–208, Suzhou, China. Association for Computational Linguistics.
Cite (Informal):
Hybrid Intelligence for Logical Fallacy Detection (Kutepova & Khatib, HCINLP 2025)
Copy Citation:
PDF:
https://preview.aclanthology.org/author-page-yu-wang-polytechnic/2025.hcinlp-1.16.pdf