Who’s the Author? How Explanations Impact User Reliance in AI-Assisted Authorship Attribution

Calvin Bao, Connor Baumler, Hal Daumé Iii, Marine Carpuat


Abstract
Despite growing interest in explainable NLP, it remains unclear how explanation strategies shape user behavior in tasks like authorship identification, where relevant textual features may be difficult for lay users to pinpoint. To support their analysis of text style, we consider two explanation types: example-based style rewrites and feature-based rationales, generated using a LLM-based pipeline. We measured how explanations impact user behavior in a controlled study (n=95) where participants completed authorship identification tasks with our types of assistance. While no explanation type improved overall task accuracy, fine-grained reliance patterns (CITATION) revealed that rewrites supported appropriate reliance, whereas presenting both explanation types increased AI overreliance, minimizing participant self-reliance. We find that participants exhibiting better reliance behaviors had focused explanation needs, contrasting with the diffused preferences of those who overrelied on AI, or incorrectly self-relied. These findings highlight the need for adaptive explanation systems that tailor support based on specific user reliance behaviors.
Anthology ID:
2025.findings-emnlp.1380
Volume:
Findings of the Association for Computational Linguistics: EMNLP 2025
Month:
November
Year:
2025
Address:
Suzhou, China
Editors:
Christos Christodoulopoulos, Tanmoy Chakraborty, Carolyn Rose, Violet Peng
Venue:
Findings
SIG:
Publisher:
Association for Computational Linguistics
Note:
Pages:
25312–25330
Language:
URL:
https://preview.aclanthology.org/author-page-yu-wang-polytechnic/2025.findings-emnlp.1380/
DOI:
10.18653/v1/2025.findings-emnlp.1380
Bibkey:
Cite (ACL):
Calvin Bao, Connor Baumler, Hal Daumé Iii, and Marine Carpuat. 2025. Who’s the Author? How Explanations Impact User Reliance in AI-Assisted Authorship Attribution. In Findings of the Association for Computational Linguistics: EMNLP 2025, pages 25312–25330, Suzhou, China. Association for Computational Linguistics.
Cite (Informal):
Who’s the Author? How Explanations Impact User Reliance in AI-Assisted Authorship Attribution (Bao et al., Findings 2025)
Copy Citation:
PDF:
https://preview.aclanthology.org/author-page-yu-wang-polytechnic/2025.findings-emnlp.1380.pdf
Checklist:
 2025.findings-emnlp.1380.checklist.pdf