AI, Take the Wheel: What Drives Delegation and Trust in Human–Computer Cooperative Question Answering?

Maharshi Gor, Yoo Yeon Sung, Yu Hou, Eve Fleisig, Zhu Irene Ying, Tianyi Zhou, Jordan Lee Boyd-Graber


Abstract
AI systems are fallible, and humans can make mistakes in deciding whether to trustAI over their own judgment. Thus, improving human-AI collaboration requires that we understand when,why, and how humans decide to rely on AI. We study two reliance decisions: delegating a task toAI without seeing its output (whether AI is used) and evaluating AI suggestions to decidewhether to adopt them how AI output shapes final decisions).Both matter for effective collaboration, yet prior work lacks naturalistic experiments capturing both patternsfor the same users. We address this gap by studying collaborative human–AI teams competing in aquestion-answering game in which humans can choose when and how to work with AI agents to win.Our 24 matches pair 23 expert humans with 16 AI agents, capturing 387 delegation and 1440 adoption decisions.While human–AI collaboration performs better than either AI or humansalone, humans make suboptimal collaboration decisions, bothunder-relying on correct AI suggestions (3.7% of opportunities missed) and over-relying when AI misleads them (1.5%).Both parties contribute wrong answers: reported model confidence is near chance when humans and AI disagree, while confirmation bias drives higher under-reliance (60.7%) when an AI suggestion agrees with humans’ initial incorrect answer.
Anthology ID:
2026.findings-acl.422
Volume:
Findings of the Association for Computational Linguistics: ACL 2026
Month:
July
Year:
2026
Address:
San Diego, California, United States
Editors:
Maria Liakata, Viviane P. Moreira, Jiajun Zhang, David Jurgens
Venue:
Findings
SIG:
Publisher:
Association for Computational Linguistics
Note:
Pages:
8664–8689
Language:
URL:
https://preview.aclanthology.org/ingest-acl/2026.findings-acl.422/
DOI:
Bibkey:
Cite (ACL):
Maharshi Gor, Yoo Yeon Sung, Yu Hou, Eve Fleisig, Zhu Irene Ying, Tianyi Zhou, and Jordan Lee Boyd-Graber. 2026. AI, Take the Wheel: What Drives Delegation and Trust in Human–Computer Cooperative Question Answering?. In Findings of the Association for Computational Linguistics: ACL 2026, pages 8664–8689, San Diego, California, United States. Association for Computational Linguistics.
Cite (Informal):
AI, Take the Wheel: What Drives Delegation and Trust in Human–Computer Cooperative Question Answering? (Gor et al., Findings 2026)
Copy Citation:
PDF:
https://preview.aclanthology.org/ingest-acl/2026.findings-acl.422.pdf
Checklist:
 2026.findings-acl.422.checklist.pdf