ARC: Argument Representation and Coverage Analysis for Zero-Shot Long Document Summarization with Instruction Following LLMs

Mohamed Elaraby, Diane Litman


Abstract
We introduce Argument Representation Coverage (ARC), a bottom-up evaluation framework that assesses how well summaries preserve structured salient arguments, a crucial issue in summarizing high-stakes domains such as law. ARC provides an interpretable lens by distinguishing between different information types to be covered and by separating omissions from factual errors.Using ARC, we evaluate summaries from eight open-weight LLMs in two domains where argument roles are central: long legal opinions and scientific articles. Our results show that while LLMs capture some salient roles, they frequently omit critical information, particularly when arguments are sparsely distributed across the input. Moreover, ARC uncovers systematic patterns—showing how context window positional bias and role-specific preferences shape argument coverage—providing actionable guidance for developing more complete and reliable summarization strategies.
Anthology ID:
2026.eacl-long.167
Volume:
Proceedings of the 19th Conference of the European Chapter of the Association for Computational Linguistics (Volume 1: Long Papers)
Month:
March
Year:
2026
Address:
Rabat, Morocco
Editors:
Vera Demberg, Kentaro Inui, Lluís Marquez
Venue:
EACL
SIG:
Publisher:
Association for Computational Linguistics
Note:
Pages:
3626–3643
Language:
URL:
https://preview.aclanthology.org/ingest-eacl/2026.eacl-long.167/
DOI:
Bibkey:
Cite (ACL):
Mohamed Elaraby and Diane Litman. 2026. ARC: Argument Representation and Coverage Analysis for Zero-Shot Long Document Summarization with Instruction Following LLMs. In Proceedings of the 19th Conference of the European Chapter of the Association for Computational Linguistics (Volume 1: Long Papers), pages 3626–3643, Rabat, Morocco. Association for Computational Linguistics.
Cite (Informal):
ARC: Argument Representation and Coverage Analysis for Zero-Shot Long Document Summarization with Instruction Following LLMs (Elaraby & Litman, EACL 2026)
Copy Citation:
PDF:
https://preview.aclanthology.org/ingest-eacl/2026.eacl-long.167.pdf