A Tale of Two Revisions: Summarizing Changes Across Document Versions

Santosh T.y.s.s, Natwar Modani, Apoorv Saxena

Abstract
Document revision is a crucial aspect of the writing process, particularly in collaborative environments where multiple authors contribute simultaneously. However, current tools lack an efficient way to provide a comprehensive overview of changes between versions, leading to difficulties in understanding revisions. To address this, we propose a novel task of providing thematic summary of changes between document versions, organizing individual edits based on shared themes. We assess capabilities of LLMs on this task and further introduce three strategies to tackle this task: (i) representing the input of two documents along with edits in the ‘diff’ format (ii) a two-stage task decomposition with individual edit description generation as an intermediate task and (iii) clustering based chunking and subsequent merging techniques for handling longer documents. Our experiments demonstrate the effectiveness of our approach in improving the model’s capacity to handle this complex task. Additionally, we introduce ChangeSumm, a curated dataset comprising human-written thematic summaries for pairs of document versions, to facilitate evaluation and further research in this direction.
Anthology ID:
2024.findings-acl.190
Volume:
Findings of the Association for Computational Linguistics ACL 2024
Month:
August
Year:
2024
Address:
Bangkok, Thailand and virtual meeting
Editors:
Lun-Wei Ku, Andre Martins, Vivek Srikumar
Venue:
Findings
SIG:
Publisher:
Association for Computational Linguistics
Note:
Pages:
3195–3211
Language:
URL:
https://aclanthology.org/2024.findings-acl.190
DOI:
10.18653/v1/2024.findings-acl.190
Bibkey:
Cite (ACL):
Santosh T.y.s.s, Natwar Modani, and Apoorv Saxena. 2024. A Tale of Two Revisions: Summarizing Changes Across Document Versions. In Findings of the Association for Computational Linguistics ACL 2024, pages 3195–3211, Bangkok, Thailand and virtual meeting. Association for Computational Linguistics.
Cite (Informal):
A Tale of Two Revisions: Summarizing Changes Across Document Versions (T.y.s.s et al., Findings 2024)
Copy Citation:
PDF:
https://preview.aclanthology.org/nschneid-patch-5/2024.findings-acl.190.pdf
PDF Search