Abstract
We present results from a study investigating how users perceive text quality and readability in extractive and abstractive summaries. We trained two summarisation models on Swedish news data and used these to produce summaries of articles. With the produced summaries, we conducted an online survey in which the extractive summaries were compared to the abstractive summaries in terms of fluency, adequacy and simplicity. We found statistically significant differences in perceived fluency and adequacy between abstractive and extractive summaries but no statistically significant difference in simplicity. Extractive summaries were preferred in most cases, possibly due to the types of errors the summaries tend to have.- Anthology ID:
- 2022.lrec-1.32
- Volume:
- Proceedings of the Thirteenth Language Resources and Evaluation Conference
- Month:
- June
- Year:
- 2022
- Address:
- Marseille, France
- Venue:
- LREC
- SIG:
- Publisher:
- European Language Resources Association
- Note:
- Pages:
- 305–312
- Language:
- URL:
- https://aclanthology.org/2022.lrec-1.32
- DOI:
- Cite (ACL):
- Julius Monsen and Evelina Rennes. 2022. Perceived Text Quality and Readability in Extractive and Abstractive Summaries. In Proceedings of the Thirteenth Language Resources and Evaluation Conference, pages 305–312, Marseille, France. European Language Resources Association.
- Cite (Informal):
- Perceived Text Quality and Readability in Extractive and Abstractive Summaries (Monsen & Rennes, LREC 2022)
- PDF:
- https://preview.aclanthology.org/remove-xml-comments/2022.lrec-1.32.pdf
- Data
- CNN/Daily Mail