SciReviewGen: A Large-scale Dataset for Automatic Literature Review Generation
Tetsu Kasanishi, Masaru Isonuma, Junichiro Mori, Ichiro Sakata
Abstract
Automatic literature review generation is one of the most challenging tasks in natural language processing. Although large language models have tackled literature review generation, the absence of large-scale datasets has been a stumbling block to the progress. We release SciReviewGen, consisting of over 10,000 literature reviews and 690,000 papers cited in the reviews. Based on the dataset, we evaluate recent transformer-based summarization models on the literature review generation task, including Fusion-in-Decoder extended for literature review generation. Human evaluation results show that some machine-generated summaries are comparable to human-written reviews, while revealing the challenges of automatic literature review generation such as hallucinations and a lack of detailed information. Our dataset and code are available at [https://github.com/tetsu9923/SciReviewGen](https://github.com/tetsu9923/SciReviewGen).- Anthology ID:
- 2023.findings-acl.418
- Volume:
- Findings of the Association for Computational Linguistics: ACL 2023
- Month:
- July
- Year:
- 2023
- Address:
- Toronto, Canada
- Editors:
- Anna Rogers, Jordan Boyd-Graber, Naoaki Okazaki
- Venue:
- Findings
- SIG:
- Publisher:
- Association for Computational Linguistics
- Note:
- Pages:
- 6695–6715
- Language:
- URL:
- https://aclanthology.org/2023.findings-acl.418
- DOI:
- 10.18653/v1/2023.findings-acl.418
- Cite (ACL):
- Tetsu Kasanishi, Masaru Isonuma, Junichiro Mori, and Ichiro Sakata. 2023. SciReviewGen: A Large-scale Dataset for Automatic Literature Review Generation. In Findings of the Association for Computational Linguistics: ACL 2023, pages 6695–6715, Toronto, Canada. Association for Computational Linguistics.
- Cite (Informal):
- SciReviewGen: A Large-scale Dataset for Automatic Literature Review Generation (Kasanishi et al., Findings 2023)
- PDF:
- https://preview.aclanthology.org/emnlp-22-attachments/2023.findings-acl.418.pdf