An Analysis of Tasks and Datasets in Peer Reviewing
Moritz Staudinger, Wojciech Kusa, Florina Piroi, Allan Hanbury
Abstract
Taking note of the current challenges of the peer review system, this paper inventories the research tasks for analysing and possibly automating parts of the reviewing process, like matching submissions with a reviewer’s domain of expertise. For each of these tasks we list their associated datasets, analysing their quality in terms of available documentation of creation and use. Building up on this, we give a set of recommendations to take into account when collecting and releasing data.- Anthology ID:
- 2024.sdp-1.24
- Volume:
- Proceedings of the Fourth Workshop on Scholarly Document Processing (SDP 2024)
- Month:
- August
- Year:
- 2024
- Address:
- Bangkok, Thailand
- Editors:
- Tirthankar Ghosal, Amanpreet Singh, Anita Waard, Philipp Mayr, Aakanksha Naik, Orion Weller, Yoonjoo Lee, Shannon Shen, Yanxia Qin
- Venues:
- sdp | WS
- SIG:
- Publisher:
- Association for Computational Linguistics
- Note:
- Pages:
- 257–268
- Language:
- URL:
- https://aclanthology.org/2024.sdp-1.24
- DOI:
- Cite (ACL):
- Moritz Staudinger, Wojciech Kusa, Florina Piroi, and Allan Hanbury. 2024. An Analysis of Tasks and Datasets in Peer Reviewing. In Proceedings of the Fourth Workshop on Scholarly Document Processing (SDP 2024), pages 257–268, Bangkok, Thailand. Association for Computational Linguistics.
- Cite (Informal):
- An Analysis of Tasks and Datasets in Peer Reviewing (Staudinger et al., sdp-WS 2024)
- PDF:
- https://preview.aclanthology.org/nschneid-patch-4/2024.sdp-1.24.pdf