Abstract
Despite peer-reviewing being an essential component of academia since the 1600s, it has repeatedly received criticisms for lack of transparency and consistency. We posit that recent work in machine learning and explainable AI provide tools that enable insights into the decisions from a given peer-review process. We start by simulating the peer-review process using an ML classifier and extracting global explanations in the form of linguistic features that affect the acceptance of a scientific paper for publication on an open peer-review dataset. Second, since such global explanations do not justify causal interpretations, we propose a methodology for detecting confounding effects in natural language and generating explanations, disentangled from textual confounders, in the form of lexicons. Our proposed linguistic explanation methodology indicates the following on a case dataset of ICLR submissions: a) the organising committee follows, for the most part, the recommendations of reviewers, and b) the paper’s main characteristics that led to reviewers recommending acceptance for publication are originality, clarity and substance.- Anthology ID:
- 2021.cinlp-1.4
- Volume:
- Proceedings of the First Workshop on Causal Inference and NLP
- Month:
- November
- Year:
- 2021
- Address:
- Punta Cana, Dominican Republic
- Editors:
- Amir Feder, Katherine Keith, Emaad Manzoor, Reid Pryzant, Dhanya Sridhar, Zach Wood-Doughty, Jacob Eisenstein, Justin Grimmer, Roi Reichart, Molly Roberts, Uri Shalit, Brandon Stewart, Victor Veitch, Diyi Yang
- Venue:
- CINLP
- SIG:
- Publisher:
- Association for Computational Linguistics
- Note:
- Pages:
- 44–60
- Language:
- URL:
- https://aclanthology.org/2021.cinlp-1.4
- DOI:
- 10.18653/v1/2021.cinlp-1.4
- Cite (ACL):
- Panagiotis Fytas, Georgios Rizos, and Lucia Specia. 2021. What Makes a Scientific Paper be Accepted for Publication?. In Proceedings of the First Workshop on Causal Inference and NLP, pages 44–60, Punta Cana, Dominican Republic. Association for Computational Linguistics.
- Cite (Informal):
- What Makes a Scientific Paper be Accepted for Publication? (Fytas et al., CINLP 2021)
- PDF:
- https://preview.aclanthology.org/emnlp22-frontmatter/2021.cinlp-1.4.pdf
- Data
- PeerRead