On the Rigour of Scientific Writing: Criteria, Analysis, and Insights

Joseph James, Chenghao Xiao, Yucheng Li, Chenghua Lin

Abstract
Rigour is crucial for scientific research as it ensures the reproducibility and validity of results and findings. Despite its importance, little work exists on modelling rigour computationally, and there is a lack of analysis on whether these criteria can effectively signal or measure the rigour of scientific papers in practice. In this paper, we introduce a bottom-up, data-driven framework to automatically identify and define rigour criteria and assess their relevance in scientific writing. Our framework includes rigour keyword extraction, detailed rigour definition generation, and salient criteria identification. Furthermore, our framework is domain-agnostic and can be tailored to the evaluation of scientific rigour for different areas, accommodating the distinct salient criteria across fields. We conducted comprehensive experiments based on datasets collected from different domains (e.g. ICLR, ACL) to demonstrate the effectiveness of our framework in modelling rigour. In addition, we analyse linguist patterns of rigour, revealing that framing certainty is crucial for enhancing the perception of scientific rigour, while suggestion certainty and probability uncertainty diminish it.
Anthology ID:
2024.findings-emnlp.380
Volume:
Findings of the Association for Computational Linguistics: EMNLP 2024
Month:
November
Year:
2024
Address:
Miami, Florida, USA
Editors:
Yaser Al-Onaizan, Mohit Bansal, Yun-Nung Chen
Venue:
Findings
SIG:
Publisher:
Association for Computational Linguistics
Note:
Pages:
6523–6538
Language:
URL:
https://preview.aclanthology.org/icon-24-ingestion/2024.findings-emnlp.380/
DOI:
10.18653/v1/2024.findings-emnlp.380
Bibkey:
Cite (ACL):
Joseph James, Chenghao Xiao, Yucheng Li, and Chenghua Lin. 2024. On the Rigour of Scientific Writing: Criteria, Analysis, and Insights. In Findings of the Association for Computational Linguistics: EMNLP 2024, pages 6523–6538, Miami, Florida, USA. Association for Computational Linguistics.
Cite (Informal):
On the Rigour of Scientific Writing: Criteria, Analysis, and Insights (James et al., Findings 2024)
Copy Citation:
PDF:
https://preview.aclanthology.org/icon-24-ingestion/2024.findings-emnlp.380.pdf
PDF Search Fix data