@inproceedings{hewett-stede-2025-disagreements,
    title = "Disagreements in analyses of rhetorical text structure: A new dataset and first analyses",
    author = "Hewett, Freya  and
      Stede, Manfred",
    editor = "Peng, Siyao  and
      Rehbein, Ines",
    booktitle = "Proceedings of the 19th Linguistic Annotation Workshop (LAW-XIX-2025)",
    month = jul,
    year = "2025",
    address = "Vienna, Austria",
    publisher = "Association for Computational Linguistics",
    url = "https://preview.aclanthology.org/display_plenaries/2025.law-1.3/",
    pages = "35--47",
    ISBN = "979-8-89176-262-6",
    abstract = "Discourse structure annotation is known to involve a high level of subjectivity, which often results in low inter-annotator agreement. In this paper, we focus on ``legitimate disagreements'', by which we refer to multiple valid annotations for a text or text segment. We provide a new dataset of English and German texts, where each text comes with two parallel analyses (both done by well-trained annotators) in the framework of Rhetorical Structure Theory. Using the RST Tace tool, we build a list of all conflicting annotation decisions and present some statistics for the corpus. Thereafter, we undertake a qualitative analysis of the disagreements and propose a typology of underlying reasons. From this we derive the need to differentiate two kinds of ambiguities in RST annotation: those that result from inherent ``everyday'' linguistic ambiguity, and those that arise from specifications in the theory and/or the annotation schemes."
}