Testing the Annotation Consistency of Hallidayan Transitivity Processes: A Multi-variable Structural Approach

Min Dong, Xiaoyan Liu, Alex Chengyu Fang


Abstract
SFL seeks to explain identifiable, observable phenomena of language use in context through the application of a theoretical framework which models language as a functional, meaning making system (Halliday & Matthiessen 2004). Due to the lack of explicit annotation criteria and the divide between conceptual vs. syntactic criteria in practice, it has been a tough job to achieve consistency in the annotation of Hallidayn transitivity processes. The present study proposed that explicit structural and syntactic criteria should be adopted as a basis. Drawing on syntactic and grammatical features as judgement cues, we applied structurally oriented criteria for the annotation of the process categories and participant roles combining a set of interrelated syntactic variables and established the annotation criteria for contextualised circumstantial categories in structural as well as semantic terms. An experiment was carried out to test the usefulness of these annotation criteria, applying percent agreement and Cohen’s kappa as measurements of interrater reliability between the two annotators in each of the five pairs. The results verified our assumptions, albeit rather mildly, and, more significantly, offered some first empirical indications about the practical consistency of transitivity analysis in SFL. In the future work, the research team expect to draw on the insights and experience from some of the ISO standards devoted to semantic annotation such as dialogue acts (Bunt et al. 2012) and semantic roles (ISO-24617-4, 2014).
Anthology ID:
2022.isa-1.7
Volume:
Proceedings of the 18th Joint ACL - ISO Workshop on Interoperable Semantic Annotation within LREC2022
Month:
June
Year:
2022
Address:
Marseille, France
Venue:
ISA
SIG:
Publisher:
European Language Resources Association
Note:
Pages:
53–60
Language:
URL:
https://aclanthology.org/2022.isa-1.7
DOI:
Bibkey:
Cite (ACL):
Min Dong, Xiaoyan Liu, and Alex Chengyu Fang. 2022. Testing the Annotation Consistency of Hallidayan Transitivity Processes: A Multi-variable Structural Approach. In Proceedings of the 18th Joint ACL - ISO Workshop on Interoperable Semantic Annotation within LREC2022, pages 53–60, Marseille, France. European Language Resources Association.
Cite (Informal):
Testing the Annotation Consistency of Hallidayan Transitivity Processes: A Multi-variable Structural Approach (Dong et al., ISA 2022)
Copy Citation:
PDF:
https://preview.aclanthology.org/remove-xml-comments/2022.isa-1.7.pdf