Parser agreement and disagreement in L2 Korean UD: Implications for human-in-the-loop annotation

Hakyung Sung, Gyu-Ho Shin


Abstract
We propose a simplified human-in-the-loop workflow for second language (L2) Korean morphosyntactic annotation by leveraging agreement between two domain-adapted parsers. We first evaluate whether parser agreement can serve as a proxy for annotation correctness by comparing it with independent human judgments. The results show strong correspondence between parser and human judgments, supporting the feasibility of semi-automatic L2-Korean UD annotation. Further analysis demonstrates that parser disagreements cluster in linguistically predictable domains such as grammatical-relation distinctions and clause-boundary ambiguity. While many disagreement cases are tractable for iterative model refinement, others reflect deeper representational challenges inherent in parsing and tagging L2-Korean corpora.
Anthology ID:
2026.law-main.2
Volume:
Proceedings of the 20th Linguistic Annotation Workshop (LAW XX)
Month:
July
Year:
2026
Address:
San Diego, California, USA
Editors:
Yang Janet Liu, Luke Gessler
Venues:
LAW | WS
SIG:
Publisher:
Association for Computational Linguistics
Note:
Pages:
12–21
Language:
URL:
https://preview.aclanthology.org/ingest-acl-workshops/2026.law-main.2/
DOI:
Bibkey:
Cite (ACL):
Hakyung Sung and Gyu-Ho Shin. 2026. Parser agreement and disagreement in L2 Korean UD: Implications for human-in-the-loop annotation. In Proceedings of the 20th Linguistic Annotation Workshop (LAW XX), pages 12–21, San Diego, California, USA. Association for Computational Linguistics.
Cite (Informal):
Parser agreement and disagreement in L2 Korean UD: Implications for human-in-the-loop annotation (Sung & Shin, LAW 2026)
Copy Citation:
PDF:
https://preview.aclanthology.org/ingest-acl-workshops/2026.law-main.2.pdf