@inproceedings{alacam-etal-2025-disentangling,
    title = "Disentangling Subjectivity and Uncertainty for Hate Speech Annotation and Modeling using Gaze",
    author = {Alacam, {\"O}zge  and
      Hoeken, Sanne  and
      S{\"a}uberli, Andreas  and
      Gr{\"o}ner, Hannes  and
      Frassinelli, Diego  and
      Zarrie{\ss}, Sina  and
      Plank, Barbara},
    editor = "Christodoulopoulos, Christos  and
      Chakraborty, Tanmoy  and
      Rose, Carolyn  and
      Peng, Violet",
    booktitle = "Proceedings of the 2025 Conference on Empirical Methods in Natural Language Processing",
    month = nov,
    year = "2025",
    address = "Suzhou, China",
    publisher = "Association for Computational Linguistics",
    url = "https://preview.aclanthology.org/ingest-emnlp/2025.emnlp-main.1460/",
    pages = "28695--28712",
    ISBN = "979-8-89176-332-6",
    abstract = "Variation is inherent in opinion-based annotation tasks like sentiment or hate speech analysis. It does not only arise from errors, fatigue, or sentence ambiguity but also from genuine differences in opinion shaped by background, experience, and culture. In this paper, first, we show how annotators' confidence ratings can be great use for disentangling subjective variation from uncertainty, without relying on specific features present in the data (text, gaze, etc.). Our goal is to establish distinctive dimensions of variation which are often not clearly separated in existing work on modeling annotator variation. We illustrate our approach through a hate speech detection task, demonstrating that models are affected differently by instances of uncertainty and subjectivity. In addition, we show that human gaze patterns offer valuable indicators of subjective evaluation and uncertainty. Disclaimer: This paper contains sentences that may be offensive."
}