@inproceedings{jang-etal-2024-context,
    title = "Context vs. Human Disagreement in Sarcasm Detection",
    author = "Jang, Hyewon  and
      Jakob, Moritz  and
      Frassinelli, Diego",
    editor = "Ghosh, Debanjan  and
      Muresan, Smaranda  and
      Feldman, Anna  and
      Chakrabarty, Tuhin  and
      Liu, Emmy",
    booktitle = "Proceedings of the 4th Workshop on Figurative Language Processing (FigLang 2024)",
    month = jun,
    year = "2024",
    address = "Mexico City, Mexico (Hybrid)",
    publisher = "Association for Computational Linguistics",
    url = "https://aclanthology.org/2024.figlang-1.1",
    pages = "1--7",
    abstract = "Prior work has highlighted the importance of context in the identification of sarcasm by humans and language models. This work examines how much context is required for a better identification of sarcasm by both parties. We collect textual responses to dialogical prompts and sarcasm judgment to the responses placed after long contexts, short contexts, and no contexts. We find that both for humans and language models, the presence of context is generally important in identifying sarcasm in the response. But increasing the amount of context provides no added benefit to humans (long = short {\textgreater} none). This is the same for language models, but only on easily agreed-upon sentences; for sentences with disagreement among human evaluators, different models show different behavior. We also show how sarcasm detection patterns stay consistent as the amount of context is manipulated despite the low agreement in human evaluation.",
}