@inproceedings{sancheti-rudinger-2025-tracking,
    title = "Tracking Evolving Relationship Between Characters in Books in the Era of Large Language Models",
    author = "Sancheti, Abhilasha  and
      Rudinger, Rachel",
    editor = "Clark, Elizabeth  and
      Lal, Yash Kumar  and
      Chaturvedi, Snigdha  and
      Iyyer, Mohit  and
      Brei, Anneliese  and
      Modi, Ashutosh  and
      Chandu, Khyathi Raghavi",
    booktitle = "Proceedings of the The 7th Workshop on Narrative Understanding",
    month = may,
    year = "2025",
    address = "Albuquerque, New Mexico",
    publisher = "Association for Computational Linguistics",
    url = "https://preview.aclanthology.org/ingest-emnlp/2025.wnu-1.12/",
    doi = "10.18653/v1/2025.wnu-1.12",
    pages = "64--82",
    ISBN = "979-8-89176-247-3",
    abstract = "This work aims to assess the zero-shot social reasoning capabilities of LLMs by proposing various strategies based on the granularity of information used to track the fine-grained evolution in the relationship between characters in a book. Without gold annotations, we thoroughly analyze the agreements between predictions from multiple LLMs and manually examine their consensus at a local and global level via the task of trope prediction. Our findings reveal low-to-moderate agreement among LLMs and humans, reflecting the complexity of the task. Analysis shows that LLMs are sensitive to subtle contextual changes and often rely on surface-level cues. Humans, too, may interpret relationships differently, leading to disagreements in annotations."
}