@inproceedings{yang-etal-2022-gpt,
    title = "What {GPT} Knows About Who is Who",
    author = "Yang, Xiaohan  and
      Peynetti, Eduardo  and
      Meerman, Vasco  and
      Tanner, Chris",
    editor = "Tafreshi, Shabnam  and
      Sedoc, Jo{\~a}o  and
      Rogers, Anna  and
      Drozd, Aleksandr  and
      Rumshisky, Anna  and
      Akula, Arjun",
    booktitle = "Proceedings of the Third Workshop on Insights from Negative Results in NLP",
    month = may,
    year = "2022",
    address = "Dublin, Ireland",
    publisher = "Association for Computational Linguistics",
    url = "https://preview.aclanthology.org/fix-sig-urls/2022.insights-1.10/",
    doi = "10.18653/v1/2022.insights-1.10",
    pages = "75--81",
    abstract = "Coreference resolution {--} which is a crucial task for understanding discourse and language at large {--} has yet to witness widespread benefits from large language models (LLMs). Moreover, coreference resolution systems largely rely on supervised labels, which are highly expensive and difficult to annotate, thus making it ripe for prompt engineering. In this paper, we introduce a QA-based prompt-engineering method and discern \textit{generative}, pre-trained LLMs' abilities and limitations toward the task of coreference resolution. Our experiments show that GPT-2 and GPT-Neo can return valid answers, but that their capabilities to identify coreferent mentions are limited and prompt-sensitive, leading to inconsistent results."
}