@inproceedings{wu-ebling-2024-investigating,
    title = "Investigating Ableism in {LLM}s through Multi-turn Conversation",
    author = "Wu, Guojun  and
      Ebling, Sarah",
    editor = "Dementieva, Daryna  and
      Ignat, Oana  and
      Jin, Zhijing  and
      Mihalcea, Rada  and
      Piatti, Giorgio  and
      Tetreault, Joel  and
      Wilson, Steven  and
      Zhao, Jieyu",
    booktitle = "Proceedings of the Third Workshop on NLP for Positive Impact",
    month = nov,
    year = "2024",
    address = "Miami, Florida, USA",
    publisher = "Association for Computational Linguistics",
    url = "https://preview.aclanthology.org/ingest-emnlp/2024.nlp4pi-1.18/",
    doi = "10.18653/v1/2024.nlp4pi-1.18",
    pages = "202--210",
    abstract = "To reveal ableism (i.e., bias against persons with disabilities) in large language models (LLMs), we introduce a novel approach involving multi-turn conversations, enabling a comparative assessment. Initially, we prompt the LLM to elaborate short biographies, followed by a request to incorporate information about a disability. Finally, we employ several methods to identify the top words that distinguish the disability-integrated biographies from those without. This comparative setting helps us uncover how LLMs handle disability-related information and reveal underlying biases. We observe that LLMs tend to highlight disabilities in a manner that can be perceived as patronizing or as implying that overcoming challenges is unexpected due to the disability."
}