@inproceedings{cyberey-etal-2025-prevalent,
    title = "Do Prevalent Bias Metrics Capture Allocational Harms from {LLM}s?",
    author = "Cyberey, Hannah  and
      Ji, Yangfeng  and
      Evans, David",
    editor = "Drozd, Aleksandr  and
      Sedoc, Jo{\~a}o  and
      Tafreshi, Shabnam  and
      Akula, Arjun  and
      Shu, Raphael",
    booktitle = "The Sixth Workshop on Insights from Negative Results in NLP",
    month = may,
    year = "2025",
    address = "Albuquerque, New Mexico",
    publisher = "Association for Computational Linguistics",
    url = "https://preview.aclanthology.org/fix-sig-urls/2025.insights-1.5/",
    pages = "34--45",
    ISBN = "979-8-89176-240-4",
    abstract = "Allocational harms occur when resources or opportunities are unfairly withheld from specific groups. Many proposed bias measures ignore the discrepancy between predictions, which are what the proposed methods consider, and decisions that are made as a result of those predictions. Our work examines the reliability of current bias metrics in assessing allocational harms arising from predictions of large language models (LLMs). We evaluate their predictive validity and utility for model selection across ten LLMs and two allocation tasks. Our results reveal that commonly-used bias metrics based on average performance gap and distribution distance fail to reliably capture group disparities in allocation outcomes. Our work highlights the need to account for how model predictions are used in decisions, in particular in contexts where they are influenced by how limited resources are allocated."
}