@inproceedings{wagh-srivastava-2025-dentist,
    title = "``The dentist is an involved parent, the bartender is not'': Revealing Implicit Biases in {QA} with Implicit {BBQ}",
    author = "Wagh, Aarushi  and
      Srivastava, Saniya",
    editor = "Akter, Mousumi  and
      Chowdhury, Tahiya  and
      Eger, Steffen  and
      Leiter, Christoph  and
      Opitz, Juri  and
      {\c{C}}ano, Erion",
    booktitle = "Proceedings of the 5th Workshop on Evaluation and Comparison of NLP Systems",
    month = dec,
    year = "2025",
    address = "Mumbai, India",
    publisher = "Association for Computational Linguistics",
    url = "https://preview.aclanthology.org/ingest-ijcnlp-aacl/2025.eval4nlp-1.7/",
    pages = "85--90",
    ISBN = "979-8-89176-305-0",
    abstract = "Existing benchmarks evaluating biases in large language models (LLMs) primarily rely on explicit cues, declaring protected attributes like religion, race, gender by name. However, real-world interactions often contain implicit biases, inferred subtly through names, cultural cues, or traits. This critical oversight creates a significant blind spot in fairness evaluation. We introduce ImplicitBBQ, a benchmark extending the Bias Benchmark for QA (BBQ) with implicitly cued protected attributes across 6 categories. Our evaluation of GPT-4o on ImplicitBBQ illustrates troubling performance disparity from explicit BBQ prompts, with accuracy declining up to 7{\%} in the ``sexual orientation'' subcategory and consistent decline located across most other categories. This indicates that current LLMs contain implicit biases undetected by explicit benchmarks. ImplicitBBQ offers a crucial tool for nuanced fairness evaluation in NLP."
}