@inproceedings{ruiz-fernandez-etal-2026-esbbq,
    title = "{E}s{BBQ} and {C}a{BBQ}: The {S}panish and {C}atalan Bias Benchmarks for Question Answering",
    author = "Ruiz-Fern{\'a}ndez, Valle  and
      Mina, Mario  and
      Falc{\~a}o, J{\'u}lia  and
      Vasquez Reina, Luis Antonio  and
      Salles, Anna  and
      Gonzalez-Agirre, Aitor  and
      Perez-de-Vi{\~n}aspre, Olatz",
    editor = "Piperidis, Stelios  and
      Bel, N{\'u}ria  and
      van den Heuvel, Henk  and
      Ide, Nancy  and
      Krek, Simon  and
      Toral, Antonio",
    booktitle = "Proceedings of the Fifteenth Language Resources and Evaluation Conference",
    month = may,
    year = "2026",
    address = "Palma de Mallorca, Spain",
    publisher = "ELRA Language Resource Association",
    url = "https://preview.aclanthology.org/test-year-match/2026.lrec-1.309/",
    doi = "10.63317/2u47873noowf",
    pages = "3884--3907",
    abstract = "Previous literature has largely shown that Large Language Models (LLMs) perpetuate social biases learnt from their pre-training data. Given the notable lack of resources for social bias evaluation in languages other than English, and for social contexts outside of the United States, this paper introduces the Spanish and the Catalan Bias Benchmarks for Question Answering (EsBBQ and CaBBQ). Based on the original BBQ, these two parallel datasets are designed to assess social bias across 10 categories using a multiple-choice QA setting, now adapted to the Spanish and Catalan languages and to the social context of Spain. We report evaluation results on different LLMs, factoring in model family, size and variant. Our results show that models tend to fail to choose the correct answer in ambiguous scenarios, and that high QA accuracy often correlates with greater reliance on social biases."
}