@inproceedings{kamruzzaman-etal-2025-banstereoset,
    title = "{B}an{S}tereo{S}et: A Dataset to Measure Stereotypical Social Biases in {LLM}s for {B}angla",
    author = "Kamruzzaman, Mahammed  and
      Monsur, Abdullah Al  and
      Das, Shrabon Kumar  and
      Hassan, Enamul  and
      Kim, Gene Louis",
    editor = "Che, Wanxiang  and
      Nabende, Joyce  and
      Shutova, Ekaterina  and
      Pilehvar, Mohammad Taher",
    booktitle = "Findings of the Association for Computational Linguistics: ACL 2025",
    month = jul,
    year = "2025",
    address = "Vienna, Austria",
    publisher = "Association for Computational Linguistics",
    url = "https://preview.aclanthology.org/ingest-nlpsi/2025.findings-acl.179/",
    doi = "10.18653/v1/2025.findings-acl.179",
    pages = "3450--3460",
    ISBN = "979-8-89176-256-5",
    abstract = "This study presents \textit{\textbf{BanStereoSet}}, a dataset designed to evaluate stereotypical social biases in multilingual LLMs for the Bangla language. In an effort to extend the focus of bias research beyond English-centric datasets, we have localized the content from the StereoSet, IndiBias, and kamruzzaman-etal{'}s datasets, producing a resource tailored to capture biases prevalent within the Bangla-speaking community. Our BanStereoSet dataset consists of 1,194 sentences spanning 9 categories of bias: race, profession, gender, ageism, beauty, beauty in profession, region, caste, and religion. This dataset not only serves as a crucial tool for measuring bias in multilingual LLMs but also facilitates the exploration of stereotypical bias across different social categories, potentially guiding the development of more equitable language technologies in \textit{Bangladeshi} contexts. Our analysis of several language models using this dataset indicates significant biases, reinforcing the necessity for culturally and linguistically adapted datasets to develop more equitable language technologies."
}