Murja Sani Gadanya
2026
HaYo: Repurposing DiaSafety Dataset for Dialogue Safety Evaluation in Hausa and Yoruba
Tunde Oluwaseyi Ajayi | Bolade Deborah Ashaolu | Falalu Ibrahim Lawan | Daud Olamide Abolade | Amina Imam Abubakar | Oluwatosin Ayomide Akinrinde | Murja Sani Gadanya | Omodolapo Dorcas Ashaolu | Abubakar Khalid Auwal | Adewumi Awujoola | Shamsuddeen Umaru Adamu | Israel Olawole Ashaolu | Mihael Arcan | Paul Buitelaar
Proceedings of Resources for African Indigenous Languages (RAIL) 2026 @ LREC 2026
Tunde Oluwaseyi Ajayi | Bolade Deborah Ashaolu | Falalu Ibrahim Lawan | Daud Olamide Abolade | Amina Imam Abubakar | Oluwatosin Ayomide Akinrinde | Murja Sani Gadanya | Omodolapo Dorcas Ashaolu | Abubakar Khalid Auwal | Adewumi Awujoola | Shamsuddeen Umaru Adamu | Israel Olawole Ashaolu | Mihael Arcan | Paul Buitelaar
Proceedings of Resources for African Indigenous Languages (RAIL) 2026 @ LREC 2026
Research efforts aimed at detecting unsafe dialogues have resulted in creation of benchmark datasets and models for evaluation. The benchmarks mostly exist in English and other high resourced languages. In order to address the challenge of unavailability of dialogue safety evaluation dataset in Hausa and Yorùbá, we repurporse DiaSafety dataset to develop HaYo dataset, by providing contextualised human annotation of dialogues in DiaSafety. We provide dialogues in Hausa and Yorùbá, obtained by human translation of dialogues in the DiaSafety dataset, to raters who are native speakers. The dialogues are annotated as Unsafe or Safe. We evaluate seven models with moderation, conversational or multilingual capabilities in terms of F1 Score. Using McNemar test, we observe that the predictions of GPT-4.1 and Gemma-3-12b-it on HaYo are statistically significant at p < 0.05. In our evaluation with instructions in English, we observe lower F1 scores in six out of the seven models, comparing the performance on DiaSafety and HaYo labels. The model predictions were inconsistent with the labels in the HaYo dataset when instructions and dialogues were provided in Hausa and Yorùbá. Compared to providing instructions in English, the issues range from responses in unspecified languages to underperformance in terms of F1 score. We plan to release the HaYo dataset to the public to promote dialogue safety research, especially in under-resourced languages.
2025
BRIGHTER: BRIdging the Gap in Human-Annotated Textual Emotion Recognition Datasets for 28 Languages
Shamsuddeen Hassan Muhammad | Nedjma Ousidhoum | Idris Abdulmumin | Jan Philip Wahle | Terry Ruas | Meriem Beloucif | Christine de Kock | Nirmal Surange | Daniela Teodorescu | Ibrahim Said Ahmad | David Ifeoluwa Adelani | Alham Fikri Aji | Felermino D. M. A. Ali | Ilseyar Alimova | Vladimir Araujo | Nikolay Babakov | Naomi Baes | Ana-Maria Bucur | Andiswa Bukula | Guanqun Cao | Rodrigo Tufiño | Rendi Chevi | Chiamaka Ijeoma Chukwuneke | Alexandra Ciobotaru | Daryna Dementieva | Murja Sani Gadanya | Robert Geislinger | Bela Gipp | Oumaima Hourrane | Oana Ignat | Falalu Ibrahim Lawan | Rooweither Mabuya | Rahmad Mahendra | Vukosi Marivate | Alexander Panchenko | Andrew Piper | Charles Henrique Porto Ferreira | Vitaly Protasov | Samuel Rutunda | Manish Shrivastava | Aura Cristina Udrea | Lilian Diana Awuor Wanzare | Sophie Wu | Florian Valentin Wunderlich | Hanif Muhammad Zhafran | Tianhui Zhang | Yi Zhou | Saif M. Mohammad
Proceedings of the 63rd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers)
Shamsuddeen Hassan Muhammad | Nedjma Ousidhoum | Idris Abdulmumin | Jan Philip Wahle | Terry Ruas | Meriem Beloucif | Christine de Kock | Nirmal Surange | Daniela Teodorescu | Ibrahim Said Ahmad | David Ifeoluwa Adelani | Alham Fikri Aji | Felermino D. M. A. Ali | Ilseyar Alimova | Vladimir Araujo | Nikolay Babakov | Naomi Baes | Ana-Maria Bucur | Andiswa Bukula | Guanqun Cao | Rodrigo Tufiño | Rendi Chevi | Chiamaka Ijeoma Chukwuneke | Alexandra Ciobotaru | Daryna Dementieva | Murja Sani Gadanya | Robert Geislinger | Bela Gipp | Oumaima Hourrane | Oana Ignat | Falalu Ibrahim Lawan | Rooweither Mabuya | Rahmad Mahendra | Vukosi Marivate | Alexander Panchenko | Andrew Piper | Charles Henrique Porto Ferreira | Vitaly Protasov | Samuel Rutunda | Manish Shrivastava | Aura Cristina Udrea | Lilian Diana Awuor Wanzare | Sophie Wu | Florian Valentin Wunderlich | Hanif Muhammad Zhafran | Tianhui Zhang | Yi Zhou | Saif M. Mohammad
Proceedings of the 63rd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers)
People worldwide use language in subtle and complex ways to express emotions. Although emotion recognition–an umbrella term for several NLP tasks–impacts various applications within NLP and beyond, most work in this area has focused on high-resource languages. This has led to significant disparities in research efforts and proposed solutions, particularly for under-resourced languages, which often lack high-quality annotated datasets.In this paper, we present BRIGHTER–a collection of multi-labeled, emotion-annotated datasets in 28 different languages and across several domains. BRIGHTER primarily covers low-resource languages from Africa, Asia, Eastern Europe, and Latin America, with instances labeled by fluent speakers. We highlight the challenges related to the data collection and annotation processes, and then report experimental results for monolingual and crosslingual multi-label emotion identification, as well as emotion intensity recognition. We analyse the variability in performance across languages and text domains, both with and without the use of LLMs, and show that the BRIGHTER datasets represent a meaningful step towards addressing the gap in text-based emotion recognition.
Search
Fix author
Co-authors
- Falalu Ibrahim Lawan 2
- Idris Abdulmumin 1
- Daud Olamide Abolade 1
- Amina Imam Abubakar 1
- Shamsuddeen Umaru Adamu 1
- David Ifeoluwa Adelani 1
- Ibrahim Said Ahmad 1
- Tunde Oluwaseyi Ajayi 1
- Alham Fikri Aji 1
- Oluwatosin Ayomide Akinrinde 1
- Felermino D. M. A. Ali 1
- Ilseyar Alimova 1
- Vladimir Araujo 1
- Mihael Arcan 1
- Bolade Deborah Ashaolu 1
- Israel Olawole Ashaolu 1
- Omodolapo Dorcas Ashaolu 1
- Abubakar Khalid Auwal 1
- Adewumi Awujoola 1
- Nikolay Babakov 1
- Naomi Baes 1
- Meriem Beloucif 1
- Ana-Maria Bucur 1
- Paul Buitelaar 1
- Andiswa Bukula 1
- Guanqun Cao 1
- Rendi Chevi 1
- Chiamaka Ijeoma Chukwuneke 1
- Alexandra Ciobotaru 1
- Daryna Dementieva 1
- Charles Henrique Porto Ferreira 1
- Robert Geislinger 1
- Bela Gipp 1
- Oumaima Hourrane 1
- Oana Ignat 1
- Rooweither Mabuya 1
- Rahmad Mahendra 1
- Vukosi Marivate 1
- Saif Mohammad 1
- Shamsuddeen Hassan Muhammad 1
- Nedjma Ousidhoum 1
- Alexander Panchenko 1
- Andrew Piper 1
- Vitaly Protasov 1
- Terry Ruas 1
- Samuel Rutunda 1
- Manish Shrivastava 1
- Nirmal Surange 1
- Daniela Teodorescu 1
- Rodrigo Tufiño 1
- Aura Cristina Udrea 1
- Jan Philip Wahle 1
- Lilian Diana Awuor Wanzare 1
- Sophie Wu 1
- Florian Valentin Wunderlich 1
- Hanif Muhammad Zhafran 1
- Tianhui Zhang 1
- Yi Zhou 1
- Christine de Kock 1