Safiyyah Saleem


2025

pdf bib
BOUQuET : dataset, Benchmark and Open initiative for Universal Quality Evaluation in Translation
Pierre Andrews | Mikel Artetxe | Mariano Coria Meglioli | Marta R. Costa-jussà | Joe Chuang | David Dale | Mark Duppenthaler | Nathanial Paul Ekberg | Cynthia Gao | Daniel Edward Licht | Jean Maillard | Alexandre Mourachko | Christophe Ropers | Safiyyah Saleem | Eduardo Sánchez | Ioannis Tsiamas | Arina Turkatenko | Albert Ventayol-Boada | Shireen Yates
Proceedings of the 2025 Conference on Empirical Methods in Natural Language Processing

BOUQuET is a multi-way, multicentric and multi-register/domain dataset and benchmark, and a broader collaborative initiative. This dataset is handcrafted in 8 non-English languages (i.e. Egyptian Arabic and Modern Standard Arabic, French, German, Hindi, Indonesian, Mandarin Chinese, Russian, and Spanish). Each of these source languages are representative of the most widely spoken ones and therefore they have the potential to serve as pivot languages that will enable more accurate translations. The dataset is multicentric to enforce representation of multilingual language features. In addition, the dataset goes beyond the sentence level, as it is organized in paragraphs of various lengths. Compared with related machine translation datasets, we show that BOUQuET has a broader representation of domains while simplifying the translation task for non-experts. Therefore, BOUQuET is specially suitable for crowd-source extension for which we are launching a call aim-ing at collecting a multi-way parallel corpus covering any written language. The dataset is freely available at https://huggingface.co/datasets/facebook/bouquet.

2022

pdf bib
Findings of the WMT’22 Shared Task on Large-Scale Machine Translation Evaluation for African Languages
David Ifeoluwa Adelani | Md Mahfuz Ibn Alam | Antonios Anastasopoulos | Akshita Bhagia | Marta R. Costa-jussà | Jesse Dodge | Fahim Faisal | Christian Federmann | Natalia Fedorova | Francisco Guzmán | Sergey Koshelev | Jean Maillard | Vukosi Marivate | Jonathan Mbuya | Alexandre Mourachko | Safiyyah Saleem | Holger Schwenk | Guillaume Wenzek
Proceedings of the Seventh Conference on Machine Translation (WMT)

We present the results of the WMT’22 SharedTask on Large-Scale Machine Translation Evaluation for African Languages. The shared taskincluded both a data and a systems track, alongwith additional innovations, such as a focus onAfrican languages and extensive human evaluation of submitted systems. We received 14system submissions from 8 teams, as well as6 data track contributions. We report a largeprogress in the quality of translation for Africanlanguages since the last iteration of this sharedtask: there is an increase of about 7.5 BLEUpoints across 72 language pairs, and the average BLEU scores went from 15.09 to 22.60.

2010

pdf bib
Argument Optionality in the LinGO Grammar Matrix
Safiyyah Saleem | Emily M. Bender
Coling 2010: Posters

pdf bib
Grammar Prototyping and Testing with the LinGO Grammar Matrix Customization System
Emily M. Bender | Scott Drellishak | Antske Fokkens | Michael Wayne Goodman | Daniel P. Mills | Laurie Poulson | Safiyyah Saleem
Proceedings of the ACL 2010 System Demonstrations