@article{samir-etal-2025-comparative,
    title = "A Comparative Approach for Auditing Multilingual Phonetic Transcript Archives",
    author = "Samir, Farhan  and
      Ahn, Emily P.  and
      Prakash, Shreya  and
      Soskuthy, M{\'a}rton  and
      Shwartz, Vered  and
      Zhu, Jian",
    journal = "Transactions of the Association for Computational Linguistics",
    volume = "13",
    year = "2025",
    address = "Cambridge, MA",
    publisher = "MIT Press",
    url = "https://preview.aclanthology.org/corrections-2025-07/2025.tacl-1.29/",
    doi = "10.1162/tacl_a_00759",
    pages = "595--612",
    abstract = "Curating datasets that span multiple languages is challenging. To make the collection more scalable, researchers often incorporate one or more imperfect classifiers in the process, like language identification models. These models, however, are prone to failure, resulting in some language partitions being unreliable for downstream tasks. We introduce a statistical test, the Preference Proportion Test, for identifying such unreliable partitions. By annotating only 20 samples for a language partition, we are able to identify systematic transcription errors for 10 language partitions in a recent large multilingual transcribed audio archive, X-IPAPack (Zhu et al., 2024). We find that filtering these low-quality partitions out when training models for the downstream task of phonetic transcription brings substantial benefits, most notably a 25.7{\%} relative improvement on transcribing recordings in out-of-distribution languages. Our work contributes an effective method for auditing multilingual audio archives.1"
}