@inproceedings{dent-etal-2025-identifying,
    title = "Identifying Rare Languages in {C}ommon {C}rawl Data is a Needles-in-a-Haystack Problem",
    author = "Dent, Rasul  and
      Ortiz Suarez, Pedro  and
      Cl{\'e}rice, Thibault  and
      Sagot, Beno{\^i}t",
    editor = "Christodoulopoulos, Christos  and
      Chakraborty, Tanmoy  and
      Rose, Carolyn  and
      Peng, Violet",
    booktitle = "Findings of the Association for Computational Linguistics: EMNLP 2025",
    month = nov,
    year = "2025",
    address = "Suzhou, China",
    publisher = "Association for Computational Linguistics",
    url = "https://preview.aclanthology.org/author-page-yu-wang-polytechnic/2025.findings-emnlp.77/",
    doi = "10.18653/v1/2025.findings-emnlp.77",
    pages = "1460--1473",
    ISBN = "979-8-89176-335-7",
    abstract = "Automatic language identification is frequentlyframed as a multi-class classification problem.However, when creating digital corpora forless commonly written languages, it may bemore appropriate to consider it a data min-ing problem. For these varieties, one knowsahead of time that the vast majority of doc-uments are of little interest. By minimizingresources spent on classifying such documents,we can create corpora covering previously over-looked languages faster than existing pipelines.To demonstrate the effectiveness of the tar-geted mining perspective, we introduce a newpipeline that can filter a single snapshot in twohours. We also provide web corpora for severalFrench-based Creoles."
}