@inproceedings{sukhadia-chowdhury-2026-harness,
    title = "{HARNESS}: Lightweight Distilled {A}rabic Speech Foundation Models",
    author = "Sukhadia, Vrunda Nileshkumar  and
      Chowdhury, Shammur Absar",
    editor = "Hosseini-Kivanani, Nina  and
      Brutti, Alessio  and
      Matassoni, Marco  and
      Dowerah, Sandipana  and
      Liga, Davide  and
      Schommer, Christoph",
    booktitle = "Proceedings of Speech Language Models in Low-Resource Settings: Performance, Evaluation, and Bias Analysis ({SPEAKABLE}) @ {LREC} 2026",
    month = may,
    year = "2026",
    address = "Palma, Mallorca (Spain)",
    publisher = "ELRA Language Resources Association (ELRA)",
    url = "https://preview.aclanthology.org/cawl-year/2026.speakable-1.12/",
    doi = "10.63317/2x7bxm8g49ju",
    pages = "109--117",
    abstract = "Large self-supervised speech (SSL) models achieve strong downstream performance, but their size limits deployment in resource-constrained settings. We present HArnESS, an Arabic-centric self-supervised speech model family trained from scratch with iterative self-distillation, together with lightweight student variants that offer strong accuracy-efficiency trade-offs on Automatic Speech Recognition (ASR), Dialect Identification (DID), and Speech Emotion Recognition (SER). Our approach begins with a large bilingual Arabic-English teacher and progressively distills its knowledge into compressed student models while preserving Arabic-relevant acoustic and paralinguistic representations. We further study PCA-based compression of the teacher supervision signal to better match the capacity of shallow and thin students. Compared with HuBERT and XLS-R, HArnESS consistently improves performance on Arabic downstream tasks, while the compressed models remain competitive under substantial structural reduction. These results position HArnESS as a practical and accessible Arabic-centric SSL foundation for real-world speech applications."
}