Building Better: Avoiding Pitfalls in Developing Language Resources when Data is Scarce

Nedjma Ousidhoum, Meriem Beloucif, Saif M. Mohammad


Abstract
Language is a form of symbolic capital that affects people’s lives in many ways (Bourdieu1977,1991). As a powerful means of communication, it reflects identities, cultures, traditions, and societies more broadly. Therefore, data in a given language should be regarded as more than just a collection of tokens. Rigorous data collection and labeling practices are essential for developing more human-centered and socially aware technologies. Although there has been growing interest in under-resourced languages within the NLP community, work in this area faces unique challenges, such as data scarcity and limited access to qualified annotators.In this paper, we collect feedback from individuals directly involved in and impacted by NLP artefacts for medium- and low-resource languages. We conduct both quantitative and qualitative analyses of their responses and highlight key issues related to: (1) data quality, including linguistic and cultural appropriateness; and (2) the ethics of common annotation practices, such as the misuse of participatory research. Based on these findings, we make several recommendations for creating high-quality language artefacts that reflect the cultural milieu of their speakers, while also respecting the dignity and labor of data workers.
Anthology ID:
2025.acl-long.435
Volume:
Proceedings of the 63rd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers)
Month:
July
Year:
2025
Address:
Vienna, Austria
Editors:
Wanxiang Che, Joyce Nabende, Ekaterina Shutova, Mohammad Taher Pilehvar
Venue:
ACL
SIG:
Publisher:
Association for Computational Linguistics
Note:
Pages:
8881–8894
Language:
URL:
https://preview.aclanthology.org/ingestion-acl-25/2025.acl-long.435/
DOI:
Bibkey:
Cite (ACL):
Nedjma Ousidhoum, Meriem Beloucif, and Saif M. Mohammad. 2025. Building Better: Avoiding Pitfalls in Developing Language Resources when Data is Scarce. In Proceedings of the 63rd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 8881–8894, Vienna, Austria. Association for Computational Linguistics.
Cite (Informal):
Building Better: Avoiding Pitfalls in Developing Language Resources when Data is Scarce (Ousidhoum et al., ACL 2025)
Copy Citation:
PDF:
https://preview.aclanthology.org/ingestion-acl-25/2025.acl-long.435.pdf