Representativeness as a Forgotten Lesson for Multilingual and Code-switched Data Collection and Preparation

A. Seza Doğruöz, Sunayana Sitaram, Zheng Xin Yong

Abstract
Multilingualism is widespread around the world and code-switching (CSW) is a common practice among different language pairs/tuples across locations and regions. However, there is still not much progress in building successful CSW systems, despite the recent advances in Massive Multilingual Language Models (MMLMs). We investigate the reasons behind this setback through a critical study about the existing CSW data sets (68) across language pairs in terms of the collection and preparation (e.g. transcription and annotation) stages. This in-depth analysis reveals that a) most CSW data involves English ignoring other language pairs/tuples b) there are flaws in terms of representativeness in data collection and preparation stages due to ignoring the location based, socio-demographic and register variation in CSW. In addition, lack of clarity on the data selection and filtering stages shadow the representativeness of CSW data sets. We conclude by providing a short check-list to improve the representativeness for forthcoming studies involving CSW data collection and preparation.
Anthology ID:
2023.findings-emnlp.382
Volume:
Findings of the Association for Computational Linguistics: EMNLP 2023
Month:
December
Year:
2023
Address:
Singapore
Editors:
Houda Bouamor, Juan Pino, Kalika Bali
Venue:
Findings
SIG:
Publisher:
Association for Computational Linguistics
Note:
Pages:
5751–5767
Language:
URL:
https://preview.aclanthology.org/build-pipeline-with-new-library/2023.findings-emnlp.382/
DOI:
10.18653/v1/2023.findings-emnlp.382
Bibkey:
Cite (ACL):
A. Seza Doğruöz, Sunayana Sitaram, and Zheng Xin Yong. 2023. Representativeness as a Forgotten Lesson for Multilingual and Code-switched Data Collection and Preparation. In Findings of the Association for Computational Linguistics: EMNLP 2023, pages 5751–5767, Singapore. Association for Computational Linguistics.
Cite (Informal):
Representativeness as a Forgotten Lesson for Multilingual and Code-switched Data Collection and Preparation (Doğruöz et al., Findings 2023)
Copy Citation:
PDF:
https://preview.aclanthology.org/build-pipeline-with-new-library/2023.findings-emnlp.382.pdf
PDF Search Fix metadata