Abstract
We propose a method for identifying monolingual textual segments in multilingual documents. It requires only a minimal number of linguistic resources – word lists and monolingual corpora – and can therefore be adapted to many under-resourced languages. Taking these languages into account when processing multilingual documents in NLP tools is important as it can contribute to the creation of essential textual resources. This language identification task – code switching detection being its most complex form – can also provide added value to various existing data or tools. Our research demonstrates that a language identification module performing well on short texts can be used to efficiently analyse a document through a sliding window. The results obtained for code switching identification – between 87.29% and 97.97% accuracy – are state-of-the-art, which is confirmed by the benchmarks performed on the few available systems that have been used on our test data.- Anthology ID:
- 2022.sigul-1.15
- Volume:
- Proceedings of the 1st Annual Meeting of the ELRA/ISCA Special Interest Group on Under-Resourced Languages
- Month:
- June
- Year:
- 2022
- Address:
- Marseille, France
- Editors:
- Maite Melero, Sakriani Sakti, Claudia Soria
- Venue:
- SIGUL
- SIG:
- SIGUL
- Publisher:
- European Language Resources Association
- Note:
- Pages:
- 112–121
- Language:
- URL:
- https://aclanthology.org/2022.sigul-1.15
- DOI:
- Cite (ACL):
- Laurent Kevers. 2022. CoSwID, a Code Switching Identification Method Suitable for Under-Resourced Languages. In Proceedings of the 1st Annual Meeting of the ELRA/ISCA Special Interest Group on Under-Resourced Languages, pages 112–121, Marseille, France. European Language Resources Association.
- Cite (Informal):
- CoSwID, a Code Switching Identification Method Suitable for Under-Resourced Languages (Kevers, SIGUL 2022)
- PDF:
- https://preview.aclanthology.org/nschneid-patch-5/2022.sigul-1.15.pdf