Abstract
We present a new method to bootstrap filter Twitter language ID labels in our dataset for automatic language identification (LID). Our method combines geo-location, original Twitter LID labels, and Amazon Mechanical Turk to resolve missing and unreliable labels. We are the first to compare LID classification performance using the MIRA algorithm and langid.py. We show classifier performance on different versions of our dataset with high accuracy using only Twitter data, without ground truth, and very few training examples. We also show how Platt Scaling can be use to calibrate MIRA classifier output values into a probability distribution over candidate classes, making the output more intuitive. Our method allows for fine-grained distinctions between similar languages and dialects and allows us to rediscover the language composition of our Twitter dataset.- Anthology ID:
- W17-1209
- Volume:
- Proceedings of the Fourth Workshop on NLP for Similar Languages, Varieties and Dialects (VarDial)
- Month:
- April
- Year:
- 2017
- Address:
- Valencia, Spain
- Editors:
- Preslav Nakov, Marcos Zampieri, Nikola Ljubešić, Jörg Tiedemann, Shevin Malmasi, Ahmed Ali
- Venue:
- VarDial
- SIG:
- Publisher:
- Association for Computational Linguistics
- Note:
- Pages:
- 73–83
- Language:
- URL:
- https://preview.aclanthology.org/build-pipeline-with-new-library/W17-1209/
- DOI:
- 10.18653/v1/W17-1209
- Cite (ACL):
- Jennifer Williams and Charlie Dagli. 2017. Twitter Language Identification Of Similar Languages And Dialects Without Ground Truth. In Proceedings of the Fourth Workshop on NLP for Similar Languages, Varieties and Dialects (VarDial), pages 73–83, Valencia, Spain. Association for Computational Linguistics.
- Cite (Informal):
- Twitter Language Identification Of Similar Languages And Dialects Without Ground Truth (Williams & Dagli, VarDial 2017)
- PDF:
- https://preview.aclanthology.org/build-pipeline-with-new-library/W17-1209.pdf