Abstract
We present an approach to multi-class classification using an encoder-decoder transformer model. We trained a network to identify French varieties using the same scripts we use to train an encoder-decoder machine translation model. With some slight modification to the data preparation and inference parameters, we showed that the same tools used for machine translation can be easily re-used to achieve competitive performance for classification. On the French Dialectal Identification (FDI) task, we scored 32.4 on weighted F1, but this is far from a simple naive bayes classifier that outperforms a neural encoder-decoder model at 41.27 weighted F1.- Anthology ID:
- 2022.vardial-1.9
- Volume:
- Proceedings of the Ninth Workshop on NLP for Similar Languages, Varieties and Dialects
- Month:
- October
- Year:
- 2022
- Address:
- Gyeongju, Republic of Korea
- Editors:
- Yves Scherrer, Tommi Jauhiainen, Nikola Ljubešić, Preslav Nakov, Jörg Tiedemann, Marcos Zampieri
- Venue:
- VarDial
- SIG:
- Publisher:
- Association for Computational Linguistics
- Note:
- Pages:
- 80–85
- Language:
- URL:
- https://aclanthology.org/2022.vardial-1.9
- DOI:
- Cite (ACL):
- Nat Gillin. 2022. Is Encoder-Decoder Transformer the Shiny Hammer?. In Proceedings of the Ninth Workshop on NLP for Similar Languages, Varieties and Dialects, pages 80–85, Gyeongju, Republic of Korea. Association for Computational Linguistics.
- Cite (Informal):
- Is Encoder-Decoder Transformer the Shiny Hammer? (Gillin, VarDial 2022)
- PDF:
- https://preview.aclanthology.org/proper-vol2-ingestion/2022.vardial-1.9.pdf