Creating Data in Icelandic for Text Normalization
Helga Svala Sigurðardóttir, Anna Björk Nikulásdóttir, Jón Guðnason
Abstract
There is no natural way to acquire normalized data so we try to create good enough data to attempt more advanced methods for text normalization. We manually annotated the first normalized corpus in Icelandic, 40,000 sentences, and developed Regína, a rule-based system for text normalization. Regína gets 90.83% accuracy compared to the manually annotated corpus on non-standard words. Regína showed a significant improvement in accuracy when compared to an older normalization system for Icelandic. The normalized corpus and Regína will be released as open source.- Anthology ID:
- 2021.nodalida-main.45
- Volume:
- Proceedings of the 23rd Nordic Conference on Computational Linguistics (NoDaLiDa)
- Month:
- May 31--2 June
- Year:
- 2021
- Address:
- Reykjavik, Iceland (Online)
- Editors:
- Simon Dobnik, Lilja Øvrelid
- Venue:
- NoDaLiDa
- SIG:
- Publisher:
- Linköping University Electronic Press, Sweden
- Note:
- Pages:
- 404–412
- Language:
- URL:
- https://aclanthology.org/2021.nodalida-main.45
- DOI:
- Cite (ACL):
- Helga Svala Sigurðardóttir, Anna Björk Nikulásdóttir, and Jón Guðnason. 2021. Creating Data in Icelandic for Text Normalization. In Proceedings of the 23rd Nordic Conference on Computational Linguistics (NoDaLiDa), pages 404–412, Reykjavik, Iceland (Online). Linköping University Electronic Press, Sweden.
- Cite (Informal):
- Creating Data in Icelandic for Text Normalization (Sigurðardóttir et al., NoDaLiDa 2021)
- PDF:
- https://preview.aclanthology.org/fix-dup-bibkey/2021.nodalida-main.45.pdf