Abstract
This paper investigates the use of character n-gram frequencies for identifying complex words in English, German and Spanish texts. The approach is based on the assumption that complex words are likely to contain different character sequences than simple words. The multinomial Naive Bayes classifier was used with n-grams of different lengths as features, and the best results were obtained for the combination of 2-grams and 4-grams. This variant was submitted to the Complex Word Identification Shared Task 2018 for all texts and achieved F-scores between 70% and 83%. The system was ranked in the middle range for all English texts, as third of fourteen submissions for German, and as tenth of seventeen submissions for Spanish. The method is not very convenient for the cross-language task, achieving only 59% on the French text.- Anthology ID:
- W18-0541
- Volume:
- Proceedings of the Thirteenth Workshop on Innovative Use of NLP for Building Educational Applications
- Month:
- June
- Year:
- 2018
- Address:
- New Orleans, Louisiana
- Editors:
- Joel Tetreault, Jill Burstein, Ekaterina Kochmar, Claudia Leacock, Helen Yannakoudakis
- Venue:
- BEA
- SIG:
- SIGEDU
- Publisher:
- Association for Computational Linguistics
- Note:
- Pages:
- 341–348
- Language:
- URL:
- https://aclanthology.org/W18-0541
- DOI:
- 10.18653/v1/W18-0541
- Cite (ACL):
- Maja Popović. 2018. Complex Word Identification Using Character n-grams. In Proceedings of the Thirteenth Workshop on Innovative Use of NLP for Building Educational Applications, pages 341–348, New Orleans, Louisiana. Association for Computational Linguistics.
- Cite (Informal):
- Complex Word Identification Using Character n-grams (Popović, BEA 2018)
- PDF:
- https://preview.aclanthology.org/emnlp22-frontmatter/W18-0541.pdf