Abstract
This paper suggests a novel Vietnamese segmentation approach for text categorization. Instead of using an annotated training corpus or a lexicon which are still lacking in Vietnamese, we use both statistical information extracted directly from a commercial search engine and a genetic algorithm to find the optimal routes to segmentation. The extracted information includes document frequency and n-gram mutual information. Our experiment results obtained on the segmentation and categorization of online news abstracts are very promising. It matches near 80 % human judgment on segmentation and over 90 % micro-averaging F1 in categorization. The processing time is less than one second per document when statistical information is cached.- Anthology ID:
- 2006.jeptalnrecital-poster.20
- Volume:
- Actes de la 13ème conférence sur le Traitement Automatique des Langues Naturelles. Posters
- Month:
- April
- Year:
- 2006
- Address:
- Leuven, Belgique
- Editors:
- Piet Mertens, Cédrick Fairon, Anne Dister, Patrick Watrin
- Venue:
- JEP/TALN/RECITAL
- SIG:
- Publisher:
- ATALA
- Note:
- Pages:
- 561–570
- Language:
- URL:
- https://aclanthology.org/2006.jeptalnrecital-poster.20
- DOI:
- Cite (ACL):
- Hung Nguyen Thanh and Khanh Bui Doan. 2006. Word Segmentation for Vietnamese Text Categorization An Internet-based Statistic and Genetic Algorithm Approach. In Actes de la 13ème conférence sur le Traitement Automatique des Langues Naturelles. Posters, pages 561–570, Leuven, Belgique. ATALA.
- Cite (Informal):
- Word Segmentation for Vietnamese Text Categorization An Internet-based Statistic and Genetic Algorithm Approach (Nguyen Thanh & Bui Doan, JEP/TALN/RECITAL 2006)
- PDF:
- https://preview.aclanthology.org/ingest-2024-clasp/2006.jeptalnrecital-poster.20.pdf