|
||||||||||
| PREV CLASS NEXT CLASS | FRAMES NO FRAMES | |||||||||
| SUMMARY: NESTED | FIELD | CONSTR | METHOD | DETAIL: FIELD | CONSTR | METHOD | |||||||||
java.lang.Objectszte.csd.Tokenizer
public class Tokenizer
Tokenizer is an utility class for task-dependent tokenization and lemmatization.
| Constructor Summary | |
|---|---|
Tokenizer()
|
|
Tokenizer(edu.stanford.nlp.process.DocumentPreprocessor preproc)
|
|
| Method Summary | |
|---|---|
void |
lemmatiseOn()
|
void |
serialize()
|
java.util.List<Sentence> |
tokenise(java.lang.String text)
|
| Methods inherited from class java.lang.Object |
|---|
equals, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait |
| Constructor Detail |
|---|
public Tokenizer()
public Tokenizer(edu.stanford.nlp.process.DocumentPreprocessor preproc)
| Method Detail |
|---|
public void lemmatiseOn()
public java.util.List<Sentence> tokenise(java.lang.String text)
public void serialize()
|
||||||||||
| PREV CLASS NEXT CLASS | FRAMES NO FRAMES | |||||||||
| SUMMARY: NESTED | FIELD | CONSTR | METHOD | DETAIL: FIELD | CONSTR | METHOD | |||||||||