|
||||||||||
PREV CLASS NEXT CLASS | FRAMES NO FRAMES | |||||||||
SUMMARY: NESTED | FIELD | CONSTR | METHOD | DETAIL: FIELD | CONSTR | METHOD |
java.lang.Objectszte.csd.Tokenizer
public class Tokenizer
Tokenizer is an utility class for task-dependent tokenization and lemmatization.
Constructor Summary | |
---|---|
Tokenizer()
|
|
Tokenizer(edu.stanford.nlp.process.DocumentPreprocessor preproc)
|
Method Summary | |
---|---|
void |
lemmatiseOn()
|
void |
serialize()
|
java.util.List<Sentence> |
tokenise(java.lang.String text)
|
Methods inherited from class java.lang.Object |
---|
equals, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait |
Constructor Detail |
---|
public Tokenizer()
public Tokenizer(edu.stanford.nlp.process.DocumentPreprocessor preproc)
Method Detail |
---|
public void lemmatiseOn()
public java.util.List<Sentence> tokenise(java.lang.String text)
public void serialize()
|
||||||||||
PREV CLASS NEXT CLASS | FRAMES NO FRAMES | |||||||||
SUMMARY: NESTED | FIELD | CONSTR | METHOD | DETAIL: FIELD | CONSTR | METHOD |