Łukasz Dębowski
2016
Upper Bound of Entropy Rate Revisited —A New Extrapolation of Compressed Large-Scale Corpora—
Ryosuke Takahira
|
Kumiko Tanaka-Ishii
|
Łukasz Dębowski
Proceedings of the Workshop on Computational Linguistics for Linguistic Complexity (CL4LC)
The article presents results of entropy rate estimation for human languages across six languages by using large, state-of-the-art corpora of up to 7.8 gigabytes. To obtain the estimates for data length tending to infinity, we use an extrapolation function given by an ansatz. Whereas some ansatzes of this kind were proposed in previous research papers, here we introduce a stretched exponential extrapolation function that has a smaller error of fit. In this way, we uncover a possibility that the entropy rates of human languages are positive but 20% smaller than previously reported.
2004
A Search Tool for Corpora with Positional Tagsets and Ambiguities
Adam Przepiórkowski
|
Zygmunt Krynicki
|
Łukasz Dębowski
|
Marcin Woliński
|
Daniel Janus
|
Piotr Bański
Proceedings of the Fourth International Conference on Language Resources and Evaluation (LREC’04)
2003
A Reconfigurable Stochastic Tagger for Languages with Complex Tag Structure
Łukasz Dębowski
Proceedings of the 2003 EACL Workshop on Morphological Processing of Slavic Languages
Search