Sentence boundary detection: a comparison of paradigms for improving MT quality
Daniel J. Walker, David E. Clements, Maki Darwin, Jan W. Amtrup
Abstract
The reliable detection of sentence boundaries in running text is one of the first important steps in preparing an input document for translation. Although this is often neglected, it is necessary to obtain a translation with a high degree of quality. In this paper, we present a comparison of different paradigms for the detection of sentence boundaries in written text. We compare three different approaches: Directly encoding the knowledge in a program, a rule-based system relying on regular expressions to describe boundaries, and a statistical maximum-entropy learning algorithm to obtain knowledge about boundaries. Using the statistical system, we obtain a recall of 98.14%, classifying boundaries of six types, and using a training corpus of under 10,000 sentences.- Anthology ID:
- 2001.mtsummit-papers.66
- Volume:
- Proceedings of Machine Translation Summit VIII
- Month:
- September 18-22
- Year:
- 2001
- Address:
- Santiago de Compostela, Spain
- Editor:
- Bente Maegaard
- Venue:
- MTSummit
- SIG:
- Publisher:
- Note:
- Pages:
- Language:
- URL:
- https://aclanthology.org/2001.mtsummit-papers.66
- DOI:
- Cite (ACL):
- Daniel J. Walker, David E. Clements, Maki Darwin, and Jan W. Amtrup. 2001. Sentence boundary detection: a comparison of paradigms for improving MT quality. In Proceedings of Machine Translation Summit VIII, Santiago de Compostela, Spain.
- Cite (Informal):
- Sentence boundary detection: a comparison of paradigms for improving MT quality (Walker et al., MTSummit 2001)
- PDF:
- https://preview.aclanthology.org/nschneid-patch-3/2001.mtsummit-papers.66.pdf