Discosuite - A parser test suite for German discontinuous structures

Wolfgang Maier, Miriam Kaeshammer, Peter Baumann, Sandra Kübler

Abstract
Parser evaluation traditionally relies on evaluation metrics which deliver a single aggregate score over all sentences in the parser output, such as PARSEVAL. However, for the evaluation of parser performance concerning a particular phenomenon, a test suite of sentences is needed in which this phenomenon has been identified. In recent years, the parsing of discontinuous structures has received a rising interest. Therefore, in this paper, we present a test suite for testing the performance of dependency and constituency parsers on non-projective dependencies and discontinuous constituents for German. The test suite is based on the newly released TIGER treebank version 2.2. It provides a unique possibility of benchmarking parsers on non-local syntactic relationships in German, for constituents and dependencies. We include a linguistic analysis of the phenomena that cause discontinuity in the TIGER annotation, thereby closing gaps in previous literature. The linguistic phenomena we investigate include extraposition, a placeholder/repeated element construction, topicalization, scrambling, local movement, parentheticals, and fronting of pronouns.
Anthology ID:
L14-1222
Volume:
Proceedings of the Ninth International Conference on Language Resources and Evaluation (LREC'14)
Month:
May
Year:
2014
Address:
Reykjavik, Iceland
Editors:
Nicoletta Calzolari, Khalid Choukri, Thierry Declerck, Hrafn Loftsson, Bente Maegaard, Joseph Mariani, Asuncion Moreno, Jan Odijk, Stelios Piperidis
Venue:
LREC
SIG:
Publisher:
European Language Resources Association (ELRA)
Note:
Pages:
2905–2912
Language:
URL:
http://www.lrec-conf.org/proceedings/lrec2014/pdf/230_Paper.pdf
DOI:
Bibkey:
Cite (ACL):
Wolfgang Maier, Miriam Kaeshammer, Peter Baumann, and Sandra Kübler. 2014. Discosuite - A parser test suite for German discontinuous structures. In Proceedings of the Ninth International Conference on Language Resources and Evaluation (LREC'14), pages 2905–2912, Reykjavik, Iceland. European Language Resources Association (ELRA).
Cite (Informal):
Discosuite - A parser test suite for German discontinuous structures (Maier et al., LREC 2014)
Copy Citation:
PDF:
http://www.lrec-conf.org/proceedings/lrec2014/pdf/230_Paper.pdf
PDF Search