Abstract
This paper describes our submission to the Text Complexity DE Challenge 2022 (Mohtaj et al., 2022). We evaluate a pairwise regression model that predicts the relative difference in complexity of two sentences, instead of predicting a complexity score from a single sentence. In consequence, the model returns samples of scores (as many as there are training sentences) instead of a point estimate. Due to an error in the submission, test set results are unavailable. However, we show by cross-validation that pairwise regression does not improve performance over standard regression models using sentence embeddings taken from pretrained language models as input. Furthermore, we do not find the distribution standard deviations to reflect differences in “uncertainty” of the model predictions in an useful way.- Anthology ID:
- 2022.germeval-1.8
- Volume:
- Proceedings of the GermEval 2022 Workshop on Text Complexity Assessment of German Text
- Month:
- September
- Year:
- 2022
- Address:
- Potsdam, Germany
- Editors:
- Sebastian Möller, Salar Mohtaj, Babak Naderi
- Venue:
- GermEval
- SIG:
- Publisher:
- Association for Computational Linguistics
- Note:
- Pages:
- 45–50
- Language:
- URL:
- https://aclanthology.org/2022.germeval-1.8
- DOI:
- Cite (ACL):
- Leander Girrbach. 2022. Text Complexity DE Challenge 2022 Submission Description: Pairwise Regression for Complexity Prediction. In Proceedings of the GermEval 2022 Workshop on Text Complexity Assessment of German Text, pages 45–50, Potsdam, Germany. Association for Computational Linguistics.
- Cite (Informal):
- Text Complexity DE Challenge 2022 Submission Description: Pairwise Regression for Complexity Prediction (Girrbach, GermEval 2022)
- PDF:
- https://preview.aclanthology.org/nschneid-patch-4/2022.germeval-1.8.pdf
- Data
- TextComplexityDE