Abstract
We present a very simple model for text quality assessment based on a deep convolutional neural network, where the only supervision required is one corpus of user-generated text of varying quality, and one contrasting text corpus of consistently high quality. Our model is able to provide local quality assessments in different parts of a text, which allows visual feedback about where potentially problematic parts of the text are located, as well as a way to evaluate which textual features are captured by our model. We evaluate our method on two corpora: a large corpus of manually graded student essays and a longitudinal corpus of language learner written production, and find that the text quality metric learned by our model is a fairly strong predictor of both essay grade and learner proficiency level.- Anthology ID:
- W17-5031
- Volume:
- Proceedings of the 12th Workshop on Innovative Use of NLP for Building Educational Applications
- Month:
- September
- Year:
- 2017
- Address:
- Copenhagen, Denmark
- Editors:
- Joel Tetreault, Jill Burstein, Claudia Leacock, Helen Yannakoudakis
- Venue:
- BEA
- SIG:
- SIGEDU
- Publisher:
- Association for Computational Linguistics
- Note:
- Pages:
- 282–286
- Language:
- URL:
- https://aclanthology.org/W17-5031
- DOI:
- 10.18653/v1/W17-5031
- Cite (ACL):
- Robert Östling and Gintare Grigonyte. 2017. Transparent text quality assessment with convolutional neural networks. In Proceedings of the 12th Workshop on Innovative Use of NLP for Building Educational Applications, pages 282–286, Copenhagen, Denmark. Association for Computational Linguistics.
- Cite (Informal):
- Transparent text quality assessment with convolutional neural networks (Östling & Grigonyte, BEA 2017)
- PDF:
- https://preview.aclanthology.org/nschneid-patch-5/W17-5031.pdf