A Web-based Framework for Collecting and Assessing Highlighted Sentences in a Document
Sasha Spala, Franck Dernoncourt, Walter Chang, Carl Dockhorn
Abstract
Automatically highlighting a text aims at identifying key portions that are the most important to a reader. In this paper, we present a web-based framework designed to efficiently and scalably crowdsource two independent but related tasks: collecting highlight annotations, and comparing the performance of automated highlighting systems. The first task is necessary to understand human preferences and train supervised automated highlighting systems. The second task yields a more accurate and fine-grained evaluation than existing automated performance metrics.- Anthology ID:
- C18-2017
- Volume:
- Proceedings of the 27th International Conference on Computational Linguistics: System Demonstrations
- Month:
- August
- Year:
- 2018
- Address:
- Santa Fe, New Mexico
- Editor:
- Dongyan Zhao
- Venue:
- COLING
- SIG:
- Publisher:
- Association for Computational Linguistics
- Note:
- Pages:
- 78–81
- Language:
- URL:
- https://aclanthology.org/C18-2017
- DOI:
- Cite (ACL):
- Sasha Spala, Franck Dernoncourt, Walter Chang, and Carl Dockhorn. 2018. A Web-based Framework for Collecting and Assessing Highlighted Sentences in a Document. In Proceedings of the 27th International Conference on Computational Linguistics: System Demonstrations, pages 78–81, Santa Fe, New Mexico. Association for Computational Linguistics.
- Cite (Informal):
- A Web-based Framework for Collecting and Assessing Highlighted Sentences in a Document (Spala et al., COLING 2018)
- PDF:
- https://preview.aclanthology.org/ml4al-ingestion/C18-2017.pdf