Abstract
In modern text annotation projects, crowdsourced annotations are often aggregated using item response models or by majority vote. Recently, item response models enhanced with generative data models have been shown to yield substantial benefits over those with conditional or no data models. However, suitable generative data models do not exist for many tasks, such as semantic labeling tasks. When no generative data model exists, we demonstrate that similar benefits may be derived by conditionally modeling documents that have been previously embedded in a semantic space using recent work in vector space models. We use this approach to show state-of-the-art results on a variety of semantic annotation aggregation tasks.- Anthology ID:
- C16-1168
- Volume:
- Proceedings of COLING 2016, the 26th International Conference on Computational Linguistics: Technical Papers
- Month:
- December
- Year:
- 2016
- Address:
- Osaka, Japan
- Editors:
- Yuji Matsumoto, Rashmi Prasad
- Venue:
- COLING
- SIG:
- Publisher:
- The COLING 2016 Organizing Committee
- Note:
- Pages:
- 1787–1796
- Language:
- URL:
- https://aclanthology.org/C16-1168
- DOI:
- Cite (ACL):
- Paul Felt, Eric Ringger, and Kevin Seppi. 2016. Semantic Annotation Aggregation with Conditional Crowdsourcing Models and Word Embeddings. In Proceedings of COLING 2016, the 26th International Conference on Computational Linguistics: Technical Papers, pages 1787–1796, Osaka, Japan. The COLING 2016 Organizing Committee.
- Cite (Informal):
- Semantic Annotation Aggregation with Conditional Crowdsourcing Models and Word Embeddings (Felt et al., COLING 2016)
- PDF:
- https://preview.aclanthology.org/nschneid-patch-4/C16-1168.pdf