Abstract
We present Appraise, an open-source framework for crowd-based annotation tasks, notably for evaluation of machine translation output. This is the software used to run the yearly evaluation campaigns for shared tasks at the WMT Conference on Machine Translation. It has also been used at IWSLT 2017 and, recently, to measure human parity for machine translation for Chinese to English news text. The demo will present the full end-to-end lifecycle of an Appraise evaluation campaign, from task creation to annotation and interpretation of results.- Anthology ID:
- C18-2019
- Volume:
- Proceedings of the 27th International Conference on Computational Linguistics: System Demonstrations
- Month:
- August
- Year:
- 2018
- Address:
- Santa Fe, New Mexico
- Editor:
- Dongyan Zhao
- Venue:
- COLING
- SIG:
- Publisher:
- Association for Computational Linguistics
- Note:
- Pages:
- 86–88
- Language:
- URL:
- https://aclanthology.org/C18-2019
- DOI:
- Cite (ACL):
- Christian Federmann. 2018. Appraise Evaluation Framework for Machine Translation. In Proceedings of the 27th International Conference on Computational Linguistics: System Demonstrations, pages 86–88, Santa Fe, New Mexico. Association for Computational Linguistics.
- Cite (Informal):
- Appraise Evaluation Framework for Machine Translation (Federmann, COLING 2018)
- PDF:
- https://preview.aclanthology.org/fix-dup-bibkey/C18-2019.pdf