NUBIA: NeUral Based Interchangeability Assessor for Text Generation
Hassan Kane, Muhammed Yusuf Kocyigit, Ali Abdalla, Pelkins Ajanoh, Mohamed Coulibali
Abstract
We present NUBIA, a methodology to build automatic evaluation metrics for text generation using only machine learning models as core components. A typical NUBIA model is composed of three modules: a neural feature extractor, an aggregator and a calibrator. We demonstrate an implementation of NUBIA showing competitive performance with stateof-the art metrics used to evaluate machine translation and state-of-the art results for image captions quality evaluation. In addition to strong performance, NUBIA models have the advantage of being modular and improve in synergy with advances in text generation models.- Anthology ID:
- 2020.evalnlgeval-1.4
- Volume:
- Proceedings of the 1st Workshop on Evaluating NLG Evaluation
- Month:
- December
- Year:
- 2020
- Address:
- Online (Dublin, Ireland)
- Editors:
- Shubham Agarwal, Ondřej Dušek, Sebastian Gehrmann, Dimitra Gkatzia, Ioannis Konstas, Emiel Van Miltenburg, Sashank Santhanam
- Venue:
- EvalNLGEval
- SIG:
- SIGGEN
- Publisher:
- Association for Computational Linguistics
- Note:
- Pages:
- 28–37
- Language:
- URL:
- https://aclanthology.org/2020.evalnlgeval-1.4
- DOI:
- Cite (ACL):
- Hassan Kane, Muhammed Yusuf Kocyigit, Ali Abdalla, Pelkins Ajanoh, and Mohamed Coulibali. 2020. NUBIA: NeUral Based Interchangeability Assessor for Text Generation. In Proceedings of the 1st Workshop on Evaluating NLG Evaluation, pages 28–37, Online (Dublin, Ireland). Association for Computational Linguistics.
- Cite (Informal):
- NUBIA: NeUral Based Interchangeability Assessor for Text Generation (Kane et al., EvalNLGEval 2020)
- PDF:
- https://preview.aclanthology.org/naacl-24-ws-corrections/2020.evalnlgeval-1.4.pdf
- Data
- GLUE