It’s MBR All the Way Down: Modern Generation Techniques Through the Lens of Minimum Bayes Risk

Amanda Bertsch, Alex Xie, Graham Neubig, Matthew Gormley

Abstract
Minimum Bayes Risk (MBR) decoding is a method for choosing the outputs of a machine learning system based not on the output with the highest probability, but the output with the lowest risk (expected error) among multiple candidates. It is a simple but powerful method: for an additional cost at inference time, MBR provides reliable several-point improvements across metrics for a wide variety of tasks without any additional data or training. Despite this, MBR is not frequently applied in NLP works, and knowledge of the method itself is limited. We first provide an introduction to the method and the recent literature. We show that several recent methods that do not reference MBR can be written as special cases of MBR; this reformulation provides additional theoretical justification for the performance of these methods, explaining some results that were previously only empirical. We provide theoretical and empirical results about the effectiveness of various MBR variants and make concrete recommendations for the application of MBR in NLP models, including future directions in this area.
Anthology ID:
2023.bigpicture-1.9
Volume:
Proceedings of the Big Picture Workshop
Month:
December
Year:
2023
Address:
Singapore
Editors:
Yanai Elazar, Allyson Ettinger, Nora Kassner, Sebastian Ruder, Noah A. Smith
Venue:
BigPicture
SIG:
Publisher:
Association for Computational Linguistics
Note:
Pages:
108–122
Language:
URL:
https://aclanthology.org/2023.bigpicture-1.9
DOI:
10.18653/v1/2023.bigpicture-1.9
Bibkey:
Cite (ACL):
Amanda Bertsch, Alex Xie, Graham Neubig, and Matthew Gormley. 2023. It’s MBR All the Way Down: Modern Generation Techniques Through the Lens of Minimum Bayes Risk. In Proceedings of the Big Picture Workshop, pages 108–122, Singapore. Association for Computational Linguistics.
Cite (Informal):
It’s MBR All the Way Down: Modern Generation Techniques Through the Lens of Minimum Bayes Risk (Bertsch et al., BigPicture 2023)
Copy Citation:
PDF:
https://preview.aclanthology.org/nschneid-patch-5/2023.bigpicture-1.9.pdf
PDF Search