Local Temperature Beam Search: Avoid Neural Text DeGeneration via Enhanced Calibration

Dongkyu Lee, Gyeonghun Kim, Janghoon Han, Taesuk Hong, Yi-Reun Kim, Stanley Jungkyu Choi, Nevin L. Zhang

Abstract
Previous studies have constantly observed that a language model repeats itself, creating repetitions in an output sequence. To cope with the issue, stochastic decoding schemes have been the de facto approaches; the strategies add randomness in inference, hence avoiding the “self-loop”. However, the remedy comes at the cost of sacrificing output quality due to the randomness involved. In this work, we introduce a deterministic decoding scheme, local temperature beam search. This inference algorithm is an embarrassingly simple variant of beam search, yet it reduces repetition, whose level is superior to that of a sampling-based decoding algorithm, while maintaining the level of coherence as in beam search. Our idea is rooted in the concept of model calibration; we view a repetition as a casualty from overconfidence in a model. Therefore, our work mitigates the miscalibration present in the course of inference with a post-calibration approach applied in beam-specific manner. Our inference scheme is validated on text completion tasks, in which the repetition problem is seen most clearly, and is exhaustively compared with existing inference schemes.
Anthology ID:
2023.findings-acl.628
Volume:
Findings of the Association for Computational Linguistics: ACL 2023
Month:
July
Year:
2023
Address:
Toronto, Canada
Editors:
Anna Rogers, Jordan Boyd-Graber, Naoaki Okazaki
Venue:
Findings
SIG:
Publisher:
Association for Computational Linguistics
Note:
Pages:
9903–9915
Language:
URL:
https://aclanthology.org/2023.findings-acl.628
DOI:
10.18653/v1/2023.findings-acl.628
Bibkey:
Cite (ACL):
Dongkyu Lee, Gyeonghun Kim, Janghoon Han, Taesuk Hong, Yi-Reun Kim, Stanley Jungkyu Choi, and Nevin L. Zhang. 2023. Local Temperature Beam Search: Avoid Neural Text DeGeneration via Enhanced Calibration. In Findings of the Association for Computational Linguistics: ACL 2023, pages 9903–9915, Toronto, Canada. Association for Computational Linguistics.
Cite (Informal):
Local Temperature Beam Search: Avoid Neural Text DeGeneration via Enhanced Calibration (Lee et al., Findings 2023)
Copy Citation:
PDF:
https://preview.aclanthology.org/naacl24-info/2023.findings-acl.628.pdf
PDF Search