GPoeT: a Language Model Trained for Rhyme Generation on Synthetic Data
Andrei Popescu-Belis, Àlex R. Atrio, Bastien Bernath, Etienne Boisson, Teo Ferrari, Xavier Theimer-Lienhard, Giorgos Vernikos
Abstract
Poem generation with language models requires the modeling of rhyming patterns. We propose a novel solution for learning to rhyme, based on synthetic data generated with a rule-based rhyming algorithm. The algorithm and an evaluation metric use a phonetic dictionary and the definitions of perfect and assonant rhymes. We fine-tune a GPT-2 English model with 124M parameters on 142 MB of natural poems and find that this model generates consecutive rhymes infrequently (11%). We then fine-tune the model on 6 MB of synthetic quatrains with consecutive rhymes (AABB) and obtain nearly 60% of rhyming lines in samples generated by the model. Alternating rhymes (ABAB) are more difficult to model because of longer-range dependencies, but they are still learnable from synthetic data, reaching 45% of rhyming lines in generated samples.- Anthology ID:
- 2023.latechclfl-1.2
- Volume:
- Proceedings of the 7th Joint SIGHUM Workshop on Computational Linguistics for Cultural Heritage, Social Sciences, Humanities and Literature
- Month:
- May
- Year:
- 2023
- Address:
- Dubrovnik, Croatia
- Editors:
- Stefania Degaetano-Ortlieb, Anna Kazantseva, Nils Reiter, Stan Szpakowicz
- Venue:
- LaTeCHCLfL
- SIG:
- Publisher:
- Association for Computational Linguistics
- Note:
- Pages:
- 10–20
- Language:
- URL:
- https://preview.aclanthology.org/add_missing_videos/2023.latechclfl-1.2/
- DOI:
- 10.18653/v1/2023.latechclfl-1.2
- Cite (ACL):
- Andrei Popescu-Belis, Àlex R. Atrio, Bastien Bernath, Etienne Boisson, Teo Ferrari, Xavier Theimer-Lienhard, and Giorgos Vernikos. 2023. GPoeT: a Language Model Trained for Rhyme Generation on Synthetic Data. In Proceedings of the 7th Joint SIGHUM Workshop on Computational Linguistics for Cultural Heritage, Social Sciences, Humanities and Literature, pages 10–20, Dubrovnik, Croatia. Association for Computational Linguistics.
- Cite (Informal):
- GPoeT: a Language Model Trained for Rhyme Generation on Synthetic Data (Popescu-Belis et al., LaTeCHCLfL 2023)
- PDF:
- https://preview.aclanthology.org/add_missing_videos/2023.latechclfl-1.2.pdf