@inproceedings{popescu-belis-etal-2023-gpoet,
    title = "{GP}oe{T}: a Language Model Trained for Rhyme Generation on Synthetic Data",
    author = "Popescu-Belis, Andrei  and
      Atrio, {\`A}lex R.  and
      Bernath, Bastien  and
      Boisson, Etienne  and
      Ferrari, Teo  and
      Theimer-Lienhard, Xavier  and
      Vernikos, Giorgos",
    editor = "Degaetano-Ortlieb, Stefania  and
      Kazantseva, Anna  and
      Reiter, Nils  and
      Szpakowicz, Stan",
    booktitle = "Proceedings of the 7th Joint SIGHUM Workshop on Computational Linguistics for Cultural Heritage, Social Sciences, Humanities and Literature",
    month = may,
    year = "2023",
    address = "Dubrovnik, Croatia",
    publisher = "Association for Computational Linguistics",
    url = "https://preview.aclanthology.org/add_missing_videos/2023.latechclfl-1.2/",
    doi = "10.18653/v1/2023.latechclfl-1.2",
    pages = "10--20",
    abstract = "Poem generation with language models requires the modeling of rhyming patterns. We propose a novel solution for learning to rhyme, based on synthetic data generated with a rule-based rhyming algorithm. The algorithm and an evaluation metric use a phonetic dictionary and the definitions of perfect and assonant rhymes. We fine-tune a GPT-2 English model with 124M parameters on 142 MB of natural poems and find that this model generates consecutive rhymes infrequently (11{\%}). We then fine-tune the model on 6 MB of synthetic quatrains with consecutive rhymes (AABB) and obtain nearly 60{\%} of rhyming lines in samples generated by the model. Alternating rhymes (ABAB) are more difficult to model because of longer-range dependencies, but they are still learnable from synthetic data, reaching 45{\%} of rhyming lines in generated samples."
}