@inproceedings{jamil-etal-2025-diy,
    title = "$\textit{Do It Yourself (DIY)}$: Modifying Images for Poems in a Zero-Shot Setting Using Weighted Prompt Manipulation",
    author = "Jamil, Sofia  and
      Charan, Kotla Sai  and
      Saha, Sriparna  and
      Goswami, Koustava  and
      J, Joseph K",
    editor = "Christodoulopoulos, Christos  and
      Chakraborty, Tanmoy  and
      Rose, Carolyn  and
      Peng, Violet",
    booktitle = "Proceedings of the 2025 Conference on Empirical Methods in Natural Language Processing",
    month = nov,
    year = "2025",
    address = "Suzhou, China",
    publisher = "Association for Computational Linguistics",
    url = "https://preview.aclanthology.org/ingest-emnlp/2025.emnlp-main.994/",
    pages = "19668--19676",
    ISBN = "979-8-89176-332-6",
    abstract = "Poetry is an expressive form of art that invites multiple interpretations, as readers often bring their own emotions, experiences, and cultural backgrounds into their understanding of a poem. Recognizing this, we aim to generate images for poems and improve these images in a zero-shot setting, enabling audiences to modify images as per their requirements. To achieve this, we introduce a novel $\textit{Weighted Prompt Manipulation (WPM)}$ technique, which systematically modifies attention weights and text embeddings within diffusion models. By dynamically adjusting the importance of specific words, $\textit{WPM}$ enhances or suppresses their influence in the final generated image, leading to semantically richer and more contextually accurate visualizations. Our approach exploits diffusion models and large language models (LLMs) such as GPT in conjunction with existing poetry datasets, ensuring a comprehensive and structured methodology for improved image generation in the literary domain. To the best of our knowledge, this is the first attempt at integrating weighted prompt manipulation for enhancing imagery in poetic language."
}