@inproceedings{balepur-etal-2025-whose,
    title = "Whose Boat Does it Float? Improving Personalization in Preference Tuning via Inferred User Personas",
    author = "Balepur, Nishant  and
      Padmakumar, Vishakh  and
      Yang, Fumeng  and
      Feng, Shi  and
      Rudinger, Rachel  and
      Boyd-Graber, Jordan Lee",
    editor = "Che, Wanxiang  and
      Nabende, Joyce  and
      Shutova, Ekaterina  and
      Pilehvar, Mohammad Taher",
    booktitle = "Proceedings of the 63rd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers)",
    month = jul,
    year = "2025",
    address = "Vienna, Austria",
    publisher = "Association for Computational Linguistics",
    url = "https://preview.aclanthology.org/ingestion-acl-25/2025.acl-long.168/",
    pages = "3371--3393",
    ISBN = "979-8-89176-251-0",
    abstract = "LLMs are aligned to follow input instructions by learning which of two responses users prefer for a prompt. However, such preference data do not convey *why* users prefer responses that are chosen or rejected, so LLMs trained on these datasets cannot tailor responses to varied user needs. To surface these parameters of personalization, we apply *abductive reasoning* to preference data, inferring needs and interests of users, i.e., personas, that may prefer either response. We test this idea in two steps: **Persona Inference (PI)**{---}abductively inferring personas of users who prefer chosen or rejected outputs{---}and **Persona Tailoring (PT)**{---}training models to tailor outputs to personas from PI. We show: 1) LLMs infer personas accurately explaining why different users may prefer *both* chosen or rejected outputs; 2) Training on preference data augmented with PI personas via PT boosts personalization and generalizes to supporting user-written personas; and 3) Rejected response personas form harder personalization evaluations, showing PT better aids users with uncommon preferences versus typical alignment methods. We argue for an abductive view of preferences for personalization, asking not only which response is better but when, why, and for whom."
}