@inproceedings{dong-etal-2025-personalization,
    title = "When Personalization Meets Reality: A Multi-Faceted Analysis of Personalized Preference Learning",
    author = {Dong, Yijiang River  and
      Hu, Tiancheng  and
      Liu, Yinhong  and
      {\"U}st{\"u}n, Ahmet  and
      Collier, Nigel},
    editor = "Christodoulopoulos, Christos  and
      Chakraborty, Tanmoy  and
      Rose, Carolyn  and
      Peng, Violet",
    booktitle = "Findings of the Association for Computational Linguistics: EMNLP 2025",
    month = nov,
    year = "2025",
    address = "Suzhou, China",
    publisher = "Association for Computational Linguistics",
    url = "https://preview.aclanthology.org/author-page-yu-wang-polytechnic/2025.findings-emnlp.916/",
    doi = "10.18653/v1/2025.findings-emnlp.916",
    pages = "16880--16894",
    ISBN = "979-8-89176-335-7",
    abstract = "While Reinforcement Learning from Human Feedback (RLHF) is widely used to align Large Language Models (LLMs) with human preferences, it typically assumes homogeneous preferences across users, overlooking diverse human values and minority viewpoints.Although personalized preference learning addresses this by tailoring separate preferences for individual users, the field lacks standardized methods to assess its effectiveness. We present a multi-faceted evaluation framework that measures not only performance but also fairness, unintended effects, and adaptability across varying levels of preference divergence. Through extensive experiments comparing eight personalization methods across three preference datasets, we demonstrate that performance differences between methods could reach 36{\%} when users strongly disagree, and personalization can introduce up to 20{\%} safety misalignment. These findings highlight the critical need for holistic evaluation approaches to advance the development of more effective and inclusive preference learning systems."
}