Group Preference Alignment: Customizing LLM Responses from In-Situ Conversations Only When Needed

Ishani Mondal, Jack W. Stokes, Sujay Kumar Jauhar, Longqi Yang, Mengting Wan, Xiaofeng Xu, Xia Song, Jordan Lee Boyd-Graber, Jennifer Neville


Abstract
LLMs often fail to meet specialized needs of distinct user groups due to their one-size-fits-all approach, and there is limited understanding of what personalization each group expects.To address this, we propose GPA a group-aware personalization framework that captures context-specific preference variations and steers LLMs accordingly.Our approach involves: (1) Group-Aware Preference Extraction, which distills divergent preferences from real-world conversation logs into interpretable rubrics, and (2) Tailored Response Generation, using (a) GPA-CT, which adapts responses using learnt rubrics, and (b) GPA-FT, which finetunes models using rubric-guided synthetic data.Automatic and Human evaluations confirm that GPA improves group alignment without compromising perfomance on standard instruction-following benchmarks.
Anthology ID:
2025.emnlp-industry.56
Volume:
Proceedings of the 2025 Conference on Empirical Methods in Natural Language Processing: Industry Track
Month:
November
Year:
2025
Address:
Suzhou (China)
Editors:
Saloni Potdar, Lina Rojas-Barahona, Sebastien Montella
Venue:
EMNLP
SIG:
Publisher:
Association for Computational Linguistics
Note:
Pages:
825–849
Language:
URL:
https://preview.aclanthology.org/ingest-emnlp/2025.emnlp-industry.56/
DOI:
Bibkey:
Cite (ACL):
Ishani Mondal, Jack W. Stokes, Sujay Kumar Jauhar, Longqi Yang, Mengting Wan, Xiaofeng Xu, Xia Song, Jordan Lee Boyd-Graber, and Jennifer Neville. 2025. Group Preference Alignment: Customizing LLM Responses from In-Situ Conversations Only When Needed. In Proceedings of the 2025 Conference on Empirical Methods in Natural Language Processing: Industry Track, pages 825–849, Suzhou (China). Association for Computational Linguistics.
Cite (Informal):
Group Preference Alignment: Customizing LLM Responses from In-Situ Conversations Only When Needed (Mondal et al., EMNLP 2025)
Copy Citation:
PDF:
https://preview.aclanthology.org/ingest-emnlp/2025.emnlp-industry.56.pdf