@inproceedings{wang-etal-2025-user,
    title = "User Feedback Alignment for {LLM}-powered Exploration in Large-scale Recommendation Systems",
    author = "Wang, Jianling  and
      Liu, Yifan  and
      Sun, Yinghao  and
      Ma, Xuejian  and
      Wang, Yueqi  and
      Ma, He  and
      Su, Zhengyang  and
      Chen, Minmin  and
      Gao, Mingyan  and
      Dalal, Onkar  and
      Chi, Ed H.  and
      Hong, Lichan  and
      Han, Ningren  and
      Lu, Haokai",
    editor = "Rehm, Georg  and
      Li, Yunyao",
    booktitle = "Proceedings of the 63rd Annual Meeting of the Association for Computational Linguistics (Volume 6: Industry Track)",
    month = jul,
    year = "2025",
    address = "Vienna, Austria",
    publisher = "Association for Computational Linguistics",
    url = "https://preview.aclanthology.org/acl25-workshop-ingestion/2025.acl-industry.70/",
    pages = "996--1003",
    ISBN = "979-8-89176-288-6",
    abstract = "Exploration, the act of broadening user experiences beyond their established preferences, is challenging in large-scale recommendation systems due to feedback loops and limited signals on user exploration patterns. Large Language Models (LLMs) offer potential solutions by leveraging their world knowledge to recommend novel content outside these loops. A key challenge is aligning LLMs with user preferences while preserving their knowledge and reasoning. To enhance planning for new user interests using LLMs, this paper introduces a novel approach that combines hierarchical planning with LLM inference-time scaling. This method aims to improve recommendation relevancy without compromising novelty. We decouple novelty and user-alignment, training separate LLMs for each objective. We then scale up the novelty-focused LLM{'}s inference and select the best-of-n predictions using the user-aligned LLM. Live experiments demonstrate efficacy, showing significant gains in both user satisfaction (measured by watch activity and active user counts) and exploration diversity."
}