@inproceedings{liu-etal-2026-multimodal,
    title = "Multimodal Item Scoring for Natural Language Recommendation via {G}aussian Process Regression with {LLM} Relevance Judgments",
    author = "Liu, Yifan Simon  and
      Wen, Qianfeng  and
      Liang, Jiazhou  and
      Zhao, Mark  and
      Cui, Justin  and
      Korikov, Anton  and
      Toroghi, Armin  and
      Kim, Junyoung  and
      Sanner, Scott",
    editor = "Liakata, Maria  and
      Moreira, Viviane P.  and
      Zhang, Jiajun  and
      Jurgens, David",
    booktitle = "Findings of the {A}ssociation for {C}omputational {L}inguistics: {ACL} 2026",
    month = jul,
    year = "2026",
    address = "San Diego, California, United States",
    publisher = "Association for Computational Linguistics",
    url = "https://preview.aclanthology.org/ingest-acl/2026.findings-acl.1836/",
    pages = "36859--36876",
    ISBN = "979-8-89176-395-1",
    abstract = "Natural Language Recommendation (NLRec) generates item suggestions based on the relevance between user-issued NL requests and NL item description passages. Existing NLRec approaches often use Dense Retrieval (DR) to compute item relevance scores from aggregation of inner products between user request embeddings and relevant passage embeddings. However, DR views the request as the sole relevance label, thus leading to a unimodal scoring function centered on the query embedding that is often a weak proxy for query relevance. To better capture the potential multimodal distribution of the relevance scoring function that may arise from complex NLRec data, we propose **GPR-LLM** that uses Gaussian Process Regression (GPR) with LLM relevance judgments for a subset of candidate passages. Experiments on four NLRec datasets and two LLM backbones demonstrate that GPR-LLM with an RBF kernel, capable of modeling multimodal relevance scoring functions, consistently outperforms simpler unimodal kernels (dot product, cosine similarity), as well as baseline methods including DR, cross-encoder, and pointwise LLM-based relevance scoring by up to 65{\%}. Overall, GPR-LLM provides an efficient and effective approach to NLRec within a minimal LLM labeling budget."
}