@inproceedings{liu-etal-2026-openrubrics,
    title = "{O}pen{R}ubrics: Towards Scalable Synthetic Rubric Generation for Reward Modeling and {LLM} Alignment",
    author = "Liu, Tianci  and
      Xu, Ran  and
      Yu, Tony  and
      Hong, Ilgee  and
      Yang, Carl  and
      Zhao, Tuo  and
      Wang, Haoyu",
    editor = "Liakata, Maria  and
      Moreira, Viviane P.  and
      Zhang, Jiajun  and
      Jurgens, David",
    booktitle = "Proceedings of the 64th Annual Meeting of the {A}ssociation for {C}omputational {L}inguistics (Volume 1: Long Papers)",
    month = jul,
    year = "2026",
    address = "San Diego, California, United States",
    publisher = "Association for Computational Linguistics",
    url = "https://preview.aclanthology.org/ingest-acl/2026.acl-long.791/",
    pages = "17417--17437",
    ISBN = "979-8-89176-390-6",
    abstract = "Reward modeling lies at the core of reinforcement learning from human feedback (RLHF), yet most existing reward models rely on scalar or pairwise judgments that fail to capture the multifaceted nature of human preferences. Recent studies have explored \textit{rubrics-as-rewards} (RaR) that uses structured criteria to capture multiple dimensions of response quality. However, producing rubrics that are both reliable and scalable remains a key challenge. In this work, we introduce OpenRubrics, a diverse, large-scale collection of (prompt, rubric) pairs for training rubric-generation and rubric-based reward models. To elicit discriminative and comprehensive evaluation signals, we introduce \textit{Contrastive Rubric Generation} (CRG), which derives both hard rules (explicit constraints) and principles (implicit qualities) by contrasting preferred and rejected responses. We further remove noisy rubrics via preserving preference{--}label consistency. Across multiple reward-modeling benchmarks, our rubric-based reward model, Rubric-RM, surpasses strong size-matched baselines by 8.4{\%}. These gains transfer to policy models on instruction-following and biomedical benchmarks.The {model weights} and {datasets} are publicly available at https://huggingface.co/OpenRubrics."
}