@inproceedings{ruiz-etal-2025-bon,
    title = "{B}o{N} Appetit Team at {L}e{W}i{D}i-2025: Best-of-N Test-time Scaling Can Not Stomach Annotation Disagreements (Yet)",
    author = "Ruiz, Tomas  and
      Peng, Siyao  and
      Plank, Barbara  and
      Schwemmer, Carsten",
    editor = "Abercrombie, Gavin  and
      Basile, Valerio  and
      Frenda, Simona  and
      Tonelli, Sara  and
      Dudy, Shiran",
    booktitle = "Proceedings of the The 4th Workshop on Perspectivist Approaches to NLP",
    month = nov,
    year = "2025",
    address = "Suzhou, China",
    publisher = "Association for Computational Linguistics",
    url = "https://preview.aclanthology.org/ingest-emnlp/2025.nlperspectives-1.14/",
    pages = "153--170",
    ISBN = "979-8-89176-350-0",
    abstract = "Test-time scaling is a family of techniques to improve LLM outputs at inference time by performing extra computation. To the best of our knowledge, test-time scaling has been limited to domains with verifiably correct answers, like mathematics and coding. We transfer test-time scaling to the LeWiDi-2025 tasks to evaluate annotation disagreements. We experiment with three test-time scaling methods: two benchmark algorithms (Model Averaging and Majority Voting), and a Best-of-N (BoN) sampling method. The two benchmark methods improve LLM performance consistently on the LeWiDi tasks, but the BoN method does not. Our experiments suggest that the BoN method does not currently transfer from mathematics to LeWiDi tasks, and we analyze potential reasons for this gap."
}