@inproceedings{shridhar-etal-2024-art,
    title = "The {ART} of {LLM} Refinement: Ask, Refine, and Trust",
    author = "Shridhar, Kumar  and
      Sinha, Koustuv  and
      Cohen, Andrew  and
      Wang, Tianlu  and
      Yu, Ping  and
      Pasunuru, Ramakanth  and
      Sachan, Mrinmaya  and
      Weston, Jason  and
      Celikyilmaz, Asli",
    editor = "Duh, Kevin  and
      Gomez, Helena  and
      Bethard, Steven",
    booktitle = "Proceedings of the 2024 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies (Volume 1: Long Papers)",
    month = jun,
    year = "2024",
    address = "Mexico City, Mexico",
    publisher = "Association for Computational Linguistics",
    url = "https://preview.aclanthology.org/automate-pypi-publishing/2024.naacl-long.327/",
    doi = "10.18653/v1/2024.naacl-long.327",
    pages = "5872--5883",
    abstract = "Large Language Models (LLMs) have demonstrated remarkable generative abilities, but can they judge the quality of their own generations and self-improve?A popular concept, referred to as \textit{self-refinement}, postulates that LLMs can detect and correct the errors in their generations when asked to do so. However, recent empirical evidence points in the opposite direction, suggesting that LLMs often struggle to accurately identify errors when reasoning is involved. To address this, we propose a reasoning with a refinement strategy called \textit{ART: Ask, Refine, and Trust}, which \textit{asks} necessary questions to decide when an LLM should \textit{refine} its output, and uses it to affirm or deny \textit{trust} in its refinement by ranking the refinement and the initial prediction. On two multistep reasoning tasks of mathematical word problems (GSM8K) and question answering (StrategyQA), \textit{ART} achieves a performance gain of $+5$ points over self-refinement baselines, while using a much smaller model as the decision maker. We believe that \textit{ART} with smaller models, making refinement decisions can be a cost-effective alternative to fine-tuning LLMs."
}