@inproceedings{kirtania-etal-2024-logic,
    title = "{LOGIC}-{LM}++: Multi-Step Refinement for Symbolic Formulations",
    author = "Kirtania, Shashank  and
      Gupta, Priyanshu  and
      Radhakrishna, Arjun",
    editor = "Dalvi Mishra, Bhavana  and
      Durrett, Greg  and
      Jansen, Peter  and
      Lipkin, Ben  and
      Neves Ribeiro, Danilo  and
      Wong, Lionel  and
      Ye, Xi  and
      Zhao, Wenting",
    booktitle = "Proceedings of the 2nd Workshop on Natural Language Reasoning and Structured Explanations (@ACL 2024)",
    month = aug,
    year = "2024",
    address = "Bangkok, Thailand",
    publisher = "Association for Computational Linguistics",
    url = "https://preview.aclanthology.org/ingest-emnlp/2024.nlrse-1.6/",
    pages = "56--63",
    abstract = "In this paper we examine the limitations of Large Language Models (LLMs) for complex reasoning tasks. While current approaches leverage formal languages as intermediate representation for these reasoning problems, they still struggle with generating intermediate for-mal specifications with great correctness and in refining these representations. To address these issues, this paper proposes Logic-LM++, an improvement on Logic-LM (Pan et al., 2023). It uses the ability of LLMs to do pairwise comparisons, allowing the evaluation of the refinements suggested by the LLM. The paper demonstrates that Logic-LM++ outperforms Logic-LM and LLM based techniques on natural language reasoning tasks on two datasets, FOLIO, ProofWriter and AR-LSAT. Logic-LM++ show an average improvement of 18.5{\%} on standard prompting, 12.3{\%} on chain of thought prompting and 5{\%} on Logic-LM."
}