Meta-Reasoning: Semantics-Symbol Deconstruction for Large Language Models

Yiming Wang, Zhuosheng Zhang, Pei Zhang, Baosong Yang, Rui Wang

Abstract
Neural-symbolic methods have demonstrated efficiency in enhancing the reasoning abilities of large language models (LLMs). However, existing methods mainly rely on syntactically mapping natural languages to complete formal languages like Python and SQL. Those methods require that reasoning tasks be convertible into programs, which cater to the computer execution mindset and deviate from human reasoning habits. To broaden symbolic methods’ applicability and adaptability in the real world, we propose Meta-Reasoning from a linguistic perspective. This method empowers LLMs to deconstruct reasoning-independent semantic information into generic symbolic representations, thereby efficiently capturing more generalized reasoning knowledge. We conduct extensive experiments on more than ten datasets encompassing conventional reasoning tasks like arithmetic, symbolic, and logical reasoning, and the more complex interactive reasoning tasks like theory-of-mind reasoning. Experimental results demonstrate that Meta-Reasoning significantly enhances in-context reasoning accuracy, learning efficiency, out-of-domain generalization, and output stability compared to the Chain-of-Thought technique.
Anthology ID:
2024.findings-acl.34
Volume:
Findings of the Association for Computational Linguistics: ACL 2024
Month:
August
Year:
2024
Address:
Bangkok, Thailand
Editors:
Lun-Wei Ku, Andre Martins, Vivek Srikumar
Venue:
Findings
SIG:
Publisher:
Association for Computational Linguistics
Note:
Pages:
622–643
Language:
URL:
https://aclanthology.org/2024.findings-acl.34
DOI:
10.18653/v1/2024.findings-acl.34
Bibkey:
Cite (ACL):
Yiming Wang, Zhuosheng Zhang, Pei Zhang, Baosong Yang, and Rui Wang. 2024. Meta-Reasoning: Semantics-Symbol Deconstruction for Large Language Models. In Findings of the Association for Computational Linguistics: ACL 2024, pages 622–643, Bangkok, Thailand. Association for Computational Linguistics.
Cite (Informal):
Meta-Reasoning: Semantics-Symbol Deconstruction for Large Language Models (Wang et al., Findings 2024)
Copy Citation:
PDF:
https://preview.aclanthology.org/landing_page/2024.findings-acl.34.pdf
PDF Search