Abstract
Tables play a crucial role in conveying information in various domains. We propose a Plan-then-Reason framework to answer different types of user queries over tables with sentence context. The framework first plans the reasoning paths over the context, then assigns each step to program-based or textual reasoning to reach the final answer. This framework enhances the table reasoning abilities for both in-context learning and fine-tuning methods. GPT-3.5-Turbo following Plan-then-Reason framework surpasses other prompting baselines without self-consistency while using less API calls and in-context demonstrations. We also construct an instruction tuning set TrixInstruct to evaluate the effectiveness of fine-tuning with this framework. We present ProTrix model family by finetuning models on TrixInstruct. Our experiments show that ProTrix family generalizes to diverse unseen tabular tasks with only 6k training instances. We further demonstrate that ProTrix can generate accurate and faithful explanations to answer complex free-form questions. Our work underscores the importance of the planning and reasoning abilities towards a model over tabular tasks with generalizability and interpretability. We will open-source our dataset and models.- Anthology ID:
- 2024.findings-emnlp.253
- Volume:
- Findings of the Association for Computational Linguistics: EMNLP 2024
- Month:
- November
- Year:
- 2024
- Address:
- Miami, Florida, USA
- Editors:
- Yaser Al-Onaizan, Mohit Bansal, Yun-Nung Chen
- Venue:
- Findings
- SIG:
- Publisher:
- Association for Computational Linguistics
- Note:
- Pages:
- 4378–4406
- Language:
- URL:
- https://aclanthology.org/2024.findings-emnlp.253
- DOI:
- 10.18653/v1/2024.findings-emnlp.253
- Cite (ACL):
- Zirui Wu and Yansong Feng. 2024. ProTrix: Building Models for Planning and Reasoning over Tables with Sentence Context. In Findings of the Association for Computational Linguistics: EMNLP 2024, pages 4378–4406, Miami, Florida, USA. Association for Computational Linguistics.
- Cite (Informal):
- ProTrix: Building Models for Planning and Reasoning over Tables with Sentence Context (Wu & Feng, Findings 2024)
- PDF:
- https://preview.aclanthology.org/landing_page/2024.findings-emnlp.253.pdf