Improving Argument Effectiveness Across Ideologies using Instruction-tuned Large Language Models
Roxanne El Baff, Khalid Al Khatib, Milad Alshomary, Kai Konen, Benno Stein, Henning Wachsmuth
Abstract
Different political ideologies (e.g., liberal and conservative Americans) hold different worldviews, which leads to opposing stances on different issues (e.g., gun control) and, thereby, fostering societal polarization. Arguments are a means of bringing the perspectives of people with different ideologies closer together, depending on how well they reach their audience. In this paper, we study how to computationally turn ineffective arguments into effective arguments for people with certain ideologies by using instruction-tuned large language models (LLMs), looking closely at style features. For development and evaluation, we collect ineffective arguments per ideology from debate.org, and we generate about 30k, which we rewrite using three LLM methods tailored to our task: zero-shot prompting, few-shot prompting, and LLM steering. Our experiments provide evidence that LLMs naturally improve argument effectiveness for liberals. Our LLM-based and human evaluation show a clear preference towards the rewritten arguments. Code and link to the data are available here: https://github.com/roxanneelbaff/emnlp2024-iesta.- Anthology ID:
- 2024.findings-emnlp.265
- Volume:
- Findings of the Association for Computational Linguistics: EMNLP 2024
- Month:
- November
- Year:
- 2024
- Address:
- Miami, Florida, USA
- Editors:
- Yaser Al-Onaizan, Mohit Bansal, Yun-Nung Chen
- Venue:
- Findings
- SIG:
- Publisher:
- Association for Computational Linguistics
- Note:
- Pages:
- 4604–4622
- Language:
- URL:
- https://aclanthology.org/2024.findings-emnlp.265
- DOI:
- 10.18653/v1/2024.findings-emnlp.265
- Cite (ACL):
- Roxanne El Baff, Khalid Al Khatib, Milad Alshomary, Kai Konen, Benno Stein, and Henning Wachsmuth. 2024. Improving Argument Effectiveness Across Ideologies using Instruction-tuned Large Language Models. In Findings of the Association for Computational Linguistics: EMNLP 2024, pages 4604–4622, Miami, Florida, USA. Association for Computational Linguistics.
- Cite (Informal):
- Improving Argument Effectiveness Across Ideologies using Instruction-tuned Large Language Models (El Baff et al., Findings 2024)
- PDF:
- https://preview.aclanthology.org/landing_page/2024.findings-emnlp.265.pdf