RISCORE: Enhancing In-Context Riddle Solving in Language Models through Context-Reconstructed Example Augmentation
Ioannis Panagiotopoulos, George Filandrianos, Maria Lymperaiou, Giorgos Stamou
Abstract
Riddle-solving requires advanced reasoning skills, pushing Large Language Models (LLMs) to engage in abstract thinking and creative problem-solving, often revealing limitations in their cognitive abilities. In this paper, we examine the riddle-solving capabilities of LLMs using a multiple-choice format, exploring how different prompting techniques impact performance on riddles that demand diverse reasoning skills. To enhance results, we introduce RISCORE (RIddle Solving with COntext REcontruciton) a novel fully automated prompting method that generates and utilizes contextually reconstructed sentence-based puzzles in conjunction with the original examples to create few-shot exemplars. Our experiments demonstrate that RISCORE significantly improves the performance of language models in both vertical and lateral thinking tasks, surpassing traditional exemplar selection strategies across a variety of few-shot settings.- Anthology ID:
- 2025.coling-main.633
- Volume:
- Proceedings of the 31st International Conference on Computational Linguistics
- Month:
- January
- Year:
- 2025
- Address:
- Abu Dhabi, UAE
- Editors:
- Owen Rambow, Leo Wanner, Marianna Apidianaki, Hend Al-Khalifa, Barbara Di Eugenio, Steven Schockaert
- Venue:
- COLING
- SIG:
- Publisher:
- Association for Computational Linguistics
- Note:
- Pages:
- 9431–9455
- Language:
- URL:
- https://preview.aclanthology.org/fix-sig-urls/2025.coling-main.633/
- DOI:
- Cite (ACL):
- Ioannis Panagiotopoulos, George Filandrianos, Maria Lymperaiou, and Giorgos Stamou. 2025. RISCORE: Enhancing In-Context Riddle Solving in Language Models through Context-Reconstructed Example Augmentation. In Proceedings of the 31st International Conference on Computational Linguistics, pages 9431–9455, Abu Dhabi, UAE. Association for Computational Linguistics.
- Cite (Informal):
- RISCORE: Enhancing In-Context Riddle Solving in Language Models through Context-Reconstructed Example Augmentation (Panagiotopoulos et al., COLING 2025)
- PDF:
- https://preview.aclanthology.org/fix-sig-urls/2025.coling-main.633.pdf