SubmissionNumber#=%=#190
FinalPaperTitle#=%=#FtG-CoT at SemEval-2024 Task 9: Solving Sentence Puzzles Using Fine-Tuned Language Models and Zero-Shot CoT Prompting
ShortPaperTitle#=%=#
NumberOfPages#=%=#7
CopyrightSigned#=%=#Micah Zhang
JobTitle#==#
Organization#==#University of Colorado Boulder, Boulder, CO 80309
Abstract#==#Recent large language models (LLMs) can solve puzzles that require creativity and lateral thinking. To advance this front of research, we tackle SemEval-2024 Task 9: BRAINTEASER: A Novel Task Defying Common Sense. We approach this task by introducing a technique that we call Fine-tuned Generated Chain-of-Thought (FtG-CoT). It is a novel few-shot prompting method that combines a fine-tuned BERT classifier encoder with zero-shot chain-of-thought generation and a fine-tuned LLM. The fine-tuned BERT classifier provides a context-rich encoding of each example question and choice list. Zero-shot chain-of-thought generation leverages the benefits of chain-of-thought prompting without requiring manual creation of the reasoning chains. We fine-tune the LLM on the generated chains-of-thought and include a set of generated reasoning chains in the final few-shot LLM prompt to maximize the relevance and correctness of the final generated response. In this paper, we show that FtG-CoT outperforms the zero-shot prompting baseline presented in the task paper and is highly effective at solving challenging sentence puzzles achieving a perfect score on the practice set and a 0.9 score on the evaluation set.
Author{1}{Firstname}#=%=#Micah
Author{1}{Lastname}#=%=#Zhang
Author{1}{Username}#=%=#micahzhang
Author{1}{Email}#=%=#micah.zhang@colorado.edu
Author{1}{Affiliation}#=%=#University of Colorado Boulder
Author{2}{Firstname}#=%=#Shafiuddin Rehan
Author{2}{Lastname}#=%=#Ahmed
Author{2}{Username}#=%=#ahmeshaf
Author{2}{Email}#=%=#shah7567@colorado.edu
Author{2}{Affiliation}#=%=#University of Colorado Boulder
Author{3}{Firstname}#=%=#James H.
Author{3}{Lastname}#=%=#Martin
Author{3}{Username}#=%=#james.martin
Author{3}{Email}#=%=#james.martin@colorado.edu
Author{3}{Affiliation}#=%=#University of Colorado Boulder

==========
èéáğö