SubmissionNumber#=%=#283
FinalPaperTitle#=%=#Mothman at SemEval-2024 Task 9: An Iterative System for Chain-of-Thought Prompt Optimization
ShortPaperTitle#=%=#
NumberOfPages#=%=#13
CopyrightSigned#=%=#Alvin Po-Chun Chen
JobTitle#==#Researcher
Organization#==#University of Colorado, Boulder
1725 Euclid Ave,
Boulder, CO
80309
Abstract#==#Extensive research exists on the performance of large language models on logic-based tasks, whereas relatively little has been done on their ability to generate creative solutions on lateral thinking tasks. The BrainTeaser shared task tests lateral thinking and uses adversarial datasets to prevent memorization, resulting in poor performance for out-of-the-box models. We propose a system for iterative, chain-of-thought prompt engineering which optimizes prompts using human evaluation. Using this shared task, we demonstrate our system's ability to significantly improve model performance by optimizing prompts and evaluate the input dataset.
Author{1}{Firstname}#=%=#Alvin Po-Chun
Author{1}{Lastname}#=%=#Chen
Author{1}{Username}#=%=#alvinchen
Author{1}{Email}#=%=#alvin.chen@colorado.edu
Author{1}{Affiliation}#=%=#University of Colorado Boulder
Author{2}{Firstname}#=%=#Ray
Author{2}{Lastname}#=%=#Groshan
Author{2}{Email}#=%=#ray.groshan@colorado.edu
Author{2}{Affiliation}#=%=#University of Colorado Boulder
Author{3}{Firstname}#=%=#Sean
Author{3}{Lastname}#=%=#Von Bayern
Author{3}{Email}#=%=#sean.vonbayern@colorado.edu
Author{3}{Affiliation}#=%=#University of Colorado Boulder

==========
èéáğö