SubmissionNumber#=%=#250 FinalPaperTitle#=%=#HW-TSC at SemEval-2024 Task 9: Exploring Prompt Engineering Strategies for Brain Teaser Puzzles Through LLMs ShortPaperTitle#=%=# NumberOfPages#=%=#6 CopyrightSigned#=%=#Yinglu Li JobTitle#==# Organization#==# Abstract#==#Large Language Models (LLMs) have demonstrated impressive performance on many Natural Language Processing (NLP) tasks. However, their ability to solve more creative, lateral thinking puzzles remains relatively unexplored. In this work, we develop methods to enhance the lateral thinking and puzzle-solving capabilities of LLMs. We curate a dataset of word-type and sentence-type brain teasers requiring creative problem-solving abilities beyond commonsense reasoning. We first evaluate the zero-shot performance of models like GPT-3.5 and GPT-4 on this dataset. To improve their puzzle-solving skills, we employ prompting techniques like providing reasoning clues and chaining multiple examples to demonstrate the desired thinking process. We also fine-tune the state-of-the-art Mixtral 7x8b LLM on our dataset. Our methods enable the models to achieve strong results, securing 2nd and 3rd places in the brain teaser task. Our work highlights the potential of LLMs in acquiring complex reasoning abilities with the appropriate training. The efficacy of our approaches opens up new research avenues into advancing lateral thinking and creative problem-solving with AI systems. Author{1}{Firstname}#=%=#Yinglu Author{1}{Lastname}#=%=#Li Author{1}{Username}#=%=#yinglu_li Author{1}{Email}#=%=#yli17825@gmail.com Author{1}{Affiliation}#=%=#Huawei Technologies Co., Ltd. Author{2}{Firstname}#=%=#Zhao Author{2}{Lastname}#=%=#Yanqing Author{2}{Username}#=%=#zhaoyanqing8093 Author{2}{Email}#=%=#zhaoyanqing@huawei.com Author{2}{Affiliation}#=%=#Huawei Author{3}{Firstname}#=%=#Min Author{3}{Lastname}#=%=#Zhang Author{3}{Username}#=%=#zhangmin186 Author{3}{Email}#=%=#zhangmin186@huawei.com Author{3}{Affiliation}#=%=#Huawei Author{4}{Firstname}#=%=#Yadong Author{4}{Lastname}#=%=#Deng Author{4}{Email}#=%=#dengyadong@huawei.com Author{4}{Affiliation}#=%=#Huawei Author{5}{Firstname}#=%=#Aiju Author{5}{Lastname}#=%=#Geng Author{5}{Email}#=%=#gengaiju@huawei.com Author{5}{Affiliation}#=%=#Huawei Author{6}{Firstname}#=%=#Xiaoqin Author{6}{Lastname}#=%=#Liu Author{6}{Email}#=%=#liuxiaoqin09@huawei.com Author{6}{Affiliation}#=%=#Huawei Author{7}{Firstname}#=%=#Mengxin Author{7}{Lastname}#=%=#Ren Author{7}{Email}#=%=#renmengxin2@huawei.com Author{7}{Affiliation}#=%=#Huawei Author{8}{Firstname}#=%=#Yuang Author{8}{Lastname}#=%=#Li Author{8}{Username}#=%=#lyaleo Author{8}{Email}#=%=#liyuang3@huawei.com Author{8}{Affiliation}#=%=#Huawei Author{9}{Firstname}#=%=#Su Author{9}{Lastname}#=%=#Chang Author{9}{Username}#=%=#abulice Author{9}{Email}#=%=#suchang8@huawei.com Author{9}{Affiliation}#=%=#Huawei TSC Author{10}{Firstname}#=%=#Xiaofeng Author{10}{Lastname}#=%=#Zhao Author{10}{Email}#=%=#zhaoxiaofeng14@huawei.com Author{10}{Affiliation}#=%=#Huawei Author{11}{Firstname}#=%=#Xiaosong Author{11}{Lastname}#=%=#Qiao Author{11}{Username}#=%=#qiaoxiaosong Author{11}{Email}#=%=#qiaoxiaosong@huawei.com Author{11}{Affiliation}#=%=#Huawei Technologies Co Ltd Author{12}{Firstname}#=%=#Ming Author{12}{Lastname}#=%=#Zhu Author{12}{Email}#=%=#zhuming47@huawei.com Author{12}{Affiliation}#=%=#Huawei Author{13}{Firstname}#=%=#Yilun Author{13}{Lastname}#=%=#Liu Author{13}{Username}#=%=#lunyiliu Author{13}{Email}#=%=#liuyilun3@huawei.com Author{13}{Affiliation}#=%=#2012 Lab, Huawei co. LTD Author{14}{Firstname}#=%=#Mengyao Author{14}{Lastname}#=%=#Piao Author{14}{Username}#=%=#mengyaopiao Author{14}{Email}#=%=#piaomengyao1@huawei.com Author{14}{Affiliation}#=%=#HuaweiCo.Ltd Author{15}{Firstname}#=%=#Feiyu Author{15}{Lastname}#=%=#Yao Author{15}{Email}#=%=#yaofeiyu1@huawei.com Author{15}{Affiliation}#=%=#Huawei Author{16}{Firstname}#=%=#shimin Author{16}{Lastname}#=%=#tao Author{16}{Username}#=%=#taoshimin Author{16}{Email}#=%=#taoshimin@huawei.com Author{16}{Affiliation}#=%=#huawei Author{17}{Firstname}#=%=#Hao Author{17}{Lastname}#=%=#Yang Author{17}{Username}#=%=#yanghaocsg Author{17}{Email}#=%=#yanghao30@huawei.com Author{17}{Affiliation}#=%=#Huawei Co. Ltd Author{18}{Firstname}#=%=#Yanfei Author{18}{Lastname}#=%=#Jiang Author{18}{Email}#=%=#jiangyanfei@huawei.com Author{18}{Affiliation}#=%=#Huawei ========== èéáğö