SubmissionNumber#=%=#271 FinalPaperTitle#=%=#HW-TSC at SemEval-2024 Task 5: Self-Eval? A Confident LLM System for Auto Prediction and Evaluation for the Legal Argument Reasoning Task ShortPaperTitle#=%=# NumberOfPages#=%=#5 CopyrightSigned#=%=#Xiaofeng Zhao JobTitle#==# Organization#==# Abstract#==#In this article, we present an effective system for semeval-2024 task 5. The task involves assessing the feasibility of a given solution in civil litigation cases based on relevant legal provisions, issues, solutions, and analysis. This task demands a high level of proficiency in U.S. law and natural language reasoning. In this task, we designed a self-eval LLM system that simultaneously performs reasoning and self-assessment tasks. We created a confidence interval and a prompt instructing the LLM to output the answer to a question along with its confidence level. We designed a series of experiments to prove the effectiveness of the self-eval mechanism. In order to avoid the randomness of the results, the final result is obtained by voting on three results generated by the GPT-4. Our submission was conducted under zero-resource setting, and we achieved first place in the task with an F1-score of 0.8231 and an accuracy of 0.8673. Author{1}{Firstname}#=%=#Xiaofeng Author{1}{Lastname}#=%=#Zhao Author{1}{Username}#=%=#zhaoxf4 Author{1}{Email}#=%=#zhaoxiaofeng14@huawei.com Author{1}{Affiliation}#=%=#Huawei Technologies Co Ltd Author{2}{Firstname}#=%=#Xiaosong Author{2}{Lastname}#=%=#Qiao Author{2}{Username}#=%=#qiaoxiaosong Author{2}{Email}#=%=#qiaoxiaosong@huawei.com Author{2}{Affiliation}#=%=#Huawei Technologies Co Ltd Author{3}{Firstname}#=%=#Kaiwen Author{3}{Lastname}#=%=#Ou Author{3}{Email}#=%=#ko.kaiwenou@gmail.com Author{3}{Affiliation}#=%=#Huawei Technologies Co Ltd Author{4}{Firstname}#=%=#Min Author{4}{Lastname}#=%=#Zhang Author{4}{Username}#=%=#zhangmin186 Author{4}{Email}#=%=#zhangmin186@huawei.com Author{4}{Affiliation}#=%=#Huawei Author{5}{Firstname}#=%=#Su Author{5}{Lastname}#=%=#Chang Author{5}{Username}#=%=#abulice Author{5}{Email}#=%=#suchang8@huawei.com Author{5}{Affiliation}#=%=#Huawei TSC Author{6}{Firstname}#=%=#Mengyao Author{6}{Lastname}#=%=#Piao Author{6}{Username}#=%=#mengyaopiao Author{6}{Email}#=%=#piaomengyao1@huawei.com Author{6}{Affiliation}#=%=#HuaweiCo.Ltd Author{7}{Firstname}#=%=#Yuang Author{7}{Lastname}#=%=#Li Author{7}{Username}#=%=#lyaleo Author{7}{Email}#=%=#liyuang3@huawei.com Author{7}{Affiliation}#=%=#Huawei Author{8}{Firstname}#=%=#Yinglu Author{8}{Lastname}#=%=#Li Author{8}{Email}#=%=#liyinglu@huawei.com Author{8}{Affiliation}#=%=#HuaweiCo.Ltd Author{9}{Firstname}#=%=#Ming Author{9}{Lastname}#=%=#Zhu Author{9}{Email}#=%=#zhuming47@huawei.com Author{9}{Affiliation}#=%=#HuaweiCo.Ltd Author{10}{Firstname}#=%=#Yilun Author{10}{Lastname}#=%=#Liu Author{10}{Username}#=%=#lunyiliu Author{10}{Email}#=%=#liuyilun3@huawei.com Author{10}{Affiliation}#=%=#2012 Lab, Huawei co. LTD Author{11}{Firstname}#=%=#Feiyu Author{11}{Lastname}#=%=#Yao Author{11}{Email}#=%=#yaofeiyu1@huawei.com Author{11}{Affiliation}#=%=#HuaweiCo.Ltd Author{12}{Firstname}#=%=#shimin Author{12}{Lastname}#=%=#tao Author{12}{Username}#=%=#taoshimin Author{12}{Email}#=%=#taoshimin@huawei.com Author{12}{Affiliation}#=%=#huawei Author{13}{Firstname}#=%=#Hao Author{13}{Lastname}#=%=#Yang Author{13}{Username}#=%=#yanghaocsg Author{13}{Email}#=%=#yanghao30@huawei.com Author{13}{Affiliation}#=%=#Huawei Co. Ltd Author{14}{Firstname}#=%=#Yanfei Author{14}{Lastname}#=%=#Jiang Author{14}{Email}#=%=#jiangyanfei@huawei.com Author{14}{Affiliation}#=%=#HuaweiCo.Ltd ========== èéáğö