SubmissionNumber#=%=#224 FinalPaperTitle#=%=#SemEval-2024 Task 7: Numeral-Aware Language Understanding and Generation ShortPaperTitle#=%=# NumberOfPages#=%=#10 CopyrightSigned#=%=#Chung-Chi Chen JobTitle#==# Organization#==# Abstract#==#Numbers are frequently utilized in both our daily narratives and professional documents, such as clinical notes, scientific papers, financial documents, and legal court orders. The ability to understand and generate numbers is thus one of the essential aspects of evaluating large language models. In this vein, we propose a collection of datasets in SemEval-2024 Task 7 - NumEval. This collection encompasses several tasks focused on numeral-aware instances, including number prediction, natural language inference, question answering, reading comprehension, reasoning, and headline generation. This paper offers an overview of the dataset and presents the results of all subtasks in NumEval. Additionally, we contribute by summarizing participants' methods and conducting an error analysis. To the best of our knowledge, NumEval represents one of the early tasks that perform peer evaluation in SemEval's history. We will further share observations from this aspect and provide suggestions for future SemEval tasks. Author{1}{Firstname}#=%=#Chung-Chi Author{1}{Lastname}#=%=#Chen Author{1}{Username}#=%=#chunji Author{1}{Email}#=%=#chunji0112@gmail.com Author{1}{Affiliation}#=%=#National Institute of Advanced Industrial Science and Technology Author{2}{Firstname}#=%=#Jian-Tao Author{2}{Lastname}#=%=#Huang Author{2}{Username}#=%=#arrowhuang Author{2}{Email}#=%=#wzhjttw@gmail.com Author{2}{Affiliation}#=%=#Zhejiang Lab Author{3}{Firstname}#=%=#Hen-Hsen Author{3}{Lastname}#=%=#Huang Author{3}{Username}#=%=#hhhuang Author{3}{Email}#=%=#hhhuang@iis.sinica.edu.tw Author{3}{Affiliation}#=%=#Institute of Information Science, Academia Sinica Author{4}{Firstname}#=%=#Hiroya Author{4}{Lastname}#=%=#Takamura Author{4}{Username}#=%=#takamura Author{4}{Email}#=%=#takamura.hiroya@aist.go.jp Author{4}{Affiliation}#=%=#The National Institute of Advanced Industrial Science and Technology (AIST) Author{5}{Firstname}#=%=#Hsin-Hsi Author{5}{Lastname}#=%=#Chen Author{5}{Username}#=%=#hhchen Author{5}{Email}#=%=#hhchen@ntu.edu.tw Author{5}{Affiliation}#=%=#National Taiwan University ========== èéáğö