SubmissionNumber#=%=#137
FinalPaperTitle#=%=#Noot Noot at SemEval-2024 Task 7: Numerical Reasoning and Headline Generation
ShortPaperTitle#=%=#
NumberOfPages#=%=#5
CopyrightSigned#=%=#Sankalp Bahad
JobTitle#==#
Organization#==#
Abstract#==#Natural language processing (NLP) models
have achieved remarkable progress in recent
years, particularly in tasks related to semantic
analysis. However, many existing benchmarks
primarily focus on lexical and syntactic un-
derstanding, often overlooking the importance
of numerical reasoning abilities. In this pa-
per, we argue for the necessity of incorporating
numeral-awareness into NLP evaluations and
propose two distinct tasks to assess this capabil-
ity: Numerical Reasoning and Headline Gener-
ation. We present datasets curated for each task
and evaluate various approaches using both au-
tomatic and human evaluation metrics. Our
results demonstrate the diverse strategies em-
ployed by participating teams and highlight the
promising performance of emerging models
like Mixtral 8x7b instruct. We discuss the im-
plications of our findings and suggest avenues
for future research in advancing numeral-aware
language understanding and generation.
Author{1}{Firstname}#=%=#Sankalp Sanjay
Author{1}{Lastname}#=%=#Bahad
Author{1}{Username}#=%=#sankalp_bahad
Author{1}{Email}#=%=#sankalp.bahad@research.iiit.ac.in
Author{1}{Affiliation}#=%=#IIIT Hyderabad
Author{2}{Firstname}#=%=#Yash
Author{2}{Lastname}#=%=#Bhaskar
Author{2}{Email}#=%=#yash.bhaskar@research.iiit.ac.in
Author{2}{Affiliation}#=%=#IIIT Hyderabad
Author{3}{Firstname}#=%=#Parameswari
Author{3}{Lastname}#=%=#Krishnamurthy
Author{3}{Username}#=%=#parameswari
Author{3}{Email}#=%=#parameshkrishnaa@gmail.com
Author{3}{Affiliation}#=%=#Assistant Professor, IIIT Hyderabad

==========
èéáğö