Abstract
This study investigates Semantic TextualRelated- ness (STR) within Natural LanguageProcessing (NLP) through experiments conducted on a dataset from the SemEval-2024STR task. The dataset comprises train instances with three features (PairID, Text, andScore) and test instances with two features(PairID and Text), where sentence pairs areseparated by '/n’ in the Text column. UsingBERT(sentence transformers pipeline), we explore two approaches: one with fine-tuning(Track A: Supervised) and another without finetuning (Track B: UnSupervised). Fine-tuningthe BERT pipeline yielded a Spearman correlation coefficient of 0.803, while without finetuning, a coefficient of 0.693 was attained usingcosine similarity. The study concludes by emphasizing the significance of STR in NLP tasks,highlighting the role of pre-trained languagemodels like BERT and Sentence Transformersin enhancing semantic relatedness assessments.- Anthology ID:
- 2024.semeval-1.129
- Volume:
- Proceedings of the 18th International Workshop on Semantic Evaluation (SemEval-2024)
- Month:
- June
- Year:
- 2024
- Address:
- Mexico City, Mexico
- Editors:
- Atul Kr. Ojha, A. Seza Doğruöz, Harish Tayyar Madabushi, Giovanni Da San Martino, Sara Rosenthal, Aiala Rosá
- Venue:
- SemEval
- SIG:
- SIGLEX
- Publisher:
- Association for Computational Linguistics
- Note:
- Pages:
- 902–906
- Language:
- URL:
- https://aclanthology.org/2024.semeval-1.129
- DOI:
- Cite (ACL):
- Anand Kumar and Hemanth Kumar. 2024. scaLAR SemEval-2024 Task 1: Semantic Textual Relatednes for English. In Proceedings of the 18th International Workshop on Semantic Evaluation (SemEval-2024), pages 902–906, Mexico City, Mexico. Association for Computational Linguistics.
- Cite (Informal):
- scaLAR SemEval-2024 Task 1: Semantic Textual Relatednes for English (Kumar & Kumar, SemEval 2024)
- PDF:
- https://preview.aclanthology.org/ingestion-checklist/2024.semeval-1.129.pdf