Jai Riley
2024
UAlberta at SemEval-2024 Task 1: A Potpourri of Methods for Quantifying Multilingual Semantic Textual Relatedness and Similarity
Ning Shi
|
Senyu Li
|
Guoqing Luo
|
Amirreza Mirzaei
|
Ali Rafiei
|
Jai Riley
|
Hadi Sheikhi
|
Mahvash Siavashpour
|
Mohammad Tavakoli
|
Bradley Hauer
|
Grzegorz Kondrak
Proceedings of the 18th International Workshop on Semantic Evaluation (SemEval-2024)
We describe our systems for SemEval-2024 Task 1: Semantic Textual Relatedness. We investigate the correlation between semantic relatedness and semantic similarity. Specifically, we test two hypotheses: (1) similarity is a special case of relatedness, and (2) semantic relatedness is preserved under translation. We experiment with a variety of approaches which are based on explicit semantics, downstream applications, contextual embeddings, large language models (LLMs), as well as ensembles of methods. We find empirical support for our theoretical insights. In addition, our best ensemble system yields highly competitive results in a number of diverse categories. Our code and data are available on GitHub.
Paraphrase Identification via Textual Inference
Ning Shi
|
Bradley Hauer
|
Jai Riley
|
Grzegorz Kondrak
Proceedings of the 13th Joint Conference on Lexical and Computational Semantics (*SEM 2024)
Paraphrase identification (PI) and natural language inference (NLI) are two important tasks in natural language processing. Despite their distinct objectives, an underlying connection exists, which has been notably under-explored in empirical investigations. We formalize the relationship between these semantic tasks and introduce a method for solving PI using an NLI system, including the adaptation of PI datasets for fine-tuning NLI models. Through extensive evaluations on six PI benchmarks, across both zero-shot and fine-tuned settings, we showcase the efficacy of NLI models for PI through our proposed reduction. Remarkably, our fine-tuning procedure enables NLI models to outperform dedicated PI models on PI datasets. In addition, our findings provide insights into the limitations of current PI benchmarks.
Search
Co-authors
- Ning Shi 2
- Bradley Hauer 2
- Grzegorz Kondrak 2
- Senyu Li 1
- Guoqing Luo 1
- show all...