2025
pdf
bib
abs
How to Compare Things Properly? A Study of Argument Relevance in Comparative Question Answering
Irina Nikishina
|
Saba Anwar
|
Nikolay Dolgov
|
Maria Manina
|
Daria Ignatenko
|
Artem Shelmanov
|
Chris Biemann
Proceedings of the 63rd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers)
Comparative Question Answering (CQA) lies at the intersection of Question Answering, Argument Mining, and Summarization. It poses unique challenges due to the inherently subjective nature of many questions and the need to integrate diverse perspectives. Although the CQA task can be addressed using recently emerged instruction-following Large Language Models (LLMs), challenges such as hallucinations in their outputs and the lack of transparent argument provenance remain significant limitations.To address these challenges, we construct a manually curated dataset comprising arguments annotated with their relevance. These arguments are further used to answer comparative questions, enabling precise traceability and faithfulness. Furthermore, we define explicit criteria for an “ideal” comparison and introduce a benchmark for evaluating the outputs of various Retrieval-Augmented Generation (RAG) models with respect to argument relevance. All code and data are publicly released to support further research.
2022
pdf
bib
More Like This: Semantic Retrieval with Linguistic Information
Steffen Remus
|
Gregor Wiedemann
|
Saba Anwar
|
Fynn Petersen-Frey
|
Seid Muhie Yimam
|
Chris Biemann
Proceedings of the 18th Conference on Natural Language Processing (KONVENS 2022)
2021
pdf
bib
abs
SCoT: Sense Clustering over Time: a tool for the analysis of lexical change
Christian Haase
|
Saba Anwar
|
Seid Muhie Yimam
|
Alexander Friedrich
|
Chris Biemann
Proceedings of the 16th Conference of the European Chapter of the Association for Computational Linguistics: System Demonstrations
We present Sense Clustering over Time (SCoT), a novel network-based tool for analysing lexical change. SCoT represents the meanings of a word as clusters of similar words. It visualises their formation, change, and demise. There are two main approaches to the exploration of dynamic networks: the discrete one compares a series of clustered graphs from separate points in time. The continuous one analyses the changes of one dynamic network over a time-span. SCoT offers a new hybrid solution. First, it aggregates time-stamped documents into intervals and calculates one sense graph per discrete interval. Then, it merges the static graphs to a new type of dynamic semantic neighbourhood graph over time. The resulting sense clusters offer uniquely detailed insights into lexical change over continuous intervals with model transparency and provenance. SCoT has been successfully used in a European study on the changing meaning of ‘crisis’.
2020
pdf
bib
abs
Generating Lexical Representations of Frames using Lexical Substitution
Saba Anwar
|
Artem Shelmanov
|
Alexander Panchenko
|
Chris Biemann
Proceedings of the Probability and Meaning Conference (PaM 2020)
Semantic frames are formal linguistic structures describing situations/actions/events, e.g. Commercial transfer of goods. Each frame provides a set of roles corresponding to the situation participants, e.g. Buyer and Goods, and lexical units (LUs) – words and phrases that can evoke this particular frame in texts, e.g. Sell. The scarcity of annotated resources hinders wider adoption of frame semantics across languages and domains. We investigate a simple yet effective method, lexical substitution with word representation models, to automatically expand a small set of frame-annotated sentences with new words for their respective roles and LUs. We evaluate the expansion quality using FrameNet. Contextualized models demonstrate overall superior performance compared to the non-contextualized ones on roles. However, the latter show comparable performance on the task of LU expansion.
2019
pdf
bib
abs
HHMM at SemEval-2019 Task 2: Unsupervised Frame Induction using Contextualized Word Embeddings
Saba Anwar
|
Dmitry Ustalov
|
Nikolay Arefyev
|
Simone Paolo Ponzetto
|
Chris Biemann
|
Alexander Panchenko
Proceedings of the 13th International Workshop on Semantic Evaluation
We present our system for semantic frame induction that showed the best performance in Subtask B.1 and finished as the runner-up in Subtask A of the SemEval 2019 Task 2 on unsupervised semantic frame induction (Qasem-iZadeh et al., 2019). Our approach separates this task into two independent steps: verb clustering using word and their context embeddings and role labeling by combining these embeddings with syntactical features. A simple combination of these steps shows very competitive results and can be extended to process other datasets and languages.