2025
pdf
bib
abs
Towards Reliable Retrieval in RAG Systems for Large Legal Datasets
Markus Reuter
|
Tobias Lingenberg
|
Ruta Liepina
|
Francesca Lagioia
|
Marco Lippi
|
Giovanni Sartor
|
Andrea Passerini
|
Burcu Sayin
Proceedings of the Natural Legal Language Processing Workshop 2025
Retrieval-Augmented Generation (RAG) is a promising approach to mitigate hallucinations in Large Language Models (LLMs) for legal applications, but its reliability is critically dependent on the accuracy of the retrieval step. This is particularly challenging in the legal domain, where large databases of structurally similar documents often cause retrieval systems to fail. In this paper, we address this challenge by first identifying and quantifying a critical failure mode we term Document-Level Retrieval Mismatch (DRM), where the retriever selects information from entirely incorrect source documents. To mitigate DRM, we investigate a simple and computationally efficient technique which we refer to as Summary-Augmented Chunking (SAC). This method enhances each text chunk with a document-level synthetic summary, thereby injecting crucial global context that would otherwise be lost during a standard chunking process. Our experiments on a diverse set of legal information retrieval tasks show that SAC greatly reduces DRM and, consequently, also improves text-level retrieval precision and recall. Interestingly, we find that a generic summarization strategy outperforms an approach that incorporates legal expert domain knowledge to target specific legal elements. Our work provides evidence that this practical, scalable, and easily integrable technique enhances the reliability of RAG systems when applied to large-scale legal document datasets.
2021
pdf
bib
abs
A Corpus for Multilingual Analysis of Online Terms of Service
Kasper Drawzeski
|
Andrea Galassi
|
Agnieszka Jablonowska
|
Francesca Lagioia
|
Marco Lippi
|
Hans Wolfgang Micklitz
|
Giovanni Sartor
|
Giacomo Tagiuri
|
Paolo Torroni
Proceedings of the Natural Legal Language Processing Workshop 2021
We present the first annotated corpus for multilingual analysis of potentially unfair clauses in online Terms of Service. The data set comprises a total of 100 contracts, obtained from 25 documents annotated in four different languages: English, German, Italian, and Polish. For each contract, potentially unfair clauses for the consumer are annotated, for nine different unfairness categories. We show how a simple yet efficient annotation projection technique based on sentence embeddings could be used to automatically transfer annotations across languages.
2020
pdf
bib
abs
Cross-lingual Annotation Projection in Legal Texts
Andrea Galassi
|
Kasper Drazewski
|
Marco Lippi
|
Paolo Torroni
Proceedings of the 28th International Conference on Computational Linguistics
We study annotation projection in text classification problems where source documents are published in multiple languages and may not be an exact translation of one another. In particular, we focus on the detection of unfair clauses in privacy policies and terms of service. We present the first English-German parallel asymmetric corpus for the task at hand. We study and compare several language-agnostic sentence-level projection methods. Our results indicate that a combination of word embeddings and dynamic time warping performs best.
2018
pdf
bib
abs
Argumentative Link Prediction using Residual Networks and Multi-Objective Learning
Andrea Galassi
|
Marco Lippi
|
Paolo Torroni
Proceedings of the 5th Workshop on Argument Mining
We explore the use of residual networks for argumentation mining, with an emphasis on link prediction. The method we propose makes no assumptions on document or argument structure. We evaluate it on a challenging dataset consisting of user-generated comments collected from an online platform. Results show that our model outperforms an equivalent deep network and offers results comparable with state-of-the-art methods that rely on domain knowledge.
pdf
bib
abs
Predicting the Usefulness of Amazon Reviews Using Off-The-Shelf Argumentation Mining
Marco Passon
|
Marco Lippi
|
Giuseppe Serra
|
Carlo Tasso
Proceedings of the 5th Workshop on Argument Mining
Internet users generate content at unprecedented rates. Building intelligent systems capable of discriminating useful content within this ocean of information is thus becoming a urgent need. In this paper, we aim to predict the usefulness of Amazon reviews, and to do this we exploit features coming from an off-the-shelf argumentation mining system. We argue that the usefulness of a review, in fact, is strictly related to its argumentative content, whereas the use of an already trained system avoids the costly need of relabeling a novel dataset. Results obtained on a large publicly available corpus support this hypothesis.