Abstract
We present a new NLP task and dataset from the domain of the U.S. civil procedure. Each instance of the dataset consists of a general introduction to the case, a particular question, and a possible solution argument, accompanied by a detailed analysis of why the argument applies in that case. Since the dataset is based on a book aimed at law students, we believe that it represents a truly complex task for benchmarking modern legal language models. Our baseline evaluation shows that fine-tuning a legal transformer provides some advantage over random baseline models, but our analysis reveals that the actual ability to infer legal arguments remains a challenging open research question.- Anthology ID:
- 2022.nllp-1.17
- Volume:
- Proceedings of the Natural Legal Language Processing Workshop 2022
- Month:
- December
- Year:
- 2022
- Address:
- Abu Dhabi, United Arab Emirates (Hybrid)
- Editors:
- Nikolaos Aletras, Ilias Chalkidis, Leslie Barrett, Cătălina Goanță, Daniel Preoțiuc-Pietro
- Venue:
- NLLP
- SIG:
- Publisher:
- Association for Computational Linguistics
- Note:
- Pages:
- 194–207
- Language:
- URL:
- https://aclanthology.org/2022.nllp-1.17
- DOI:
- 10.18653/v1/2022.nllp-1.17
- Cite (ACL):
- Leonard Bongard, Lena Held, and Ivan Habernal. 2022. The Legal Argument Reasoning Task in Civil Procedure. In Proceedings of the Natural Legal Language Processing Workshop 2022, pages 194–207, Abu Dhabi, United Arab Emirates (Hybrid). Association for Computational Linguistics.
- Cite (Informal):
- The Legal Argument Reasoning Task in Civil Procedure (Bongard et al., NLLP 2022)
- PDF:
- https://preview.aclanthology.org/nschneid-patch-4/2022.nllp-1.17.pdf