Prompt Engineering for Named Entity Extraction from Portuguese Legal Documents

Giovanni Maffeo, Catarina Silva, Hugo Gonçalo Oliveira


Abstract
The growing volume and complexity of legal texts highlight the need for automatic methods capable of extracting structured information from unstructured documents. Motivated by the limited availability and high cost of annotated legal data, this challenge is even more severe for the Portuguese language. This work investigates whether prompt engineering over Large Language Models (LLMs) can effectively support legal Named Entity Recognition (NER) in low-supervision and low-resource settings through In-Context Learning (ICL). Using the LeNER-Br corpus, we evaluate category-specific prompts, different chunking sizes, and prompt engineering strategies. Entity-level evaluation using Exact Match Micro F1 shows that prompt engineering has a stronger impact on performance than other strategies. The best results were obtained with larger models, the 4-bit quantised Qwen-2.5:32B and GPT-5.2, achieving scores of 57.9% and 71.9%, respectively, highlighting the potential of this approach as an alternative to traditional supervised NER pipelines.
Anthology ID:
2026.propor-1.116
Volume:
Proceedings of the 17th International Conference on Computational Processing of Portuguese (PROPOR 2026) - Vol. 1
Month:
April
Year:
2026
Address:
Salvador, Brazil
Editors:
Marlo Souza, Iria de-Dios-Flores, Diana Santos, Larissa Freitas, Jackson Wilke da Cruz Souza, Eugénio Ribeiro
Venue:
PROPOR
SIG:
Publisher:
Association for Computational Linguistics
Note:
Pages:
1092–1097
Language:
URL:
https://preview.aclanthology.org/ingest-dnd/2026.propor-1.116/
DOI:
Bibkey:
Cite (ACL):
Giovanni Maffeo, Catarina Silva, and Hugo Gonçalo Oliveira. 2026. Prompt Engineering for Named Entity Extraction from Portuguese Legal Documents. In Proceedings of the 17th International Conference on Computational Processing of Portuguese (PROPOR 2026) - Vol. 1, pages 1092–1097, Salvador, Brazil. Association for Computational Linguistics.
Cite (Informal):
Prompt Engineering for Named Entity Extraction from Portuguese Legal Documents (Maffeo et al., PROPOR 2026)
Copy Citation:
PDF:
https://preview.aclanthology.org/ingest-dnd/2026.propor-1.116.pdf