@inproceedings{nunes-etal-2026-twenty,
    title = "Twenty Years of {HAREM}: A Reproducible Audit and Reassessment of {P}ortuguese Named Entity Recognition",
    author = "Nunes, Rafael O.  and
      Spritzer, Andr{\'e}  and
      Freitas, Carla M. D. S.  and
      Balreira, Dennis G.",
    editor = "Souza, Marlo  and
      de-Dios-Flores, Iria  and
      Santos, Diana  and
      Freitas, Larissa  and
      Souza, Jackson Wilke da Cruz  and
      Ribeiro, Eug{\'e}nio",
    booktitle = "Proceedings of the 17th International Conference on Computational Processing of {P}ortuguese ({PROPOR} 2026) - Vol. 1",
    month = apr,
    year = "2026",
    address = "Salvador, Brazil",
    publisher = "Association for Computational Linguistics",
    url = "https://preview.aclanthology.org/ingest-dnd/2026.propor-1.35/",
    pages = "351--359",
    ISBN = "979-8-89176-387-6",
    abstract = "For two decades, the HAREM corpus has served as the foundational benchmark for Portuguese Named Entity Recognition (NER), establishing its evaluation paradigm. Virtually all major progress has been measured against its fixed train/test split. This paper presents the first systematic audit of this split, revealing 153 overlapping (contaminated) sentences. We re-evaluate 13 NER models (ranging from CRFs to Transformers) on both the original and a new, decontaminated version of the corpus. Our statistical analysis reveals that decontamination has a significant (p {\ensuremath{<}} 0.05) and positive impact on the majority of models. We find that performance gains are most pronounced in the F1{\_}text{macro} score (up to +4 points), demonstrating that the contamination primarily harmed generalization on rare entity types. Furthermore, our audit reveals clear evidence of overfitting in some models that benefited from data leakage. We conclude that even minor contamination can distort performance metrics and mask true model generalization. We release our decontaminated benchmark to ensure more reliable future evaluations."
}