@inproceedings{anuradha-etal-2025-holobert,
    title = "{H}olo{BERT}: Pre-Trained Transformer Model for Historical Narratives",
    author = "Anuradha, Isuri  and
      Ha, Le An  and
      Mitkov, Ruslan",
    editor = "Angelova, Galia  and
      Kunilovskaya, Maria  and
      Escribe, Marie  and
      Mitkov, Ruslan",
    booktitle = "Proceedings of the 15th International Conference on Recent Advances in Natural Language Processing - Natural Language Processing in the Generative AI Era",
    month = sep,
    year = "2025",
    address = "Varna, Bulgaria",
    publisher = "INCOMA Ltd., Shoumen, Bulgaria",
    url = "https://preview.aclanthology.org/corrections-2026-01/2025.ranlp-1.12/",
    pages = "105--110",
    abstract = "Oral texts often contain spontaneous, unstructured language with features like disfluencies, colloquialisms, and non-standard syntax. In this paper, we investigate how further pretraining language models with specialised learning objectives for oral and transcribed texts to enhance Named Entity Recognition (NER) performance in Holocaust-related discourse. To evaluate our models, we compare the extracted named entities (NE) against those from other pretrained models on historical texts and generative AI models such as GPT. Furthermore, we demonstrate practical applications of the recognised NEs by linking them to a knowledge base as structured metadata and representing them in a graph format. With these contributions, our work illustrates how the further-pretrain-and-fine-tune paradigm in Natural Language Processing advances research in Digital Humanities."
}