Post-Correction of Historical Text Transcripts with Large Language Models: An Exploratory Study
Emanuela Boros, Maud Ehrmann, Matteo Romanello, Sven Najem-Meyer, Frédéric Kaplan
Abstract
The quality of automatic transcription of heritage documents, whether from printed, manuscripts or audio sources, has a decisive impact on the ability to search and process historical texts. Although significant progress has been made in text recognition (OCR, HTR, ASR), textual materials derived from library and archive collections remain largely erroneous and noisy. Effective post-transcription correction methods are therefore necessary and have been intensively researched for many years. As large language models (LLMs) have recently shown exceptional performances in a variety of text-related tasks, we investigate their ability to amend poor historical transcriptions. We evaluate fourteen foundation language models against various post-correction benchmarks comprising different languages, time periods and document types, as well as different transcription quality and origins. We compare the performance of different model sizes and different prompts of increasing complexity in zero and few-shot settings. Our evaluation shows that LLMs are anything but efficient at this task. Quantitative and qualitative analyses of results allow us to share valuable insights for future work on post-correcting historical texts with LLMs.- Anthology ID:
- 2024.latechclfl-1.14
- Volume:
- Proceedings of the 8th Joint SIGHUM Workshop on Computational Linguistics for Cultural Heritage, Social Sciences, Humanities and Literature (LaTeCH-CLfL 2024)
- Month:
- March
- Year:
- 2024
- Address:
- St. Julians, Malta
- Editors:
- Yuri Bizzoni, Stefania Degaetano-Ortlieb, Anna Kazantseva, Stan Szpakowicz
- Venues:
- LaTeCHCLfL | WS
- SIG:
- Publisher:
- Association for Computational Linguistics
- Note:
- Pages:
- 133–159
- Language:
- URL:
- https://aclanthology.org/2024.latechclfl-1.14
- DOI:
- Cite (ACL):
- Emanuela Boros, Maud Ehrmann, Matteo Romanello, Sven Najem-Meyer, and Frédéric Kaplan. 2024. Post-Correction of Historical Text Transcripts with Large Language Models: An Exploratory Study. In Proceedings of the 8th Joint SIGHUM Workshop on Computational Linguistics for Cultural Heritage, Social Sciences, Humanities and Literature (LaTeCH-CLfL 2024), pages 133–159, St. Julians, Malta. Association for Computational Linguistics.
- Cite (Informal):
- Post-Correction of Historical Text Transcripts with Large Language Models: An Exploratory Study (Boros et al., LaTeCHCLfL-WS 2024)
- PDF:
- https://preview.aclanthology.org/nschneid-patch-3/2024.latechclfl-1.14.pdf