@inproceedings{dadason-steingrimsson-2026-optimized,
    title = "Optimized for {AI}: Curating the {I}celandic {G}igaword Corpus for Stable {LLM} Training",
    author = "Da{\dh}ason, J{\'o}n Fri{\dh}rik  and
      Steingr{\'i}msson, Stein{\th}{\'o}r",
    editor = "Ba{\'n}ski, Piotr  and
      Knight, Dawn  and
      Kupietz, Marc  and
      Witt, Andreas  and
      Wr{\'o}blewska, Alina",
    booktitle = "Proceedings of the 12th Workshop on Challenges in the Management of Large Corpora",
    month = may,
    year = "2026",
    address = "Palma, Mallorca (Spain)",
    publisher = "ELRA Language Resources Association (ELRA)",
    url = "https://preview.aclanthology.org/ingest-nlpsi/2026.cmlc-1.6/",
    doi = "10.63317/3uatbht8mdrf",
    pages = "49--56",
    abstract = "The Icelandic Gigaword Corpus (IGC) is a primary resource for Icelandic NLP, with its current version containing 2.7 billion words of curated text. The IGC is traditionally distributed in a TEI-XML format, a hierarchical structure that allows for rich linguistic annotation and metadata. However, this format introduces significant friction for modern machine learning workflows. Even high-quality curated corpora have been found to contain ``unwanted'' text sequences {--} such as fragmented lists or repetitive boilerplate that may trigger instabilities during training of large language models. In this paper, we present a new processing pipeline designed to optimize the IGC for AI development. We describe a filtering approach focusing on training stability, including fuzzy deduplication to reduce the risk of data leakage, with the aim to provide high-quality data for stable model convergence. Furthermore, we introduce a new JSONL distribution format that bridges the gap between TEI-XML and machine-actionable data, facilitating easier access and safer training for models aiming to work with Icelandic."
}