@inproceedings{holgate-etal-2025-fine,
    title = "Fine-tuning {LLM}s to Extract Epilepsy Seizure Frequency Data from Health Records",
    author = "Holgate, Ben  and
      Davies, Joe  and
      Fang, Shichao  and
      Winston, Joel  and
      Teo, James  and
      Richardson, Mark",
    editor = "Demner-Fushman, Dina  and
      Ananiadou, Sophia  and
      Miwa, Makoto  and
      Tsujii, Junichi",
    booktitle = "Proceedings of the 24th Workshop on Biomedical Language Processing",
    month = aug,
    year = "2025",
    address = "Viena, Austria",
    publisher = "Association for Computational Linguistics",
    url = "https://preview.aclanthology.org/acl25-workshop-ingestion/2025.bionlp-1.5/",
    pages = "44--55",
    ISBN = "979-8-89176-275-6",
    abstract = "We developed a new methodology of extracting the frequency of a patient{'}s epilepsy seizures from unstructured, free-text outpatient clinic letters by: first, devising a singular unit of measurement for seizure frequency; and second, fine-tuning a generative Large Language Model (LLM) on our bespoke annotated dataset. We measured frequency by the number of seizures per month: one seizure or more requires an integer; and less than one a decimal. This approach enables us to track whether a patient{''}s seizures are improving or not over time. We found fine-tuning improves the F1 score of our best-performing LLM, Ministral-8B-Instruct-2410, by around three times compared to an untrained model. We also found Ministral demonstrated an impressive ability for mathematical reasoning."
}