@inproceedings{ulli-etal-2026-aurum,
    title = "Aurum at {CRF} Filling 2026: Modular {DSP}y Extractors with Qwen3-Max for Multilingual {CRF} Filling",
    author = "Ulli, Vinay Babu  and
      Kumari, Jyoti  and
      Mondal, Anindita",
    editor = "Gupta, Deepak  and
      Thompson, Paul  and
      Ananiadou, Sophia  and
      Demner-Fushman, Dina",
    booktitle = "Proceedings of the Third Workshop on Patient-Oriented Language Processing ({CL}4{H}ealth) @ {LREC} 2026",
    month = may,
    year = "2026",
    address = "Palma, Mallorca (Spain)",
    publisher = "ELRA Language Resources Association (ELRA)",
    url = "https://preview.aclanthology.org/ingest-nlpsi/2026.cl4health-1.36/",
    doi = "10.63317/4h5ugiqk5eht",
    pages = "395--401",
    abstract = "This paper describes the submission by Team Aurum to the CL4Health @ LREC 2026 Shared Task on Case Report Form (CRF) Filling from dyspnea patient clinical notes. Extracting 134 structured clinical fields using a single Large Language Model (LLM) call often leads to schema-following errors, hallucination, and poor attention over complex instructions. To address this, we propose a modular extraction pipeline built with DSPy, which decomposes the 134 CRF fields into 14 specialized, domain-specific extractors (e.g., Medical History, Lab Values, Acute Diagnoses). We conducted extensive experiments across multiple multilingual LLMs, including Llama4 Maverik, GPT-4o, GPT-4o Mini, DeepSeek-V3, Gemma-3-12B-Instruct, and Qwen-series models. Among these, Qwen3-Max (Thinking) with our optimized v2 prompts achieved the best performance on the development set with a Macro-F1 of 0.70, outperforming other evaluated models such as GPT-4o (0.68) and DeepSeek-V3 (0.66). Prompt optimization resulted in measurable gains, improving Qwen3-Max performance from 0.67 to 0.70. Using this configuration, our pipeline achieved an official Codabench Test Macro-F1 score of 0.68 in English and 0.67 in Italian, securing the 1st place ranking overall in the shared task."
}