@inproceedings{frew-etal-2026-fhirpath,
    title = "{FHIRP}ath-{QA}: Executable Question Answering over {FHIR} Electronic Health Records",
    author = "Frew, Michael  and
      Bheda, Nishit  and
      Tripp, Bryan",
    editor = "Gupta, Deepak  and
      Thompson, Paul  and
      Ananiadou, Sophia  and
      Demner-Fushman, Dina",
    booktitle = "Proceedings of the Third Workshop on Patient-Oriented Language Processing ({CL}4{H}ealth) @ {LREC} 2026",
    month = may,
    year = "2026",
    address = "Palma, Mallorca (Spain)",
    publisher = "ELRA Language Resources Association (ELRA)",
    url = "https://preview.aclanthology.org/test-year-match/2026.cl4health-1.1/",
    doi = "10.63317/5q8qi6jqnz56",
    pages = "1--14",
    abstract = "Though patients are increasingly granted digital access to their electronic health records (EHRs), existing interfaces may not support precise, trustworthy answers to patient-specific questions. Large language models (LLM) show promise in clinical question answering (QA), but retrieval-based approaches are computationally inefficient, prone to hallucination, and difficult to deploy over real-life EHRs. This work introduces FHIRPath-QA, the first open dataset and benchmark for patient-specific QA that includes open-standard FHIRPath queries over real-world clinical data. A text-to-FHIRPath QA paradigm is proposed that shifts reasoning from free-text generation to FHIRPath query synthesis. For o4-mini, this reduced average token usage by 391$\times$ relative to retrieval-first prompting (629,829 vs 1,609 tokens per question) and lowered failure rates from 0.36 to 0.09 on clinician-phrased questions. Built on MIMIC-IV on FHIR Demo, the dataset pairs over 14k natural language questions in patient and clinician phrasing with validated FHIRPath queries and answers. Empirically, the evaluated LLMs achieve at most 42{\%} accuracy, highlighting the challenge of the task, but benefit strongly from supervised fine-tuning, with query synthesis accuracy improving from 27{\%} to 79{\%} for 4o-mini. These results highlight that text-to-FHIRPath synthesis has the potential to serve as a practical foundation for safe, efficient, and interoperable consumer health applications, and the FHIRPath-QA dataset and benchmark serve as a starting point for future research on the topic. The full dataset and generation code can be accessed on \href{https://github.com/mooshifrew/fhirpath-qa}{GitHub}."
}