Learning Dynamic Representations and Policies from Multimodal Clinical Time-Series with Informative Missingness

Zihan Liang, Ziwen Pan, Ruoxuan Xiong


Abstract
Multimodal clinical records contain structured measurements and clinical notes recorded over time, offering rich temporal information about the evolution of patient health. Yet these observations are sparse, and whether they are recorded depends on the patient’s latent condition. Observation patterns also differ across modalities, as structured measurements and clinical notes arise under distinct recording processes. While prior work has developed methods that accommodate missingness in clinical time series, how to extract and use the information carried by the observation process itself remains underexplored. We therefore propose a patient representation learning framework for multimodal clinical time series that explicitly leverages informative missingness. The framework combines (1) a multimodal encoder that captures signals from structured and textual data together with their observation patterns, (2) a Bayesian filtering module that updates a latent patient state over time from observed multimodal signals, and (3) downstream modules for offline treatment policy learning and patient outcome prediction based on the learned patient state. We evaluate the framework on ICU sepsis cohorts from MIMIC-III, MIMIC-IV, and eICU. It improves both offline treatment policy learning and adverse outcome prediction, achieving FQE 0.679 versus 0.528 for clinician behavior and AUROC 0.886 for post-72-hour mortality prediction on MIMIC-III.
Anthology ID:
2026.findings-acl.1313
Volume:
Findings of the Association for Computational Linguistics: ACL 2026
Month:
July
Year:
2026
Address:
San Diego, California, United States
Editors:
Maria Liakata, Viviane P. Moreira, Jiajun Zhang, David Jurgens
Venue:
Findings
SIG:
Publisher:
Association for Computational Linguistics
Note:
Pages:
26363–26392
Language:
URL:
https://preview.aclanthology.org/ingest-acl/2026.findings-acl.1313/
DOI:
Bibkey:
Cite (ACL):
Zihan Liang, Ziwen Pan, and Ruoxuan Xiong. 2026. Learning Dynamic Representations and Policies from Multimodal Clinical Time-Series with Informative Missingness. In Findings of the Association for Computational Linguistics: ACL 2026, pages 26363–26392, San Diego, California, United States. Association for Computational Linguistics.
Cite (Informal):
Learning Dynamic Representations and Policies from Multimodal Clinical Time-Series with Informative Missingness (Liang et al., Findings 2026)
Copy Citation:
PDF:
https://preview.aclanthology.org/ingest-acl/2026.findings-acl.1313.pdf
Checklist:
 2026.findings-acl.1313.checklist.pdf