Vu Minh Hieu Phan
2026
MMCLIP: Cross-Modal Attention Masked Modelling for Medical Language-Image Pre-Training
Biao Wu | Yutong Xie | Zeyu Zhang | Vu Minh Hieu Phan | Qi Chen | Ling Chen | Qi Wu
Proceedings of the 64th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers)
Biao Wu | Yutong Xie | Zeyu Zhang | Vu Minh Hieu Phan | Qi Chen | Ling Chen | Qi Wu
Proceedings of the 64th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers)
Vision-and-language pretraining (VLP) in medicine leverages contrastive learning on image–text pairs, often enhanced with masked modeling. However, existing methods face two challenges: difficulty reconstructing key pathological features due to limited data, and reliance on either paired or image-only datasets without combining both. To address this, we propose **MMCLIP** (**M**asked **M**edical **C**ontrastive **L**anguage–**I**mage **P**re-training), which introduces two modules: **AttMIM**: Masks image features highly correlated with text to improve reconstruction of fine medical details. **EntMLM**: Masks key medical entities in text and reconstructs them using visual cues. Furthermore, **MMCLIP** incorporates unpaired data through disease-kind prompts, achieving state-of-the-art performance in zero-shot and fine-tuning across five benchmarks.