Quanyou Chu
2026
Memory-Guided Hard Data Augmentation for Multimodal Named Entity Recognition
Xinyu Liu | Kai fu | Yinghan Shi | Quanyou Chu | Ming Du | Hongya Wang | Xiaojun Meng | Jiansheng Wei | Yanghua Xiao | Bo Xu
Findings of the Association for Computational Linguistics: ACL 2026
Xinyu Liu | Kai fu | Yinghan Shi | Quanyou Chu | Ming Du | Hongya Wang | Xiaojun Meng | Jiansheng Wei | Yanghua Xiao | Bo Xu
Findings of the Association for Computational Linguistics: ACL 2026
Multimodal Named Entity Recognition relies on visual context to resolve textual ambiguities. To mitigate data scarcity, Data Augmentation (DA) has become a standard practice; however, existing methods predominantly adopt a one-size-fits-all and random perturbation paradigm, ignoring the internal state of the target model. In this paper, we first conduct a quantitative analysis, revealing that a significant portion of errors (over 30%) are model-specific, stemming from the unique biases of different architectures. To address this, we propose Memory-Guided Hard Data Augmentation, a framework designed to systematically repair these specific defects. First, we employ K-fold cross-validation to identify model-specific Hard Data. Second, we construct a Memory Tree and utilize Large Language Models (LLMs) with a clustering mechanism to induce macro-level error patterns from micro-level failures. This facilitates a paradigm shift from stateless instance-driven augmentation to a logical pattern-driven approach. Finally, we introduce an iterative augmentation mechanism that triggers recursive generation for stubborn instances that fail initial quality filters. Extensive experiments on Twitter-2015 and Twitter-2017 benchmarks demonstrate that our framework consistently yields significant performance gains across various MNER backbones.