Sicheng Gao
2026
e5-omni: Explicit Cross-modal Alignment for Omni-modal Embeddings
Haonan Chen | Sicheng Gao | Radu Timofte | Tetsuya Sakai | Zhicheng Dou
Findings of the Association for Computational Linguistics: ACL 2026
Haonan Chen | Sicheng Gao | Radu Timofte | Tetsuya Sakai | Zhicheng Dou
Findings of the Association for Computational Linguistics: ACL 2026
Modern information systems often involve different types of items, , a text query, an image, a video clip, or an audio segment. This motivates omni-modal embedding models that map heterogeneous modalities into a shared space for direct comparison. However, most recent omni-modal embeddings still rely heavily on implicit alignment inherited from pretrained vision-language model (VLM) backbones. In practice, this causes three common issues: (i) similarity logits have modality-dependent sharpness, so scores are not on a consistent scale; (ii) in-batch negatives become less effective over time because mixed-modality batches create an imbalanced hardness distribution; as a result, many negatives quickly become trivial and contribute little gradient; and (iii) embeddings across modalities show mismatched first- and second-order statistics, which makes rankings less stable. To tackle these problems, we propose e5-omni, a lightweight explicit alignment recipe that adapts off-the-shelf VLMs into robust omni-modal embedding models. e5-omni combines three simple components: (1) modality-aware temperature calibration to align similarity scales, (2) a controllable negative curriculum with debiasing to focus on confusing negatives while reducing the impact of false negatives, and (3) batch whitening with covariance regularization to better match cross-modal geometry in the shared embedding space. Experiments on MMEB-V2 and AudioCaps show consistent gains over strong bi-modal and omni-modal baselines, and the same recipe also transfers well to other VLM backbones. We release our model checkpoint at https://huggingface.co/collections/Haon-Chen/e5-omni.
AdaTooler-V: Adaptive Tool-Use for Images and Videos
Chaoyang Wang | Kaituo Feng | Dongyang Chen | Zhongyu Wang | Zhixun Li | Sicheng Gao | Meng Meng | Xu Zhou | Manyuan Zhang | Yuzhang Shang | Xiangyu Yue
Findings of the Association for Computational Linguistics: ACL 2026
Chaoyang Wang | Kaituo Feng | Dongyang Chen | Zhongyu Wang | Zhixun Li | Sicheng Gao | Meng Meng | Xu Zhou | Manyuan Zhang | Yuzhang Shang | Xiangyu Yue
Findings of the Association for Computational Linguistics: ACL 2026
Recent advances have shown that multimodal large language models (MLLMs) benefit from multimodal interleaved chain-of-thought (CoT) with vision tool interactions. However, existing open-source models often exhibit blind tool-use reasoning patterns, invoking vision tools even when they are unnecessary, which significantly increases inference overhead and degrades model performance. To this end, we propose AdaTooler-V, an MLLM that performs adaptive tool-use by determining whether a visual problem truly requires tools. First, we introduce AT-GRPO, a reinforcement learning algorithm that adaptively adjusts reward scales based on the Tool Benefit Score of each sample, encouraging the model to invoke tools only when they provide genuine improvements. Moreover, we construct two datasets to support training: AdaTooler-V-CoT-100k for SFT cold start and AdaTooler-V-300k for RL with verifiable rewards across single-image, multi-image, and video data. Experiments across twelve benchmarks demonstrate the strong reasoning capability of AdaTooler-V, outperforming existing methods in diverse visual reasoning tasks. Notably, AdaTooler-V-7B achieves an accuracy of 89.8% on the high-resolution benchmark V*, surpassing the commercial proprietary model GPT-4o and Gemini 1.5 Pro.