Dynamic Meta-Metrics: Source-Sentence Conditioned Weighting for MT Evaluation

Luke Zhang, Justin Vasselli, Aditya Khan, York Hay Ng, En-Shiun Annie Lee


Abstract
We propose Dynamic Meta-Metrics (DMM), a framework for machine translation evaluation that learns source-sentence conditioned combinations of existing metrics. Rather than relying on a single static ensemble or language-specific weighting, DMM adapts the metric combination based on properties of the source segment. We study hard conditioning, which fits an interpretable combiner per cluster, and an exploratory soft-conditioned extension whose weights vary continuously with source-cluster responsibilities. We evaluate DMM on the WMT Metrics Shared Task data across multiple language pairs using pairwise agreement measures at the system and segment levels. Across settings, MLP-based combinations outperform linear and Gaussian process-based ensembles, and introducing soft conditioning yields gains over linear models.
Anthology ID:
2026.acl-srw.80
Volume:
Proceedings of the 64th Annual Meeting of the Association for Computational Linguistics (ACL 2026)
Month:
July
Year:
2026
Address:
San Diego, California, United States
Editors:
Santosh T.Y.S.S., Juan Diego Rodriguez, Ona de Gibert
Venue:
ACL
SIG:
Publisher:
Association for Computational Linguistics
Note:
Pages:
903–912
Language:
URL:
https://preview.aclanthology.org/ingest-acl/2026.acl-srw.80/
DOI:
Bibkey:
Cite (ACL):
Luke Zhang, Justin Vasselli, Aditya Khan, York Hay Ng, and En-Shiun Annie Lee. 2026. Dynamic Meta-Metrics: Source-Sentence Conditioned Weighting for MT Evaluation. In Proceedings of the 64th Annual Meeting of the Association for Computational Linguistics (ACL 2026), pages 903–912, San Diego, California, United States. Association for Computational Linguistics.
Cite (Informal):
Dynamic Meta-Metrics: Source-Sentence Conditioned Weighting for MT Evaluation (Zhang et al., ACL 2026)
Copy Citation:
PDF:
https://preview.aclanthology.org/ingest-acl/2026.acl-srw.80.pdf