@inproceedings{manna-etal-2025-paying,
    title = "Are We Paying Attention to Her? Investigating Gender Disambiguation and Attention in Machine Translation",
    author = "Manna, Chiara  and
      Alishahi, Afra  and
      Blain, Fr{\'e}d{\'e}ric  and
      Vanmassenhove, Eva",
    editor = "Hackenbuchner, Jani{\c{c}}a  and
      Bentivogli, Luisa  and
      Daems, Joke  and
      Manna, Chiara  and
      Savoldi, Beatrice  and
      Vanmassenhove, Eva",
    booktitle = "Proceedings of the 3rd Workshop on Gender-Inclusive Translation Technologies (GITT 2025)",
    month = jun,
    year = "2025",
    address = "Geneva, Switzerland",
    publisher = "European Association for Machine Translation",
    url = "https://preview.aclanthology.org/mtsummit-25-ingestion/2025.gitt-1.1/",
    pages = "1--16",
    ISBN = "978-2-9701897-4-9",
    abstract = "While gender bias in modern Neural Machine Translation (NMT) systems has received much attention, the traditional evaluation metrics for these systems do not fully capture the extent to which models integrate contextual gender cues. We propose a novel evaluation metric called Minimal Pair Accuracy (MPA) which measures the reliance of models on gender cues for gender disambiguation. We evaluate a number of NMT models using this metric, we show that they ignore available gender cues in most cases in favour of (statistical) stereotypical gender interpretation. We further show that in anti-stereotypical cases, these models tend to more consistently take male gender cues into account while ignoring the female cues. Finally, we analyze the attention head weights in the encoder component of these models and show that while all models to some extent encode gender information, the male gender cues elicit a more diffused response compared to the more concentrated and specialized responses to female gender cues."
}