@inproceedings{zhou-etal-2022-richer,
    title = "Richer Countries and Richer Representations",
    author = "Zhou, Kaitlyn  and
      Ethayarajh, Kawin  and
      Jurafsky, Dan",
    editor = "Muresan, Smaranda  and
      Nakov, Preslav  and
      Villavicencio, Aline",
    booktitle = "Findings of the Association for Computational Linguistics: ACL 2022",
    month = may,
    year = "2022",
    address = "Dublin, Ireland",
    publisher = "Association for Computational Linguistics",
    url = "https://preview.aclanthology.org/add-emnlp-2024-awards/2022.findings-acl.164/",
    doi = "10.18653/v1/2022.findings-acl.164",
    pages = "2074--2085",
    abstract = "We examine whether some countries are more richly represented in embedding space than others. We find that countries whose names occur with low frequency in training corpora are more likely to be tokenized into subwords, are less semantically distinct in embedding space, and are less likely to be correctly predicted: e.g., Ghana (the correct answer and in-vocabulary) is not predicted for, {\textquotedblleft}The country producing the most cocoa is [MASK].{\textquotedblright}. Although these performance discrepancies and representational harms are due to frequency, we find that frequency is highly correlated with a country`s GDP; thus perpetuating historic power and wealth inequalities. We analyze the effectiveness of mitigation strategies; recommend that researchers report training word frequencies; and recommend future work for the community to define and design representational guarantees."
}