We investigate model calibration in the setting of zero-shot cross-lingual transfer with large-scale pre-trained language models. The level of model calibration is an important metric for evaluating the trustworthiness of predictive models. There exists an essential need for model calibration when natural language models are deployed in critical tasks. We study different post-training calibration methods in structured and unstructured prediction tasks. We find that models trained with data from the source language become less calibrated when applied to the target language and that calibration errors increase with intrinsic task difficulty and relative sparsity of training data. Moreover, we observe a potential connection between the level of calibration error and an earlier proposed measure of the distance from English to other languages. Finally, our comparison demonstrates that among other methods Temperature Scaling (TS) generalizes well to distant languages, but TS fails to calibrate more complex confidence estimation in structured predictions compared to more expressive alternatives like Gaussian Process Calibration.

Historiquement, le suffixe /ə˞/ est un suffixe diminutif correspondant au mot 儿 (<er> en pinyin) qui signifie ”petitesse”. Il relève d’une particularité du style plutôt que de la grammaire. Il apparait souvent dans la parole des locuteurs du nord de la Chine. Pour mieux comprendre le phénomène et son comportement phonologique, on présente les résultats d’une étude acoustique qui vérifie les effets de la rhoticité sur les voyelles adjacentes. Sur la base de ces résultats, on propose une représentation gestuelle du suffixe et des processus qui l’impliquent dans le cadre de la phonologie articulatoire (Browman & Goldstein1992).