@inproceedings{li-michael-2022-overconfidence,
    title = "Overconfidence in the Face of Ambiguity with Adversarial Data",
    author = "Li, Margaret  and
      Michael, Julian",
    editor = "Bartolo, Max  and
      Kirk, Hannah  and
      Rodriguez, Pedro  and
      Margatina, Katerina  and
      Thrush, Tristan  and
      Jia, Robin  and
      Stenetorp, Pontus  and
      Williams, Adina  and
      Kiela, Douwe",
    booktitle = "Proceedings of the First Workshop on Dynamic Adversarial Data Collection",
    month = jul,
    year = "2022",
    address = "Seattle, WA",
    publisher = "Association for Computational Linguistics",
    url = "https://preview.aclanthology.org/jlcl-multiple-ingestion/2022.dadc-1.4/",
    doi = "10.18653/v1/2022.dadc-1.4",
    pages = "30--40",
    abstract = "Adversarial data collection has shown promise as a method for building models which are more robust to the spurious correlations that generally appear in naturalistic data. However, adversarially-collected data may itself be subject to biases, particularly with regard to ambiguous or arguable labeling judgments. Searching for examples where an annotator disagrees with a model might over-sample ambiguous inputs, and filtering the results for high inter-annotator agreement may under-sample them. In either case, training a model on such data may produce predictable and unwanted biases. In this work, we investigate whether models trained on adversarially-collected data are miscalibrated with respect to the ambiguity of their inputs. Using Natural Language Inference models as a testbed, we find no clear difference in accuracy between naturalistically and adversarially trained models, but our model trained only on adversarially-sourced data is considerably more overconfident of its predictions and demonstrates worse calibration, especially on ambiguous inputs. This effect is mitigated, however, when naturalistic and adversarial training data are combined."
}