@inproceedings{zhang-etal-2020-language,
    title = "Do Language Embeddings capture Scales?",
    author = "Zhang, Xikun  and
      Ramachandran, Deepak  and
      Tenney, Ian  and
      Elazar, Yanai  and
      Roth, Dan",
    editor = "Alishahi, Afra  and
      Belinkov, Yonatan  and
      Chrupa{\l}a, Grzegorz  and
      Hupkes, Dieuwke  and
      Pinter, Yuval  and
      Sajjad, Hassan",
    booktitle = "Proceedings of the Third BlackboxNLP Workshop on Analyzing and Interpreting Neural Networks for NLP",
    month = nov,
    year = "2020",
    address = "Online",
    publisher = "Association for Computational Linguistics",
    url = "https://preview.aclanthology.org/Author-page-Marten-During-lu/2020.blackboxnlp-1.27/",
    doi = "10.18653/v1/2020.blackboxnlp-1.27",
    pages = "292--299",
    abstract = "Pretrained Language Models (LMs) have been shown to possess significant linguistic, common sense and factual knowledge. One form of knowledge that has not been studied yet in this context is information about the scalar magnitudes of objects. We show that pretrained language models capture a significant amount of this information but are short of the capability required for general common-sense reasoning. We identify contextual information in pre-training and numeracy as two key factors affecting their performance, and show that a simple method of canonicalizing numbers can have a significant effect on the results."
}