@inproceedings{bhathena-etal-2020-evaluating,
    title = "Evaluating Compositionality of Sentence Representation Models",
    author = "Bhathena, Hanoz  and
      Willis, Angelica  and
      Dass, Nathan",
    editor = "Gella, Spandana  and
      Welbl, Johannes  and
      Rei, Marek  and
      Petroni, Fabio  and
      Lewis, Patrick  and
      Strubell, Emma  and
      Seo, Minjoon  and
      Hajishirzi, Hannaneh",
    booktitle = "Proceedings of the 5th Workshop on Representation Learning for NLP",
    month = jul,
    year = "2020",
    address = "Online",
    publisher = "Association for Computational Linguistics",
    url = "https://preview.aclanthology.org/fix-sig-urls/2020.repl4nlp-1.22/",
    doi = "10.18653/v1/2020.repl4nlp-1.22",
    pages = "185--193",
    abstract = "We evaluate the compositionality of general-purpose sentence encoders by proposing two different metrics to quantify compositional understanding capability of sentence encoders. We introduce a novel metric, Polarity Sensitivity Scoring (PSS), which utilizes sentiment perturbations as a proxy for measuring compositionality. We then compare results from PSS with those obtained via our proposed extension of a metric called Tree Reconstruction Error (TRE) (CITATION) where compositionality is evaluated by measuring how well a true representation producing model can be approximated by a model that explicitly combines representations of its primitives."
}