@inproceedings{pimentel-etal-2019-meaning,
    title = "Meaning to Form: Measuring Systematicity as Information",
    author = "Pimentel, Tiago  and
      McCarthy, Arya D.  and
      Blasi, Damian  and
      Roark, Brian  and
      Cotterell, Ryan",
    editor = "Korhonen, Anna  and
      Traum, David  and
      M{\`a}rquez, Llu{\'i}s",
    booktitle = "Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics",
    month = jul,
    year = "2019",
    address = "Florence, Italy",
    publisher = "Association for Computational Linguistics",
    url = "https://preview.aclanthology.org/Author-page-Marten-During-lu/P19-1171/",
    doi = "10.18653/v1/P19-1171",
    pages = "1751--1764",
    abstract = "A longstanding debate in semiotics centers on the relationship between linguistic signs and their corresponding semantics: is there an arbitrary relationship between a word form and its meaning, or does some systematic phenomenon pervade? For instance, does the character bigram {\textquoteleft}gl' have any systematic relationship to the meaning of words like {\textquoteleft}glisten', {\textquoteleft}gleam' and {\textquoteleft}glow'? In this work, we offer a holistic quantification of the systematicity of the sign using mutual information and recurrent neural networks. We employ these in a data-driven and massively multilingual approach to the question, examining 106 languages. We find a statistically significant reduction in entropy when modeling a word form conditioned on its semantic representation. Encouragingly, we also recover well-attested English examples of systematic affixes. We conclude with the meta-point: Our approximate effect size (measured in bits) is quite small{---}despite some amount of systematicity between form and meaning, an arbitrary relationship and its resulting benefits dominate human language."
}