@inproceedings{cetoli-2022-dispatcher,
    title = "Dispatcher: A Message-Passing Approach to Language Modelling",
    author = "Cetoli, Alberto",
    editor = "Dobnik, Simon  and
      Grove, Julian  and
      Sayeed, Asad",
    booktitle = "Proceedings of the 2022 CLASP Conference on (Dis)embodiment",
    month = sep,
    year = "2022",
    address = "Gothenburg, Sweden",
    publisher = "Association for Computational Linguistics",
    url = "https://preview.aclanthology.org/fix-sig-urls/2022.clasp-1.3/",
    pages = "24--29",
    abstract = "This paper proposes a message-passing mechanism to address language modelling. A new layer type is introduced that aims to substitute self-attention for unidirectional sequence generation tasks. The system is shown to be competitive with existing methods: Given N tokens, the computational complexity is O(N logN) and the memory complexity is O(N) under reasonable assumptions. In the end, the Dispatcher layer is seen to achieve comparable perplexity to self-attention while being more efficient."
}