@inproceedings{tezekbayev-etal-2022-speeding,
    title = "Speeding Up Entmax",
    author = "Tezekbayev, Maxat  and
      Nikoulina, Vassilina  and
      Gall{\'e}, Matthias  and
      Assylbekov, Zhenisbek",
    editor = "Carpuat, Marine  and
      de Marneffe, Marie-Catherine  and
      Meza Ruiz, Ivan Vladimir",
    booktitle = "Findings of the Association for Computational Linguistics: NAACL 2022",
    month = jul,
    year = "2022",
    address = "Seattle, United States",
    publisher = "Association for Computational Linguistics",
    url = "https://preview.aclanthology.org/jlcl-multiple-ingestion/2022.findings-naacl.86/",
    doi = "10.18653/v1/2022.findings-naacl.86",
    pages = "1142--1158",
    abstract = "Softmax is the de facto standard for normalizing logits in modern neural networks for language processing. However, by producing a dense probability distribution each token in the vocabulary has a nonzero chance of being selected at each generation step, leading to a variety of reported problems in text generation. $\alpha$-entmax of Peters et al. (2019) solves this problem, but is unfortunately slower than softmax. In this paper, we propose an alternative to $\alpha$-entmax, which keeps its virtuous characteristics, but is as fast as optimized softmax and achieves on par or better performance in machine translation task."
}