@inproceedings{ding-koehn-2019-parallelizable,
    title = "Parallelizable Stack Long Short-Term Memory",
    author = "Ding, Shuoyang  and
      Koehn, Philipp",
    editor = "Martins, Andre  and
      Vlachos, Andreas  and
      Kozareva, Zornitsa  and
      Ravi, Sujith  and
      Lampouras, Gerasimos  and
      Niculae, Vlad  and
      Kreutzer, Julia",
    booktitle = "Proceedings of the Third Workshop on Structured Prediction for {NLP}",
    month = jun,
    year = "2019",
    address = "Minneapolis, Minnesota",
    publisher = "Association for Computational Linguistics",
    url = "https://aclanthology.org/W19-1501",
    doi = "10.18653/v1/W19-1501",
    pages = "1--6",
    abstract = "Stack Long Short-Term Memory (StackLSTM) is useful for various applications such as parsing and string-to-tree neural machine translation, but it is also known to be notoriously difficult to parallelize for GPU training due to the fact that the computations are dependent on discrete operations. In this paper, we tackle this problem by utilizing state access patterns of StackLSTM to homogenize computations with regard to different discrete operations. Our parsing experiments show that the method scales up almost linearly with increasing batch size, and our parallelized PyTorch implementation trains significantly faster compared to the Dynet C++ implementation.",
}