@inproceedings{drozdov-etal-2020-unsupervised,
    title = "Unsupervised Parsing with {S}-{DIORA}: Single Tree Encoding for Deep Inside-Outside Recursive Autoencoders",
    author = "Drozdov, Andrew  and
      Rongali, Subendhu  and
      Chen, Yi-Pei  and
      O{'}Gorman, Tim  and
      Iyyer, Mohit  and
      McCallum, Andrew",
    editor = "Webber, Bonnie  and
      Cohn, Trevor  and
      He, Yulan  and
      Liu, Yang",
    booktitle = "Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing (EMNLP)",
    month = nov,
    year = "2020",
    address = "Online",
    publisher = "Association for Computational Linguistics",
    url = "https://preview.aclanthology.org/declare-journal/2020.emnlp-main.392/",
    doi = "10.18653/v1/2020.emnlp-main.392",
    pages = "4832--4845",
    abstract = "The deep inside-outside recursive autoencoder (DIORA; Drozdov et al. 2019) is a self-supervised neural model that learns to induce syntactic tree structures for input sentences \textit{without access to labeled training data}. In this paper, we discover that while DIORA exhaustively encodes all possible binary trees of a sentence with a soft dynamic program, its vector averaging approach is locally greedy and cannot recover from errors when computing the highest scoring parse tree in bottom-up chart parsing. To fix this issue, we introduce S-DIORA, an improved variant of DIORA that encodes a single tree rather than a softly-weighted mixture of trees by employing a hard argmax operation and a beam at each cell in the chart. Our experiments show that through \textit{fine-tuning} a pre-trained DIORA with our new algorithm, we improve the state of the art in \textit{unsupervised} constituency parsing on the English WSJ Penn Treebank by 2.2-6{\%} F1, depending on the data used for fine-tuning."
}