@inproceedings{fried-klein-2018-policy,
    title = "Policy Gradient as a Proxy for Dynamic Oracles in Constituency Parsing",
    author = "Fried, Daniel  and
      Klein, Dan",
    editor = "Gurevych, Iryna  and
      Miyao, Yusuke",
    booktitle = "Proceedings of the 56th Annual Meeting of the Association for Computational Linguistics (Volume 2: Short Papers)",
    month = jul,
    year = "2018",
    address = "Melbourne, Australia",
    publisher = "Association for Computational Linguistics",
    url = "https://preview.aclanthology.org/jlcl-multiple-ingestion/P18-2075/",
    doi = "10.18653/v1/P18-2075",
    pages = "469--476",
    abstract = "Dynamic oracles provide strong supervision for training constituency parsers with exploration, but must be custom defined for a given parser`s transition system. We explore using a policy gradient method as a parser-agnostic alternative. In addition to directly optimizing for a tree-level metric such as F1, policy gradient has the potential to reduce exposure bias by allowing exploration during training; moreover, it does not require a dynamic oracle for supervision. On four constituency parsers in three languages, the method substantially outperforms static oracle likelihood training in almost all settings. For parsers where a dynamic oracle is available (including a novel oracle which we define for the transition system of Dyer et al., 2016), policy gradient typically recaptures a substantial fraction of the performance gain afforded by the dynamic oracle."
}