@inproceedings{du-etal-2024-language,
    title = "When is a Language Process a Language Model?",
    author = "Du, Li  and
      Lee, Holden  and
      Eisner, Jason  and
      Cotterell, Ryan",
    editor = "Ku, Lun-Wei  and
      Martins, Andre  and
      Srikumar, Vivek",
    booktitle = "Findings of the Association for Computational Linguistics: ACL 2024",
    month = aug,
    year = "2024",
    address = "Bangkok, Thailand",
    publisher = "Association for Computational Linguistics",
    url = "https://preview.aclanthology.org/fix-sig-urls/2024.findings-acl.659/",
    doi = "10.18653/v1/2024.findings-acl.659",
    pages = "11083--11094",
    abstract = "A language model may be viewed as a $\Sigma$-valued stochastic process for some alphabet $\Sigma$.However, in some pathological situations, such a stochastic process may ``leak'' probability mass onto the set of infinite strings and hence is not equivalent to the conventional view of a language model as a distribution over ordinary (finite) strings.Such ill-behaved language processes are referred to as *non-tight* in the literature.In this work, we study conditions of tightness through the lens of stochastic processes.In particular, by regarding the symbol as marking a stopping time and using results from martingale theory, we give characterizations of tightness that generalize our previous work [(Du et al. 2023)](https://arxiv.org/abs/2212.10502)."
}