@inproceedings{yin-etal-2026-ledom,
    title = "{LEDOM}: Reverse Language Model",
    author = "Yin, Xunjian  and
      Cheng, Sitao  and
      Xie, Yuxi  and
      Hu, Xinyu  and
      Lin, Li  and
      Wang, Xinyi  and
      Pan, Liangming  and
      Wang, William Yang  and
      Wan, Xiaojun",
    editor = "Liakata, Maria  and
      Moreira, Viviane P.  and
      Zhang, Jiajun  and
      Jurgens, David",
    booktitle = "Proceedings of the 64th Annual Meeting of the {A}ssociation for {C}omputational {L}inguistics (Volume 1: Long Papers)",
    month = jul,
    year = "2026",
    address = "San Diego, California, United States",
    publisher = "Association for Computational Linguistics",
    url = "https://preview.aclanthology.org/ingest-acl/2026.acl-long.975/",
    pages = "21306--21326",
    ISBN = "979-8-89176-390-6",
    abstract = "Autoregressive language models are trained exclusively left-to-right. We explore the complementary factorization, training right-to-left at scale, and ask what reasoning patterns emerge when a model conditions on future context to predict the past.We train LEDOM, an open-source purely reverse autoregressive language model (2B/7B parameters, 435B tokens), and find it develops capabilities distinct from forward models, including abductive inference, question synthesis, and structural handling of the reversal curse.We then explore one application of the reverse model: combining forward likelihood $P(y \mid x)$ with reverse posterior $P(x \mid y)$ through noisy channel duality. We propose Reverse Reward, which reranks forward outputs using reverse posterior estimates, and prove that bidirectional scoring penalizes hallucinated reasoning chains whose backward reconstruction degrades.Reverse Reward yields gains of up to 6.6{\%} on AIME 2024 and 15{\%} on AMC 2023 across multiple strong baselines. We release all codes at https://github.com/Arvid-pku/LEDOM."
}