@inproceedings{lovering-etal-2026-finding,
    title = "On Finding Inconsistencies in Documents",
    author = "Lovering, Charles  and
      Ebner, Seth  and
      Smock, Brandon  and
      Krumdick, Michael  and
      Rabbani, Muhammad Saad  and
      Muhammad, Ahmed  and
      Reddy, Varshini  and
      Tanner, Chris",
    editor = "Liakata, Maria  and
      Moreira, Viviane P.  and
      Zhang, Jiajun  and
      Jurgens, David",
    booktitle = "Findings of the {A}ssociation for {C}omputational {L}inguistics: {ACL} 2026",
    month = jul,
    year = "2026",
    address = "San Diego, California, United States",
    publisher = "Association for Computational Linguistics",
    url = "https://preview.aclanthology.org/ingest-acl/2026.findings-acl.1675/",
    pages = "33523--33564",
    ISBN = "979-8-89176-395-1",
    abstract = "Professionals in academia, law, and finance audit their documents because inconsistencies can result in monetary, reputational, and scientific costs. Language models (LMs) have the potential to dramatically speed up this auditing process. To understand their abilities, we introduce a benchmark, FIND (**F**inding **IN**consistencies in **D**ocuments), where each example is a document with an inconsistency inserted manually by a domain expert. Despite the documents being long, technical, and complex, the best-performing model ({`}gpt-5{`}) recovered 64{\%} of the inserted inconsistencies. Surprisingly, `gpt-5{`} also found inconsistencies already present in the original documents. For example, on 50 arXiv papers, we judged 136 out of 196 of the model{'}s suggestions to be legitimate inconsistencies missed by the original authors. However, despite these findings, even the best models miss almost half of the inconsistencies in FIND, demonstrating that inconsistency detection is still a challenging task."
}