@inproceedings{chen-etal-2025-towards,
    title = "Towards Consistent Natural-Language Explanations via Explanation-Consistency Finetuning",
    author = "Chen, Yanda  and
      Singh, Chandan  and
      Liu, Xiaodong  and
      Zuo, Simiao  and
      Yu, Bin  and
      He, He  and
      Gao, Jianfeng",
    editor = "Rambow, Owen  and
      Wanner, Leo  and
      Apidianaki, Marianna  and
      Al-Khalifa, Hend  and
      Eugenio, Barbara Di  and
      Schockaert, Steven",
    booktitle = "Proceedings of the 31st International Conference on Computational Linguistics",
    month = jan,
    year = "2025",
    address = "Abu Dhabi, UAE",
    publisher = "Association for Computational Linguistics",
    url = "https://preview.aclanthology.org/jlcl-multiple-ingestion/2025.coling-main.505/",
    pages = "7558--7568",
    abstract = "Large language models (LLMs) often generate convincing, fluent explanations. However, different from humans, they often generate inconsistent explanations on different inputs. For example, an LLM may explain {\textquotedblleft}all birds can fly{\textquotedblright} when answering the question {\textquotedblleft}Can sparrows fly?{\textquotedblright} but meanwhile answer {\textquotedblleft}no{\textquotedblright} to the related question {\textquotedblleft}Can penguins fly?{\textquotedblright}. Explanations should be consistent across related examples so that they allow humans to simulate the LLM`s decision process on multiple examples. We propose explanation-consistency finetuning (EC-finetuning), a method that adapts LLMs to generate more consistent natural-language explanations on related examples. EC-finetuning involves finetuning LLMs on synthetic data that is carefully constructed to contain consistent explanations. Across a variety of question-answering datasets in various domains, EC-finetuning yields a 10.0{\%} relative explanation consistency improvement on 4 finetuning datasets, and generalizes to 7 out-of-distribution datasets not seen during finetuning (+4.5{\%} relative). We will make our code available for reproducibility."
}