@inproceedings{liu-etal-2023-magic,
    title = "The Magic of {IF}: Investigating Causal Reasoning Abilities in Large Language Models of Code",
    author = "Liu, Xiao  and
      Yin, Da  and
      Zhang, Chen  and
      Feng, Yansong  and
      Zhao, Dongyan",
    editor = "Rogers, Anna  and
      Boyd-Graber, Jordan  and
      Okazaki, Naoaki",
    booktitle = "Findings of the Association for Computational Linguistics: ACL 2023",
    month = jul,
    year = "2023",
    address = "Toronto, Canada",
    publisher = "Association for Computational Linguistics",
    url = "https://preview.aclanthology.org/fix-sig-urls/2023.findings-acl.574/",
    doi = "10.18653/v1/2023.findings-acl.574",
    pages = "9009--9022",
    abstract = "Causal reasoning, the ability to identify cause-and-effect relationship, is crucial in human thinking. Although large language models (LLMs) succeed in many NLP tasks, it is still challenging for them to conduct complex causal reasoning like abductive reasoning and counterfactual reasoning. Given the fact that programming code may express causal relations more often and explicitly with conditional statements like ``if{``}, we want to explore whether Code-LLMs acquire better causal reasoning abilities. Our experiments show that compared to text-only LLMs, Code-LLMs with code prompts are better causal reasoners. We further intervene on the prompts from different aspects, and discover that the key point is the programming structure. Code and data are available at \url{https://github.com/xxxiaol/magic-if}."
}