@inproceedings{lomshakov-etal-2024-proconsul,
    title = "{P}ro{C}on{S}u{L}: Project Context for Code Summarization with {LLM}s",
    author = "Lomshakov, Vadim  and
      Podivilov, Andrey  and
      Savin, Sergey  and
      Baryshnikov, Oleg  and
      Lisevych, Alena  and
      Nikolenko, Sergey",
    editor = "Dernoncourt, Franck  and
      Preo{\c{t}}iuc-Pietro, Daniel  and
      Shimorina, Anastasia",
    booktitle = "Proceedings of the 2024 Conference on Empirical Methods in Natural Language Processing: Industry Track",
    month = nov,
    year = "2024",
    address = "Miami, Florida, US",
    publisher = "Association for Computational Linguistics",
    url = "https://preview.aclanthology.org/jlcl-multiple-ingestion/2024.emnlp-industry.65/",
    doi = "10.18653/v1/2024.emnlp-industry.65",
    pages = "866--880",
    abstract = "We propose Project Context for Code Summarization with LLMs (ProConSuL), a new framework to provide a large language model (LLM) with precise information about the code structure from program analysis methods such as a compiler or IDE language services and use task decomposition derived from the code structure. ProConSuL builds a call graph to provide the context from callees and uses a two-phase training method (SFT + preference alignment) to train the model to use the project context. We also provide a new evaluation benchmark for C/C++ functions and a set of proxy metrics. Experimental results demonstrate that ProConSuL allows to significantly improve code summaries and reduce the number of hallucinations compared to the base model (CodeLlama-7B-instruct). We make our code and dataset available at https://github.com/TypingCat13/ProConSuL."
}