@inproceedings{saha-etal-2024-sequential,
    title = "Sequential {API} Function Calling Using {G}raph{QL} Schema",
    author = "Saha, Avirup  and
      Mandal, Lakshmi  and
      Ganesan, Balaji  and
      Ghosh, Sambit  and
      Sindhgatta, Renuka  and
      Eberhardt, Carlos  and
      Debrunner, Dan  and
      Mehta, Sameep",
    editor = "Al-Onaizan, Yaser  and
      Bansal, Mohit  and
      Chen, Yun-Nung",
    booktitle = "Proceedings of the 2024 Conference on Empirical Methods in Natural Language Processing",
    month = nov,
    year = "2024",
    address = "Miami, Florida, USA",
    publisher = "Association for Computational Linguistics",
    url = "https://preview.aclanthology.org/fix-sig-urls/2024.emnlp-main.1083/",
    doi = "10.18653/v1/2024.emnlp-main.1083",
    pages = "19452--19458",
    abstract = "Function calling using Large Language Models (LLMs) is an active research area that aims to empower LLMs with the ability to execute APIs to perform real-world tasks. However, sequential function calling using LLMs with interdependence between functions is still under-explored. To this end, we introduce GraphQLRestBench, a dataset consisting of natural language utterances paired with function call sequences representing real-world REST API calls with variable mapping between functions. In order to represent the response structure of the functions in the LLM prompt, we use the GraphQL schema of the REST APIs. We also introduce a custom evaluation framework for our dataset consisting of four specially designed metrics. We evaluate various open-source LLMs on our dataset using few-shot Chain-of-Thought and ReAct prompting to establish a reasonable baseline."
}