@inproceedings{lysyuk-braslavski-2024-skoltech,
    title = "Skoltech at {T}ext{G}raphs-17 Shared Task: Finding {GPT}-4 Prompting Strategies for Multiple Choice Questions",
    author = "Lysyuk, Maria  and
      Braslavski, Pavel",
    editor = "Ustalov, Dmitry  and
      Gao, Yanjun  and
      Panchenko, Alexander  and
      Tutubalina, Elena  and
      Nikishina, Irina  and
      Ramesh, Arti  and
      Sakhovskiy, Andrey  and
      Usbeck, Ricardo  and
      Penn, Gerald  and
      Valentino, Marco",
    booktitle = "Proceedings of TextGraphs-17: Graph-based Methods for Natural Language Processing",
    month = aug,
    year = "2024",
    address = "Bangkok, Thailand",
    publisher = "Association for Computational Linguistics",
    url = "https://preview.aclanthology.org/jlcl-multiple-ingestion/2024.textgraphs-1.14/",
    pages = "149--153",
    abstract = "In this paper, we present our solution to the TextGraphs-17 Shared Task on Text-Graph Representations for KGQA. GPT-4 alone, with chain-of-thought reasoning and a given set of answers, achieves an F1 score of 0.78. By employing subgraph size as a feature, Wikidata answer description as an additional context, and question rephrasing technique, we further strengthen this result. These tricks help to answer questions that were not initially answered and to eliminate irrelevant, identical answers. We have managed to achieve an F1 score of 0.83 and took 2nd place, improving the score by 0.05 over the baseline. An open implementation of our method is available on GitHub."
}