@inproceedings{eppalapally-etal-2024-kapqa,
    title = "{K}a{PQA}: Knowledge-Augmented Product Question-Answering",
    author = "Eppalapally, Swetha  and
      Dangi, Daksh  and
      Bhat, Chaithra  and
      Gupta, Ankita  and
      Zhang, Ruiyi  and
      Agarwal, Shubham  and
      Bagga, Karishma  and
      Yoon, Seunghyun  and
      Lipka, Nedim  and
      Rossi, Ryan  and
      Dernoncourt, Franck",
    editor = "Yu, Wenhao  and
      Shi, Weijia  and
      Yasunaga, Michihiro  and
      Jiang, Meng  and
      Zhu, Chenguang  and
      Hajishirzi, Hannaneh  and
      Zettlemoyer, Luke  and
      Zhang, Zhihan",
    booktitle = "Proceedings of the 3rd Workshop on Knowledge Augmented Methods for NLP",
    month = aug,
    year = "2024",
    address = "Bangkok, Thailand",
    publisher = "Association for Computational Linguistics",
    url = "https://aclanthology.org/2024.knowledgenlp-1.2",
    doi = "10.18653/v1/2024.knowledgenlp-1.2",
    pages = "15--29",
    abstract = "Question-answering for domain-specific applications has recently attracted much interest due to the latest advancements in large language models (LLMs). However, accurately assessing the performance of these applications remains a challenge, mainly due to the lack of suitable benchmarks that effectively simulate real-world scenarios. To address this challenge, we introduce two product question-answering (QA) datasets focused on Adobe Acrobat and Photoshop products to help evaluate the performance of existing models on domain-specific product QA tasks. Additionally, we propose a novel knowledge-driven RAG-QA framework to enhance the performance of the models in the product QA task. Our experiments demonstrated that inducing domain knowledge through query reformulation allowed for increased retrieval and generative performance when compared to standard RAG-QA methods. This improvement, however, is slight, and thus illustrates the challenge posed by the datasets introduced.",
}