Abstract
Optimizing accuracy and performance while eliminating hallucinations of open-domain conversational large language models (LLMs) is an open research challenge. A particularly promising direction is to augment and ground LLMs with information from structured sources. This paper introduces Conversational Tables cTBLS, a three-step architecture to retrieve and generate dialogue responses grounded on retrieved tabular information. cTBLS uses Transformer encoder embeddings for Dense Table Retrieval and obtains up to 125% relative improvement over the retriever in the previous state-of-the-art system on the HyrbiDialogue dataset. cTBLS then uses a shared process between encoder and decoder models to perform a coarse+fine tabular knowledge (e.g., cell) ranking combined with a GPT-3.5 LLM response generator to yield a 2x relative improvement in ROUGE scores. Finally, human evaluators prefer cTBLs +80% of the time (coherency, fluency) and judge informativeness to be 4x better than the previous state-of-the-art.- Anthology ID:
- 2023.nlp4convai-1.6
- Volume:
- Proceedings of the 5th Workshop on NLP for Conversational AI (NLP4ConvAI 2023)
- Month:
- July
- Year:
- 2023
- Address:
- Toronto, Canada
- Editors:
- Yun-Nung Chen, Abhinav Rastogi
- Venue:
- NLP4ConvAI
- SIG:
- Publisher:
- Association for Computational Linguistics
- Note:
- Pages:
- 59–70
- Language:
- URL:
- https://aclanthology.org/2023.nlp4convai-1.6
- DOI:
- 10.18653/v1/2023.nlp4convai-1.6
- Cite (ACL):
- Anirudh S. Sundar and Larry Heck. 2023. cTBLS: Augmenting Large Language Models with Conversational Tables. In Proceedings of the 5th Workshop on NLP for Conversational AI (NLP4ConvAI 2023), pages 59–70, Toronto, Canada. Association for Computational Linguistics.
- Cite (Informal):
- cTBLS: Augmenting Large Language Models with Conversational Tables (Sundar & Heck, NLP4ConvAI 2023)
- PDF:
- https://preview.aclanthology.org/nschneid-patch-3/2023.nlp4convai-1.6.pdf