QUIS: Question-guided Insights Generation for Automated Exploratory Data Analysis
Abhijit Manatkar, Ashlesha Akella, Parthivi Gupta, Krishnasuri Narayanam
Abstract
Discovering meaningful insights from a large dataset, known as Exploratory Data Analysis (EDA), is a challenging task that requires thorough exploration and analysis of the data. Automated Data Exploration (ADE) systems use goal-oriented methods with Large Language Models and Reinforcement Learning towards full automation. However, these methods require human involvement to anticipate goals that may limit insight extraction, while fully automated systems demand significant computational resources and retraining for new datasets. We introduce QUIS, a fully automated EDA system that operates in two stages: insight generation (ISGen) driven by question generation (QUGen). The QUGen module generates questions in iterations, refining them from previous iterations to enhance coverage without human intervention or manually curated examples. The ISGen module analyzes data to produce multiple relevant insights in response to each question, requiring no prior training and enabling QUIS to adapt to new datasets.- Anthology ID:
- 2024.emnlp-industry.111
- Volume:
- Proceedings of the 2024 Conference on Empirical Methods in Natural Language Processing: Industry Track
- Month:
- November
- Year:
- 2024
- Address:
- Miami, Florida, US
- Editors:
- Franck Dernoncourt, Daniel Preoţiuc-Pietro, Anastasia Shimorina
- Venue:
- EMNLP
- SIG:
- Publisher:
- Association for Computational Linguistics
- Note:
- Pages:
- 1523–1535
- Language:
- URL:
- https://preview.aclanthology.org/fix-sig-urls/2024.emnlp-industry.111/
- DOI:
- 10.18653/v1/2024.emnlp-industry.111
- Cite (ACL):
- Abhijit Manatkar, Ashlesha Akella, Parthivi Gupta, and Krishnasuri Narayanam. 2024. QUIS: Question-guided Insights Generation for Automated Exploratory Data Analysis. In Proceedings of the 2024 Conference on Empirical Methods in Natural Language Processing: Industry Track, pages 1523–1535, Miami, Florida, US. Association for Computational Linguistics.
- Cite (Informal):
- QUIS: Question-guided Insights Generation for Automated Exploratory Data Analysis (Manatkar et al., EMNLP 2024)
- PDF:
- https://preview.aclanthology.org/fix-sig-urls/2024.emnlp-industry.111.pdf