Hy Nguyen
2024
Do Text-to-Vis Benchmarks Test Real Use of Visualisations?
Hy Nguyen
|
Xuefei He
|
Andrew Reeson
|
Cecile Paris
|
Josiah Poon
|
Jonathan K. Kummerfeld
Proceedings of the 2024 Conference on Empirical Methods in Natural Language Processing
Large language models are able to generate code for visualisations in response to simple user requests.This is a useful application and an appealing one for NLP research because plots of data provide grounding for language.However, there are relatively few benchmarks, and those that exist may not be representative of what users do in practice.This paper investigates whether benchmarks reflect real-world use through an empirical study comparing benchmark datasets with code from public repositories.Our findings reveal a substantial gap, with evaluations not testing the same distribution of chart types, attributes, and actions as real-world examples.One dataset is representative, but requires extensive modification to become a practical end-to-end benchmark. This shows that new benchmarks are needed to support the development of systems that truly address users’ visualisation needs.These observations will guide future data creation, highlighting which features hold genuine significance for users.
2022
Meeting Decision Tracker: Making Meeting Minutes with De-Contextualized Utterances
Shumpei Inoue
|
Hy Nguyen
|
Hoang Pham
|
Tsungwei Liu
|
Minh-Tien Nguyen
Proceedings of the 2nd Conference of the Asia-Pacific Chapter of the Association for Computational Linguistics and the 12th International Joint Conference on Natural Language Processing: System Demonstrations
Meetings are a universal process to make decisions in business and project collaboration. The capability to automatically itemize the decisions in daily meetings allows for extensive tracking of past discussions. To that end, we developed Meeting Decision Tracker, a prototype system to construct decision items comprising decision utterance detector (DUD) and decision utterance rewriter (DUR). We show that DUR makes a sizable contribution to improving the user experience by dealing with utterance collapse in natural conversation. An introduction video of our system is also available at https://youtu.be/TG1pJJo0Iqo.
Search
Co-authors
- Andrew Reeson 1
- Cecile Paris 1
- Hoang Pham 1
- Jonathan K. Kummerfeld 1
- Josiah Poon 1
- show all...