The PhotoBook Dataset: Building Common Ground through Visually-Grounded Dialogue
Janosch Haber, Tim Baumgärtner, Ece Takmaz, Lieke Gelderloos, Elia Bruni, Raquel Fernández
Abstract
This paper introduces the PhotoBook dataset, a large-scale collection of visually-grounded, task-oriented dialogues in English designed to investigate shared dialogue history accumulating during conversation. Taking inspiration from seminal work on dialogue analysis, we propose a data-collection task formulated as a collaborative game prompting two online participants to refer to images utilising both their visual context as well as previously established referring expressions. We provide a detailed description of the task setup and a thorough analysis of the 2,500 dialogues collected. To further illustrate the novel features of the dataset, we propose a baseline model for reference resolution which uses a simple method to take into account shared information accumulated in a reference chain. Our results show that this information is particularly important to resolve later descriptions and underline the need to develop more sophisticated models of common ground in dialogue interaction.- Anthology ID:
- P19-1184
- Original:
- P19-1184v1
- Version 2:
- P19-1184v2
- Volume:
- Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics
- Month:
- July
- Year:
- 2019
- Address:
- Florence, Italy
- Venue:
- ACL
- SIG:
- Publisher:
- Association for Computational Linguistics
- Note:
- Pages:
- 1895–1910
- Language:
- URL:
- https://aclanthology.org/P19-1184
- DOI:
- 10.18653/v1/P19-1184
- Cite (ACL):
- Janosch Haber, Tim Baumgärtner, Ece Takmaz, Lieke Gelderloos, Elia Bruni, and Raquel Fernández. 2019. The PhotoBook Dataset: Building Common Ground through Visually-Grounded Dialogue. In Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics, pages 1895–1910, Florence, Italy. Association for Computational Linguistics.
- Cite (Informal):
- The PhotoBook Dataset: Building Common Ground through Visually-Grounded Dialogue (Haber et al., ACL 2019)
- PDF:
- https://preview.aclanthology.org/ingestion-script-update/P19-1184.pdf
- Data
- PhotoBook, COCO, Visual Question Answering