A Neural, Interactive-predictive System for Multimodal Sequence to Sequence Tasks
Abstract
We present a demonstration of a neural interactive-predictive system for tackling multimodal sequence to sequence tasks. The system generates text predictions to different sequence to sequence tasks: machine translation, image and video captioning. These predictions are revised by a human agent, who introduces corrections in the form of characters. The system reacts to each correction, providing alternative hypotheses, compelling with the feedback provided by the user. The final objective is to reduce the human effort required during this correction process. This system is implemented following a client-server architecture. For accessing the system, we developed a website, which communicates with the neural model, hosted in a local server. From this website, the different tasks can be tackled following the interactive–predictive framework. We open-source all the code developed for building this system. The demonstration in hosted in http://casmacat.prhlt.upv.es/interactive-seq2seq.- Anthology ID:
- P19-3014
- Volume:
- Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics: System Demonstrations
- Month:
- July
- Year:
- 2019
- Address:
- Florence, Italy
- Editors:
- Marta R. Costa-jussà, Enrique Alfonseca
- Venue:
- ACL
- SIG:
- Publisher:
- Association for Computational Linguistics
- Note:
- Pages:
- 81–86
- Language:
- URL:
- https://aclanthology.org/P19-3014
- DOI:
- 10.18653/v1/P19-3014
- Cite (ACL):
- Álvaro Peris and Francisco Casacuberta. 2019. A Neural, Interactive-predictive System for Multimodal Sequence to Sequence Tasks. In Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics: System Demonstrations, pages 81–86, Florence, Italy. Association for Computational Linguistics.
- Cite (Informal):
- A Neural, Interactive-predictive System for Multimodal Sequence to Sequence Tasks (Peris & Casacuberta, ACL 2019)
- PDF:
- https://preview.aclanthology.org/teach-a-man-to-fish/P19-3014.pdf
- Code
- lvapeab/interactive-keras-captioning