Nicholas Roy


Leveraging Past References for Robust Language Grounding
Subhro Roy | Michael Noseworthy | Rohan Paul | Daehyung Park | Nicholas Roy
Proceedings of the 23rd Conference on Computational Natural Language Learning (CoNLL)

Grounding referring expressions to objects in an environment has traditionally been considered a one-off, ahistorical task. However, in realistic applications of grounding, multiple users will repeatedly refer to the same set of objects. As a result, past referring expressions for objects can provide strong signals for grounding subsequent referring expressions. We therefore reframe the grounding problem from the perspective of coreference detection and propose a neural network that detects when two expressions are referring to the same object. The network combines information from vision and past referring expressions to resolve which object is being referred to. Our experiments show that detecting referring expression coreference is an effective way to ground objects described by subtle visual properties, which standard visual grounding models have difficulty capturing. We also show the ability to detect object coreference allows the grounding model to perform well even when it encounters object categories not seen in the training data.


Probabilistic Dialogue Modeling for Speech-Enabled Assistive Technology
William Li | Jim Glass | Nicholas Roy | Seth Teller
Proceedings of the Fourth Workshop on Speech and Language Processing for Assistive Technologies


pdf bib
Toward Learning Perceptually Grounded Word Meanings from Unaligned Parallel Data
Stefanie Tellex | Pratiksha Thaker | Josh Joseph | Nicholas Roy
Proceedings of the Second Workshop on Semantic Interpretation in an Actionable Context


Spoken Dialogue Management Using Probabilistic Reasoning
Nicholas Roy | Joelle Pineau | Sebastian Thrun
Proceedings of the 38th Annual Meeting of the Association for Computational Linguistics