Anton Leuski


Which Model Should We Use for a Real-World Conversational Dialogue System? a Cross-Language Relevance Model or a Deep Neural Net?
Seyed Hossein Alavi | Anton Leuski | David Traum
Proceedings of the Twelfth Language Resources and Evaluation Conference

We compare two models for corpus-based selection of dialogue responses: one based on cross-language relevance with a cross-language LSTM model. Each model is tested on multiple corpora, collected from two different types of dialogue source material. Results show that while the LSTM model performs adequately on a very large corpus (millions of utterances), its performance is dominated by the cross-language relevance model for a more moderate-sized corpus (ten thousands of utterances).

Evaluation of Off-the-shelf Speech Recognizers Across Diverse Dialogue Domains
Kallirroi Georgila | Anton Leuski | Volodymyr Yanov | David Traum
Proceedings of the Twelfth Language Resources and Evaluation Conference

We evaluate several publicly available off-the-shelf (commercial and research) automatic speech recognition (ASR) systems across diverse dialogue domains (in US-English). Our evaluation is aimed at non-experts with limited experience in speech recognition. Our goal is not only to compare a variety of ASR systems on several diverse data sets but also to measure how much ASR technology has advanced since our previous large-scale evaluations on the same data sets. Our results show that the performance of each speech recognizer can vary significantly depending on the domain. Furthermore, despite major recent progress in ASR technology, current state-of-the-art speech recognizers perform poorly in domains that require special vocabulary and language models, and under noisy conditions. We expect that our evaluation will prove useful to ASR consumers and dialogue system designers.


The Niki and Julie Corpus: Collaborative Multimodal Dialogues between Humans, Robots, and Virtual Agents
Ron Artstein | Jill Boberg | Alesia Gainer | Jonathan Gratch | Emmanuel Johnson | Anton Leuski | Gale Lucas | David Traum
Proceedings of the Eleventh International Conference on Language Resources and Evaluation (LREC 2018)

ScoutBot: A Dialogue System for Collaborative Navigation
Stephanie M. Lukin | Felix Gervits | Cory J. Hayes | Pooja Moolchandani | Anton Leuski | John G. Rogers III | Carlos Sanchez Amaro | Matthew Marge | Clare R. Voss | David Traum
Proceedings of ACL 2018, System Demonstrations

ScoutBot is a dialogue interface to physical and simulated robots that supports collaborative exploration of environments. The demonstration will allow users to issue unconstrained spoken language commands to ScoutBot. ScoutBot will prompt for clarification if the user’s instruction needs additional input. It is trained on human-robot dialogue collected from Wizard-of-Oz experiments, where robot responses were initiated by a human wizard in previous interactions. The demonstration will show a simulated ground robot (Clearpath Jackal) in a simulated environment supported by ROS (Robot Operating System).


Lessons in Dialogue System Deployment
Anton Leuski | Ron Artstein
Proceedings of the 18th Annual SIGdial Meeting on Discourse and Dialogue

We analyze deployment of an interactive dialogue system in an environment where deep technical expertise might not be readily available. The initial version was created using a collection of research tools. We summarize a number of challenges with its deployment at two museums and describe a new system that simplifies the installation and user interface; reduces reliance on 3rd-party software; and provides a robust data collection mechanism.

SHIHbot: A Facebook chatbot for Sexual Health Information on HIV/AIDS
Jacqueline Brixey | Rens Hoegen | Wei Lan | Joshua Rusow | Karan Singla | Xusen Yin | Ron Artstein | Anton Leuski
Proceedings of the 18th Annual SIGdial Meeting on Discourse and Dialogue

We present the implementation of an autonomous chatbot, SHIHbot, deployed on Facebook, which answers a wide variety of sexual health questions on HIV/AIDS. The chatbot’s response database is com-piled from professional medical and public health resources in order to provide reliable information to users. The system’s backend is NPCEditor, a response selection platform trained on linked questions and answers; to our knowledge this is the first retrieval-based chatbot deployed on a large public social network.


New Dimensions in Testimony Demonstration
Ron Artstein | Alesia Gainer | Kallirroi Georgila | Anton Leuski | Ari Shapiro | David Traum
Proceedings of the 2016 Conference of the North American Chapter of the Association for Computational Linguistics: Demonstrations


Evaluating Spoken Dialogue Processing for Time-Offset Interaction
David Traum | Kallirroi Georgila | Ron Artstein | Anton Leuski
Proceedings of the 16th Annual Meeting of the Special Interest Group on Discourse and Dialogue


Which ASR should I choose for my dialogue system?
Fabrizio Morbini | Kartik Audhkhasi | Kenji Sagae | Ron Artstein | Doğan Can | Panayiotis Georgiou | Shri Narayanan | Anton Leuski | David Traum
Proceedings of the SIGDIAL 2013 Conference


The BladeMistress Corpus: From Talk to Action in Virtual Worlds
Anton Leuski | Carsten Eickhoff | James Ganis | Victor Lavrenko
Proceedings of the Eighth International Conference on Language Resources and Evaluation (LREC'12)

Virtual Worlds (VW) are online environments where people come together to interact and perform various tasks. The chat transcripts of interactions in VWs pose unique opportunities and challenges for language analysis: Firstly, the language of the transcripts is very brief, informal, and task-oriented. Secondly, in addition to chat, a VW system records users' in-world activities. Such a record could allow us to analyze how the language of interactions is linked to the users actions. For example, we can make the language analysis of the users dialogues more effective by taking into account the context of the corresponding action or we can predict or detect users actions by analyzing the content of conversations. Thirdly, a joined analysis of both the language and the actions would empower us to build effective modes of the users and their behavior. In this paper we present a corpus constructed from logs from an online multiplayer game BladeMistress. We describe the original logs, annotations that we created on the data, and summarize some of the experiments.

Reinforcement Learning of Question-Answering Dialogue Policies for Virtual Museum Guides
Teruhisa Misu | Kallirroi Georgila | Anton Leuski | David Traum
Proceedings of the 13th Annual Meeting of the Special Interest Group on Discourse and Dialogue

A Study in How NLU Performance Can Affect the Choice of Dialogue System Architecture
Anton Leuski | David DeVault
Proceedings of the 13th Annual Meeting of the Special Interest Group on Discourse and Dialogue


An Evaluation of Alternative Strategies for Implementing Dialogue Policies Using Statistical Classification and Hand-Authored Rules
David DeVault | Anton Leuski | Kenji Sagae
Proceedings of 5th International Joint Conference on Natural Language Processing

Toward Learning and Evaluation of Dialogue Policies with Text Examples
David DeVault | Anton Leuski | Kenji Sagae
Proceedings of the SIGDIAL 2011 Conference


NPCEditor: A Tool for Building Question-Answering Characters
Anton Leuski | David Traum
Proceedings of the Seventh International Conference on Language Resources and Evaluation (LREC'10)

NPCEditor is a system for building and deploying virtual characters capable of engaging a user in spoken dialog on a limited domain. The dialogue may take any form as long as the character responses can be specified a priori. For example, NPCEditor has been used for constructing question answering characters where a user asks questions and the character responds, but other scenarios are possible. At the core of the system is a state of the art statistical language classification technology for mapping from user's text input to system responses. NPCEditor combines the classifier with a database that stores the character information and relevant language data, a server that allows the character designer to deploy the completed characters, and a user-friendly editor that helps the designer to accomplish both character design and deployment tasks. In the paper we define the overall system architecture, describe individual NPCEditor components, and guide the reader through the steps of building a virtual character.


Hassan: A Virtual Human for Tactical Questioning
David Traum | Antonio Roque | Anton Leuski | Panayiotis Georgiou | Jillian Gerten | Bilyana Martinovski | Shrikanth Narayanan | Susan Robinson | Ashish Vaswani
Proceedings of the 8th SIGdial Workshop on Discourse and Dialogue


Building Effective Question Answering Characters
Anton Leuski | Ronakkumar Patel | David Traum | Brandon Kennedy
Proceedings of the 7th SIGdial Workshop on Discourse and Dialogue


Statistical Shallow Semantic Parsing despite Little Training Data
Rahul Bhagat | Anton Leuski | Eduard Hovy
Proceedings of the Ninth International Workshop on Parsing Technology

Dealing with Doctors: A Virtual Human for Non-team Interaction
David Traum | William Swartout | Jonathan Gratch | Stacy Marsella | Patrick Kenny | Eduard Hovy | Shri Narayanan | Ed Fast | Bilyana Martinovski | Rahul Baghat | Susan Robinson | Andrew Marshall | Dagen Wang | Sudeep Gandhe | Anton Leuski
Proceedings of the 6th SIGdial Workshop on Discourse and Dialogue


iNeATS: Interactive Multi-Document Summarization
Anton Leuski | Chin-Yew Lin | Eduard Hovy
The Companion Volume to the Proceedings of 41st Annual Meeting of the Association for Computational Linguistics

Desparately Seeking Cebuano
Douglas W. Oard | David Doermann | Bonnie Dorr | Daqing He | Philip Resnik | Amy Weinberg | William Byrne | Sanjeev Khudanpur | David Yarowsky | Anton Leuski | Philipp Koehn | Kevin Knight
Companion Volume of the Proceedings of HLT-NAACL 2003 - Short Papers


Monitoring the News: a TDT demonstration system
David Frey | Rahul Gupta | Vikas Khandelwal | Victor Lavrenko | Anton Leuski | James Allan
Proceedings of the First International Conference on Human Language Technology Research