2016
pdf
abs
Operational Assessment of Keyword Search on Oral History
Elizabeth Salesky
|
Jessica Ray
|
Wade Shen
Proceedings of the Tenth International Conference on Language Resources and Evaluation (LREC'16)
This project assesses the resources necessary to make oral history searchable by means of automatic speech recognition (ASR). There are many inherent challenges in applying ASR to conversational speech: smaller training set sizes and varying demographics, among others. We assess the impact of dataset size, word error rate and term-weighted value on human search capability through an information retrieval task on Mechanical Turk. We use English oral history data collected by StoryCorps, a national organization that provides all people with the opportunity to record, share and preserve their stories, and control for a variety of demographics including age, gender, birthplace, and dialect on four different training set sizes. We show comparable search performance using a standard speech recognition system as with hand-transcribed data, which is promising for increased accessibility of conversational speech and oral history archives.
2014
pdf
Exploiting Morphological, Grammatical, and Semantic Correlates for Improved Text Difficulty Assessment
Elizabeth Salesky
|
Wade Shen
Proceedings of the Ninth Workshop on Innovative Use of NLP for Building Educational Applications
pdf
Finding Good Enough: A Task-Based Evaluation of Query Biased Summarization for Cross-Language Information Retrieval
Jennifer Williams
|
Sharon Tam
|
Wade Shen
Proceedings of the 2014 Conference on Empirical Methods in Natural Language Processing (EMNLP)
2013
pdf
A Language-Independent Approach to Automatic Text Difficulty Assessment for Second-Language Learners
Wade Shen
|
Jennifer Williams
|
Tamas Marius
|
Elizabeth Salesky
Proceedings of the Second Workshop on Predicting and Improving Text Readability for Target Reader Populations
pdf
abs
The MIT-LL/AFRL IWSLT-2013 MT system
Michaeel Kazi
|
Michael Coury
|
Elizabeth Salesky
|
Jessica Ray
|
Wade Shen
|
Terry Gleason
|
Tim Anderson
|
Grant Erdmann
|
Lane Schwartz
|
Brian Ore
|
Raymond Slyh
|
Jeremy Gwinnup
|
Katherine Young
|
Michael Hutt
Proceedings of the 10th International Workshop on Spoken Language Translation: Evaluation Campaign
This paper describes the MIT-LL/AFRL statistical MT system and the improvements that were developed during the IWSLT 2013 evaluation campaign [1]. As part of these efforts, we experimented with a number of extensions to the standard phrase-based model that improve performance on the Russian to English, Chinese to English, Arabic to English, and English to French TED-talk translation task. We also applied our existing ASR system to the TED-talk lecture ASR task. We discuss the architecture of the MIT-LL/AFRL MT system, improvements over our 2012 system, and experiments we ran during the IWSLT-2013 evaluation. Specifically, we focus on 1) cross-entropy filtering of MT training data, and 2) improved optimization techniques, 3) language modeling, and 4) approximation of out-of-vocabulary words.
2012
pdf
abs
The MIT-LL/AFRL IWSLT 2012 MT system
Jennifer Drexler
|
Wade Shen
|
Tim Anderson
|
Raymond Slyh
|
Brian Ore
|
Eric Hansen
|
Terry Gleason
Proceedings of the 9th International Workshop on Spoken Language Translation: Evaluation Campaign
This paper describes the MIT-LL/AFRL statistical MT system and the improvements that were developed during the IWSLT 2012 evaluation campaign. As part of these efforts, we experimented with a number of extensions to the standard phrase-based model that improve performance on the Arabic to English and English to French TED-talk translation task. We also applied our existing ASR system to the TED-talk lecture ASR task, and combined our ASR and MT systems for the TED-talk SLT task. We discuss the architecture of the MIT-LL/AFRL MT system, improvements over our 2011 system, and experiments we ran during the IWSLT-2012 evaluation. Specifically, we focus on 1) cross-domain translation using MAP adaptation, 2) cross-entropy filtering of MT training data, and 3) improved Arabic morphology for MT preprocessing.
2011
pdf
bib
abs
The MIT-LL/AFRL IWSLT-2011 MT system
A. Ryan Aminzadeh
|
Tim Anderson
|
Ray Slyh
|
Brian Ore
|
Eric Hansen
|
Wade Shen
|
Jennifer Drexler
|
Terry Gleason
Proceedings of the 8th International Workshop on Spoken Language Translation: Evaluation Campaign
This paper describes the MIT-LL/AFRL statistical MT system and the improvements that were developed during the IWSLT 2011 evaluation campaign. As part of these efforts, we experimented with a number of extensions to the standard phrase-based model that improve performance on the Arabic to English and English to French TED-talk translation tasks. We also applied our existing ASR system to the TED-talk lecture ASR task. We discuss the architecture of the MIT-LL/AFRL MT system, improvements over our 2010 system, and experiments we ran during the IWSLT-2011 evaluation. Specifically, we focus on 1) speech recognition for lecture-like data, 2) cross-domain translation using MAP adaptation, and 3) improved Arabic morphology for MT preprocessing.
2010
pdf
abs
The MIT-LL/AFRL IWSLT-2010 MT system
Wade Shen
|
Timothy Anderson
|
Raymond Slyh
|
A. Ryan Aminzadeh
Proceedings of the 7th International Workshop on Spoken Language Translation: Evaluation Campaign
This paper describes the MIT-LL/AFRL statistical MT system and the improvements that were developed during the IWSLT 2010 evaluation campaign. As part of these efforts, we experimented with a number of extensions to the standard phrase-based model that improve performance on the Arabic and Turkish to English translation tasks. We also participated in the new French to English BTEC and English to French TALK tasks. We discuss the architecture of the MIT-LL/AFRL MT system, improvements over our 2008 system, and experiments we ran during the IWSLT-2010 evaluation. Specifically, we focus on 1) cross-domain translation using MAP adaptation, 2) Turkish morphological processing and translation, 3) improved Arabic morphology for MT preprocessing, and 4) system combination methods for machine translation.
2009
pdf
abs
The MIT-LL/AFRL IWSLT-2009 MT system
Wade Shen
|
Brian Delaney
|
A. Ryan Aminzadeh
|
Tim Anderson
|
Ray Slyh
Proceedings of the 6th International Workshop on Spoken Language Translation: Evaluation Campaign
This paper describes the MIT-LL/AFRL statistical MT system and the improvements that were developed during the IWSLT 2009 evaluation campaign. As part of these efforts, we experimented with a number of extensions to the standard phrase-based model that improve performance on the Arabic and Turkish to English translation tasks. We discuss the architecture of the MIT-LL/AFRL MT system, improvements over our 2008 system, and experiments we ran during the IWSLT-2009 evaluation. Specifically, we focus on 1) Cross-domain translation using MAP adaptation and unsupervised training, 2) Turkish morphological processing and translation, 3) improved Arabic morphology for MT preprocessing, and 4) system combination methods for machine translation.
2008
pdf
abs
The MIT-LL/AFRL IWSLT-2008 MT system.
Wade Shen
|
Brian Delaney
|
Tim Anderson
|
Ray Slyh
Proceedings of the 5th International Workshop on Spoken Language Translation: Evaluation Campaign
This paper describes the MIT-LL/AFRL statistical MT system and the improvements that were developed during the IWSLT 2008 evaluation campaign. As part of these efforts, we experimented with a number of extensions to the standard phrase-based model that improve performance for both text and speech-based translation on Chinese and Arabic translation tasks. We discuss the architecture of the MIT-LL/AFRL MT system, improvements over our 2007 system, and experiments we ran during the IWSLT-2008 evaluation. Specifically, we focus on 1) novel segmentation models for phrase-based MT, 2) improved lattice and confusion network decoding of speech input, 3) improved Arabic morphology for MT preprocessing, and 4) system combination methods for machine translation.
pdf
abs
Bridging the Gap between Linguists and Technology Developers: Large-Scale, Sociolinguistic Annotation for Dialect and Speaker Recognition
Christopher Cieri
|
Stephanie Strassel
|
Meghan Glenn
|
Reva Schwartz
|
Wade Shen
|
Joseph Campbell
Proceedings of the Sixth International Conference on Language Resources and Evaluation (LREC'08)
Recent years have seen increased interest within the speaker recognition community in high-level features including, for example, lexical choice, idiomatic expressions or syntactic structures. The promise of speaker recognition in forensic applications drives development toward systems robust to channel differences by selecting features inherently robust to channel difference. Within the language recognition community, there is growing interest in differentiating not only languages but also mutually intelligible dialects of a single language. Decades of research in dialectology suggest that high-level features can enable systems to cluster speakers according to the dialects they speak. The Phanotics (Phonetic Annotation of Typicality in Conversational Speech) project seeks to identify high-level features characteristic of American dialects, annotate a corpus for these features, use the data to dialect recognition systems and also use the categorization to create better models for speaker recognition. The data, once published, should be useful to other developers of speaker and dialect recognition systems and to dialectologists and sociolinguists. We expect the methods will generalize well beyond the speakers, dialects, and languages discussed here and should, if successful, provide a model for how linguists and technology developers can collaborate in the future for the benefit of both groups and toward a deeper understanding of how languages vary and change.
2007
pdf
ILR-Based MT Comprehension Test with Multi-Level Questions
Douglas Jones
|
Martha Herzog
|
Hussny Ibrahim
|
Arvind Jairam
|
Wade Shen
|
Edward Gibson
|
Michael Emonts
Human Language Technologies 2007: The Conference of the North American Chapter of the Association for Computational Linguistics; Companion Volume, Short Papers
pdf
Moses: Open Source Toolkit for Statistical Machine Translation
Philipp Koehn
|
Hieu Hoang
|
Alexandra Birch
|
Chris Callison-Burch
|
Marcello Federico
|
Nicola Bertoldi
|
Brooke Cowan
|
Wade Shen
|
Christine Moran
|
Richard Zens
|
Chris Dyer
|
Ondřej Bojar
|
Alexandra Constantin
|
Evan Herbst
Proceedings of the 45th Annual Meeting of the Association for Computational Linguistics Companion Volume Proceedings of the Demo and Poster Sessions
pdf
abs
The MIT-LL/AFRL IWSLT-2007 MT system
Wade Shen
|
Brian Delaney
|
Tim Anderson
|
Ray Slyh
Proceedings of the Fourth International Workshop on Spoken Language Translation
The MIT-LL/AFRL MT system implements a standard phrase-based, statistical translation model. It incorporates a number of extensions that improve performance for speech-based translation. During this evaluation our efforts focused on the rapid porting of our SMT system to a new language (Arabic) and novel approaches to translation from speech input. This paper discusses the architecture of the MIT-LL/AFRL MT system, improvements over our 2006 system, and experiments we ran during the IWSLT-2007 evaluation. Specifically, we focus on 1) experiments comparing the performance of confusion network decoding and direct lattice decoding techniques for machine translation of speech, 2) the application of lightweight morphology for Arabic MT preprocessing and 3) improved confusion network decoding.
2006
pdf
abs
Toward an Interagency Language Roundtable Based Assessment of Speech-to-Speech Translation Capabilities
Douglas Jones
|
Timothy Anderson
|
Sabine Atwell
|
Brian Delaney
|
James Dirgin
|
Michael Emots
|
Neil Granoein
|
Martha Herzog
|
Timothy Hunter
|
Sargon Jabri
|
Wade Shen
|
Jurgen Sottung
Proceedings of the 7th Conference of the Association for Machine Translation in the Americas: Technical Papers
We present observations from three exercises designed to map the effective listening and speaking skills of an operator of a speech-to-speech translation system (S2S) to the Interagency Language Roundtable (ILR) scale. Such a mapping is non-trivial, but will be useful for government and military decision makers in managing expectations of S2S technology. We observed domain-dependent S2S capabilities in the ILR range of Level 0+ to Level 1, and interactive text-based machine translation in the Level 3 range.
pdf
The JHU workshop 2006 IWSLT system
Wade Shen
|
Richard Zens
|
Nicola Bertoldi
|
Marcello Federico
Proceedings of the Third International Workshop on Spoken Language Translation: Evaluation Campaign
pdf
The MIT-LL/AFRL IWSLT-2006 MT system
Wade Shen
|
Brian Delaney
|
Tim Anderson
Proceedings of the Third International Workshop on Spoken Language Translation: Evaluation Campaign
pdf
An efficient graph search decoder for phrase-based statistical machine translation
Brian Delaney
|
Wade Shen
|
Timothy Anderson
Proceedings of the Third International Workshop on Spoken Language Translation: Papers
2005
pdf
The MIT-LL/AFRL MT System
Wade Shen
|
Brian Delaney
|
Tim Anderson
Proceedings of the Second International Workshop on Spoken Language Translation
2004
pdf
The Effect of Text Difficulty on Machine Translation Performance – A Pilot Study with ILR-Rated Texts in Spanish, Farsi, Arabic, Russian and Korean
Ray Clifford
|
Neil Granoien
|
Douglas Jones
|
Wade Shen
|
Clifford Weinstein
Proceedings of the Fourth International Conference on Language Resources and Evaluation (LREC’04)