John Chen


2020

pdf
Exploring Text Specific and Blackbox Fairness Algorithms in Multimodal Clinical NLP
John Chen | Ian Berlot-Attwell | Xindi Wang | Safwan Hossain | Frank Rudzicz
Proceedings of the 3rd Clinical Natural Language Processing Workshop

Clinical machine learning is increasingly multimodal, collected in both structured tabular formats and unstructured forms such as free text. We propose a novel task of exploring fairness on a multimodal clinical dataset, adopting equalized odds for the downstream medical prediction tasks. To this end, we investigate a modality-agnostic fairness algorithm - equalized odds post processing - and compare it to a text-specific fairness algorithm: debiased clinical word embeddings. Despite the fact that debiased word embeddings do not explicitly address equalized odds of protected groups, we show that a text-specific approach to fairness may simultaneously achieve a good balance of performance classical notions of fairness. Our work opens the door for future work at the critical intersection of clinical NLP and fairness.

pdf
A Smart System to Generate and Validate Question Answer Pairs for COVID-19 Literature
Rohan Bhambhoria | Luna Feng | Dawn Sepehr | John Chen | Conner Cowling | Sedef Kocak | Elham Dolatabadi
Proceedings of the First Workshop on Scholarly Document Processing

Automatically generating question answer (QA) pairs from the rapidly growing coronavirus-related literature is of great value to the medical community. Creating high quality QA pairs would allow researchers to build models to address scientific queries for answers which are not readily available in support of the ongoing fight against the pandemic. QA pair generation is, however, a very tedious and time consuming task requiring domain expertise for annotation and evaluation. In this paper we present our contribution in addressing some of the challenges of building a QA system without gold data. We first present a method to create QA pairs from a large semi-structured dataset through the use of transformer and rule-based models. Next, we propose a means of engaging subject matter experts (SMEs) for annotating the QA pairs through the usage of a web application. Finally, we demonstrate some experiments showcasing the effectiveness of leveraging active learning in designing a high performing model with a substantially lower annotation effort from the domain experts.

2019

pdf
A k-Nearest Neighbor Approach towards Multi-level Sequence Labeling
Yue Chen | John Chen
Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 2 (Industry Papers)

In this paper we present a new method for intent recognition for complex dialog management in low resource situations. Complex dialog management is required because our target domain is real world mixed initiative food ordering between agents and their customers, where individual customer utterances may contain multiple intents and refer to food items with complex structure. For example, a customer might say “Can I get a deluxe burger with large fries and oh put extra mayo on the burger would you?” We approach this task as a multi-level sequence labeling problem, with the constraint of limited real training data. Both traditional methods like HMM, MEMM, or CRF and newer methods like DNN or BiLSTM use only homogeneous feature sets. Newer methods perform better but also require considerably more data. Previous research has done pseudo-data synthesis to obtain the required amounts of training data. We propose to use a k-NN learner with heterogeneous feature set. We used windowed word n-grams, POS tag n-grams and pre-trained word embeddings as features. For the experiments we perform a comparison between using pseudo-data and real world data. We also perform semi-supervised self-training to obtain additional labeled data, in order to better model real world scenarios. Instead of using massive pseudo-data, we show that with only less than 1% of the data size, we can achieve better result than any of the methods above by annotating real world data. We achieve labeled bracketed F-scores of 75.46, 52.84 and 49.66 for the three levels of sequence labeling where each level has a longer word span than its previous level. Overall we achieve 60.71F. In comparison, two previous systems, MEMM and DNN-ELMO, achieved 52.32 and 45.25 respectively.

2018

pdf
Bootstrapping Multilingual Intent Models via Machine Translation for Dialog Automation
Nicholas Ruiz | Srinivas Bangalore | John Chen
Proceedings of the 21st Annual Conference of the European Association for Machine Translation

With the resurgence of chat-based dialog systems in consumer and enterprise applications, there has been much success in developing data-driven and rule-based natural language models to understand human intent. Since these models require large amounts of data and in-domain knowledge, expanding an equivalent service into new markets is disrupted by language barriers that inhibit dialog automation. This paper presents a user study to evaluate the utility of out-of-the-box machine translation technology to (1) rapidly bootstrap multilingual spoken dialog systems and (2) enable existing human analysts to understand foreign language utterances. We additionally evaluate the utility of machine translation in human assisted environments, where a portion of the traffic is processed by analysts. In English→Spanish experiments, we observe a high potential for dialog automation, as well as the potential for human analysts to process foreign language utterances with high accuracy.

2017

pdf
Underspecification in Natural Language Understanding for Dialog Automation
John Chen | Srinivas Bangalore
Proceedings of the International Conference Recent Advances in Natural Language Processing, RANLP 2017

With the increasing number of communication platforms that offer variety of ways of connecting two interlocutors, there is a resurgence of chat-based dialog systems. These systems, typically known as chatbots have been successfully applied in a range of consumer and enterprise applications. A key technology in such chat-bots is robust natural language understanding (NLU) which can significantly influence and impact the efficacy of the conversation and ultimately the user-experience. While NLU is far from perfect, this paper illustrates the role of underspecification and its impact on successful dialog completion.

2014

pdf
MVA: The Multimodal Virtual Assistant
Michael Johnston | John Chen | Patrick Ehlen | Hyuckchul Jung | Jay Lieske | Aarthi Reddy | Ethan Selfridge | Svetlana Stoyanchev | Brant Vasilieff | Jay Wilpon
Proceedings of the 15th Annual Meeting of the Special Interest Group on Discourse and Dialogue (SIGDIAL)

pdf
AT&T: The Tag&Parse Approach to Semantic Parsing of Robot Spatial Commands
Svetlana Stoyanchev | Hyuckchul Jung | John Chen | Srinivas Bangalore
Proceedings of the 8th International Workshop on Semantic Evaluation (SemEval 2014)

pdf
A Framework for Translating SMS Messages
Vivek Kumar Rangarajan Sridhar | John Chen | Srinivas Bangalore | Ron Shacham
Proceedings of COLING 2014, the 25th International Conference on Computational Linguistics: Technical Papers

2013

pdf
Segmentation Strategies for Streaming Speech Translation
Vivek Kumar Rangarajan Sridhar | John Chen | Srinivas Bangalore | Andrej Ljolje | Rathinavelu Chengalvarayan
Proceedings of the 2013 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies

2011

pdf
Unsupervised Russian POS Tagging with Appropriate Context
Li Yang | Erik Peterson | John Chen | Yana Petrova | Rohini Srihari
Proceedings of the Fifth International Workshop On Cross Lingual Information Access

2006

pdf
Automatically Extracting Nominal Mentions of Events with a Bootstrapped Probabilistic Classifier
Cassandre Creswell | Matthew J. Beal | John Chen | Thomas L. Cornell | Lars Nilsson | Rohini K. Srihari
Proceedings of the COLING/ACL 2006 Main Conference Poster Sessions

2004

pdf
Summarizing Email Threads
Owen Rambow | Lokesh Shrestha | John Chen | Christy Laurdisen
Proceedings of HLT-NAACL 2004: Short Papers

2003

pdf
Columbia’s Newsblaster: New Features and Future Directions
Kathleen McKeown | Regina Barzilay | John Chen | David Elson | David Evans | Judith Klavans | Ani Nenkova | Barry Schiffman | Sergey Sigelman
Companion Volume of the Proceedings of HLT-NAACL 2003 - Demonstrations

pdf
Use of Deep Linguistic Features for the Recognition and Labeling of Semantic Arguments
John Chen | Owen Rambow
Proceedings of the 2003 Conference on Empirical Methods in Natural Language Processing

2002

pdf
Towards Automatic Generation of Natural Language Generation Systems
John Chen | Srinivas Bangalore | Owen Rambow | Marilyn A. Walker
COLING 2002: The 19th International Conference on Computational Linguistics

pdf
Context-Free Parsing of a Tree Adjoining Grammar Using Finite-State Machines
Alexis Nasr | Owen Rambow | John Chen | Srinivas Bangalore
Proceedings of the Sixth International Workshop on Tree Adjoining Grammar and Related Frameworks (TAG+6)

pdf
Reranking an n-gram supertagger
John Chen | Srinivas Bangalore | Michael Collins | Owen Rambow
Proceedings of the Sixth International Workshop on Tree Adjoining Grammar and Related Frameworks (TAG+6)

2001

pdf
Impact of Quality and Quantity of Corpora on Stochastic Generation
Srinivas Bangalore | John Chen | Owen Rambow
Proceedings of the 2001 Conference on Empirical Methods in Natural Language Processing

2000

pdf
Automated Extraction of TAGs from the Penn Treebank
John Chen | K. Vijay-Shanker
Proceedings of the Sixth International Workshop on Parsing Technologies

The accuracy of statistical parsing models can be improved with the use of lexical information. Statistical parsing using Lexicalized tree adjoining grammar (LTAG), a kind of lexicalized grammar, has remained relatively unexplored. We believe that is largely in part due to the absence of large corpora accurately bracketed in terms of a perspicuous yet broad coverage LTAG. Our work attempts to alleviate this difficulty. We extract different LTAGs from the Penn Treebank. We show that certain strategies yield an improved extracted LTAG in terms of compactness, broad coverage, and supertagging accuracy. Furthermore, we perform a preliminary investigation in smoothing these grammars by means of an external linguistic resource, namely, the tree families of an XTAG grammar, a hand built grammar of English.

1999

pdf
New Models for Improving Supertag Disambiguation
John Chen | Srinivas Bangalore
Ninth Conference of the European Chapter of the Association for Computational Linguistics

1997

pdf
Towards a Reduced Commitment, D-Theory Style TAG Parser
John Chen | K. Vijay-Shankar
Proceedings of the Fifth International Workshop on Parsing Technologies

Many traditional TAG parsers handle ambiguity by considering all of the possible choices as they unfold during parsing. In contrast , D-theory parsers cope with ambiguity by using underspecified descriptions of trees. This paper introduces a novel approach to parsing TAG, namely one that explores how D-theoretic notions may be applied to TAG parsing. Combining the D-theoretic approach to TAG parsing as we do here raises new issues and problems. D-theoretic underspecification is used as a novel approach in the context of TAG parsing for delaying attachment decisions. Conversely, the use of TAG reveals the need for additional types of underspecification that have not been considered so far in the D-theoretic framework. These include combining sets of trees into their underspecified equivalents as well as underspecifying combinations of trees. In this paper, we examine various issues that arise in this new approach to TAG parsing and present solutions to some of the problems. We also describe other issues which need to be resolved for this method of parsing to be implemented.