Raymond Ng

Also published as: Raymond T. Ng


2023

pdf
What happens before and after: Multi-Event Commonsense in Event Coreference Resolution
Sahithya Ravi | Chris Tanner | Raymond Ng | Vered Shwartz
Proceedings of the 17th Conference of the European Chapter of the Association for Computational Linguistics

Event coreference models cluster event mentions pertaining to the same real-world event. Recent models rely on contextualized representations to recognize coreference among lexically or contextually similar mentions. However, models typically fail to leverage commonsense inferences, which is particularly limiting for resolving lexically-divergent mentions. We propose a model that extends event mentions with temporal commonsense inferences. Given a complex sentence with multiple events, e.g., “the man killed his wife and got arrested”, with the target event “arrested”, our model generates plausible events that happen before the target event – such as “the police arrived”, and after it, such as “he was sentenced”. We show that incorporating such inferences into an existing event coreference model improves its performance, and we analyze the coreferences in which such temporal knowledge is required.

pdf
COMET-M: Reasoning about Multiple Events in Complex Sentences
Sahithya Ravi | Raymond Ng | Vered Shwartz
Findings of the Association for Computational Linguistics: EMNLP 2023

Understanding the speaker’s intended meaning often involves drawing commonsense inferences to reason about what is not stated explicitly. In multi-event sentences, it requires understanding the relationships between events based on contextual knowledge. We propose COMET-M (Multi-Event), an event-centric commonsense model capable of generating commonsense inferences for a target event within a complex sentence. COMET-M builds upon COMET (Bosselut et al., 2019), which excels at generating event-centric inferences for simple sentences, but struggles with the complexity of multi-event sentences prevalent in natural text. To overcome this limitation, we curate a Multi-Event Inference (MEI) dataset of 35K human-written inferences. We train COMET-M on the human-written inferences and also create baselines using automatically labeled examples. Experimental results demonstrate the significant performance improvement of COMET-M over COMET in generating multi-event inferences. Moreover, COMET-M successfully produces distinct inferences for each target event, taking the complete context into consideration. COMET-M holds promise for downstream tasks involving natural text such as coreference resolution, dialogue, and story understanding.

2021

pdf
KW-ATTN: Knowledge Infused Attention for Accurate and Interpretable Text Classification
Hyeju Jang | Seojin Bang | Wen Xiao | Giuseppe Carenini | Raymond Ng | Young ji Lee
Proceedings of Deep Learning Inside Out (DeeLIO): The 2nd Workshop on Knowledge Extraction and Integration for Deep Learning Architectures

Text classification has wide-ranging applications in various domains. While neural network approaches have drastically advanced performance in text classification, they tend to be powered by a large amount of training data, and interpretability is often an issue. As a step towards better accuracy and interpretability especially on small data, in this paper we present a new knowledge-infused attention mechanism, called KW-ATTN (KnoWledge-infused ATTentioN) to incorporate high-level concepts from external knowledge bases into Neural Network models. We show that KW-ATTN outperforms baseline models using only words as well as other approaches using concepts by classification accuracy, which indicates that high-level concepts help model prediction. Furthermore, crowdsourced human evaluation suggests that additional concept information helps interpretability of the model.

2020

pdf
Stigma Annotation Scheme and Stigmatized Language Detection in Health-Care Discussions on Social Media
Nadiya Straton | Hyeju Jang | Raymond Ng
Proceedings of the Twelfth Language Resources and Evaluation Conference

Much research has been done within the social sciences on the interpretation and influence of stigma on human behaviour and health, which result in out-of-group exclusion, distancing, cognitive separation, status loss, discrimination, in-group pressure, and often lead to disengagement, non-adherence to treatment plan, and prescriptions by the doctor. However, little work has been conducted on computational identification of stigma in general and in social media discourse in particular. In this paper, we develop the annotation scheme and improve the annotation process for stigma identification, which can be applied to other health-care domains. The data from pro-vaccination and anti-vaccination discussion groups are annotated by trained annotators who have professional background in social science and health-care studies, therefore the group can be considered experts on the subject in comparison to non-expert crowd. Amazon MTurk annotators is another group of annotator with no knowledge on their education background, they are initially treated as non-expert crowd on the subject matter of stigma. We analyze the annotations with visualisation techniques, features from LIWC (Linguistic Inquiry and Word Count) list and make prediction based on bi-grams with traditional and deep learning models. Data augmentation method and application of CNN show high performance accuracy in comparison to other models. Success of the rigorous annotation process on identifying stigma is reconfirmed by achieving high prediction rate with CNN.

pdf
A High Precision Pipeline for Financial Knowledge Graph Construction
Sarah Elhammadi | Laks V.S. Lakshmanan | Raymond Ng | Michael Simpson | Baoxing Huai | Zhefeng Wang | Lanjun Wang
Proceedings of the 28th International Conference on Computational Linguistics

Motivated by applications such as question answering, fact checking, and data integration, there is significant interest in constructing knowledge graphs by extracting information from unstructured information sources, particularly text documents. Knowledge graphs have emerged as a standard for structured knowledge representation, whereby entities and their inter-relations are represented and conveniently stored as (subject,predicate,object) triples in a graph that can be used to power various downstream applications. The proliferation of financial news sources reporting on companies, markets, currencies, and stocks presents an opportunity for extracting valuable knowledge about this crucial domain. In this paper, we focus on constructing a knowledge graph automatically by information extraction from a large corpus of financial news articles. For that purpose, we develop a high precision knowledge extraction pipeline tailored for the financial domain. This pipeline combines multiple information extraction techniques with a financial dictionary that we built, all working together to produce over 342,000 compact extractions from over 288,000 financial news articles, with a precision of 78% at the top-100 extractions. The extracted triples are stored in a knowledge graph making them readily available for use in downstream applications.

2019

pdf
Discourse Analysis and Its Applications
Shafiq Joty | Giuseppe Carenini | Raymond Ng | Gabriel Murray
Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics: Tutorial Abstracts

Discourse processing is a suite of Natural Language Processing (NLP) tasks to uncover linguistic structures from texts at several levels, which can support many downstream applications. This involves identifying the topic structure, the coherence structure, the coreference structure, and the conversation structure for conversational discourse. Taken together, these structures can inform text summarization, machine translation, essay scoring, sentiment analysis, information extraction, question answering, and thread recovery. The tutorial starts with an overview of basic concepts in discourse analysis – monologue vs. conversation, synchronous vs. asynchronous conversation, and key linguistic structures in discourse analysis. We also give an overview of linguistic structures and corresponding discourse analysis tasks that discourse researchers are generally interested in, as well as key applications on which these discourse structures have an impact.

2017

pdf
Generating and Evaluating Summaries for Partial Email Threads: Conversational Bayesian Surprise and Silver Standards
Jordon Johnson | Vaden Masrani | Giuseppe Carenini | Raymond Ng
Proceedings of the 18th Annual SIGdial Meeting on Discourse and Dialogue

We define and motivate the problem of summarizing partial email threads. This problem introduces the challenge of generating reference summaries for partial threads when human annotation is only available for the threads as a whole, particularly when the human-selected sentences are not uniformly distributed within the threads. We propose an oracular algorithm for generating these reference summaries with arbitrary length, and we are making the resulting dataset publicly available. In addition, we apply a recent unsupervised method based on Bayesian Surprise that incorporates background knowledge into partial thread summarization, extend it with conversational features, and modify the mechanism by which it handles redundancy. Experiments with our method indicate improved performance over the baseline for shorter partial threads; and our results suggest that the potential benefits of background knowledge to partial thread summarization should be further investigated with larger datasets.

pdf
Exploring Joint Neural Model for Sentence Level Discourse Parsing and Sentiment Analysis
Bita Nejat | Giuseppe Carenini | Raymond Ng
Proceedings of the 18th Annual SIGdial Meeting on Discourse and Dialogue

Discourse Parsing and Sentiment Analysis are two fundamental tasks in Natural Language Processing that have been shown to be mutually beneficial. In this work, we design and compare two Neural Based models for jointly learning both tasks. In the proposed approach, we first create a vector representation for all the text segments in the input sentence. Next, we apply three different Recursive Neural Net models: one for discourse structure prediction, one for discourse relation prediction and one for sentiment analysis. Finally, we combine these Neural Nets in two different joint models: Multi-tasking and Pre-training. Our results on two standard corpora indicate that both methods result in improvements in each task but Multi-tasking has a bigger impact than Pre-training. Specifically for Discourse Parsing, we see improvements in the prediction of the set of contrastive relations.

2016

pdf
Training Data Enrichment for Infrequent Discourse Relations
Kailang Jiang | Giuseppe Carenini | Raymond Ng
Proceedings of COLING 2016, the 26th International Conference on Computational Linguistics: Technical Papers

Discourse parsing is a popular technique widely used in text understanding, sentiment analysis and other NLP tasks. However, for most discourse parsers, the performance varies significantly across different discourse relations. In this paper, we first validate the underfitting hypothesis, i.e., the less frequent a relation is in the training data, the poorer the performance on that relation. We then explore how to increase the number of positive training instances, without resorting to manually creating additional labeled data. We propose a training data enrichment framework that relies on co-training of two different discourse parsers on unlabeled documents. Importantly, we show that co-training alone is not sufficient. The framework requires a filtering step to ensure that only “good quality” unlabeled documents can be used for enrichment and re-training. We propose and evaluate two ways to perform the filtering. The first is to use an agreement score between the two parsers. The second is to use only the confidence score of the faster parser. Our empirical results show that agreement score can help to boost the performance on infrequent relations, and that the confidence score is a viable approximation of the agreement score for infrequent relations.

2015

pdf bib
CODRA: A Novel Discriminative Framework for Rhetorical Analysis
Shafiq Joty | Giuseppe Carenini | Raymond T. Ng
Computational Linguistics, Volume 41, Issue 3 - September 2015

2014

pdf
A Template-based Abstractive Meeting Summarization: Leveraging Summary and Source Text Relationships
Tatsuro Oya | Yashar Mehdad | Giuseppe Carenini | Raymond Ng
Proceedings of the 8th International Natural Language Generation Conference (INLG)

pdf
Detecting Disagreement in Conversations using Pseudo-Monologic Rhetorical Structure
Kelsey Allen | Giuseppe Carenini | Raymond Ng
Proceedings of the 2014 Conference on Empirical Methods in Natural Language Processing (EMNLP)

pdf
Abstractive Summarization of Product Reviews Using Discourse Structure
Shima Gerani | Yashar Mehdad | Giuseppe Carenini | Raymond T. Ng | Bita Nejat
Proceedings of the 2014 Conference on Empirical Methods in Natural Language Processing (EMNLP)

pdf
Abstractive Summarization of Spoken and Written Conversations Based on Phrasal Queries
Yashar Mehdad | Giuseppe Carenini | Raymond T. Ng
Proceedings of the 52nd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers)

2013

pdf
Abstractive Meeting Summarization with Entailment and Fusion
Yashar Mehdad | Giuseppe Carenini | Frank Tompa | Raymond T. Ng
Proceedings of the 14th European Workshop on Natural Language Generation

pdf
Dialogue Act Recognition in Synchronous and Asynchronous Conversations
Maryam Tavafi | Yashar Mehdad | Shafiq Joty | Giuseppe Carenini | Raymond Ng
Proceedings of the SIGDIAL 2013 Conference

pdf
Towards Topic Labeling with Phrase Entailment and Aggregation
Yashar Mehdad | Giuseppe Carenini | Raymond T. Ng | Shafiq Joty
Proceedings of the 2013 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies

pdf
Combining Intra- and Multi-sentential Rhetorical Parsing for Document-level Discourse Analysis
Shafiq Joty | Giuseppe Carenini | Raymond Ng | Yashar Mehdad
Proceedings of the 51st Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers)

2012

pdf bib
Using the Omega Index for Evaluating Abstractive Community Detection
Gabriel Murray | Giuseppe Carenini | Raymond Ng
Proceedings of Workshop on Evaluation Metrics and System Comparison for Automatic Summarization

pdf
A Novel Discriminative Framework for Sentence-Level Discourse Analysis
Shafiq Joty | Giuseppe Carenini | Raymond Ng
Proceedings of the 2012 Joint Conference on Empirical Methods in Natural Language Processing and Computational Natural Language Learning

2010

pdf
Interpretation and Transformation for Abstracting Conversations
Gabriel Murray | Giuseppe Carenini | Raymond Ng
Human Language Technologies: The 2010 Annual Conference of the North American Chapter of the Association for Computational Linguistics

pdf
Exploiting Conversation Structure in Unsupervised Topic Segmentation for Emails
Shafiq Joty | Giuseppe Carenini | Gabriel Murray | Raymond T. Ng
Proceedings of the 2010 Conference on Empirical Methods in Natural Language Processing

pdf
Domain Adaptation to Summarize Human Conversations
Oana Sandu | Giuseppe Carenini | Gabriel Murray | Raymond Ng
Proceedings of the 2010 Workshop on Domain Adaptation for Natural Language Processing

pdf
Generating and Validating Abstracts of Meeting Conversations: a User Study
Gabriel Murray | Giuseppe Carenini | Raymond Ng
Proceedings of the 6th International Natural Language Generation Conference

2009

pdf bib
Optimization-based Content Selection for Opinion Summarization
Jackie Chi Kit Cheung | Giuseppe Carenini | Raymond T. Ng
Proceedings of the 2009 Workshop on Language Generation and Summarisation (UCNLG+Sum 2009)

2008

pdf
Summarizing Emails with Conversational Cohesion and Subjectivity
Giuseppe Carenini | Raymond T. Ng | Xiaodong Zhou
Proceedings of ACL-08: HLT

2006

pdf
Multi-Document Summarization of Evaluative Text
Giuseppe Carenini | Raymond Ng | Adam Pauls
11th Conference of the European Chapter of the Association for Computational Linguistics