Serguei Pakhomov

Also published as: Serguei V. Pakhomov


2022

pdf
GPT-D: Inducing Dementia-related Linguistic Anomalies by Deliberate Degradation of Artificial Neural Language Models
Changye Li | David Knopman | Weizhe Xu | Trevor Cohen | Serguei Pakhomov
Proceedings of the 60th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers)

Deep learning (DL) techniques involving fine-tuning large numbers of model parameters have delivered impressive performance on the task of discriminating between language produced by cognitively healthy individuals, and those with Alzheimer’s disease (AD). However, questions remain about their ability to generalize beyond the small reference sets that are publicly available for research. As an alternative to fitting model parameters directly, we propose a novel method by which a Transformer DL model (GPT-2) pre-trained on general English text is paired with an artificially degraded version of itself (GPT-D), to compute the ratio between these two models’ perplexities on language from cognitively healthy and impaired individuals. This technique approaches state-of-the-art performance on text data from a widely used “Cookie Theft” picture description task, and unlike established alternatives also generalizes well to spontaneous conversations. Furthermore, GPT-D generates text with characteristics known to be associated with AD, demonstrating the induction of dementia-related linguistic anomalies. Our study is a step toward better understanding of the relationships between the inner workings of generative neural language models, the language that they produce, and the deleterious effects of dementia on human speech and language characteristics.

2021

pdf
Conversational Agent for Daily Living Assessment Coaching Demo
Raymond Finzel | Aditya Gaydhani | Sheena Dufresne | Maria Gini | Serguei Pakhomov
Proceedings of the 16th Conference of the European Chapter of the Association for Computational Linguistics: System Demonstrations

Conversational Agent for Daily Living Assessment Coaching (CADLAC) is a multi-modal conversational agent system designed to impersonate “individuals” with various levels of ability in activities of daily living (ADLs: e.g., dressing, bathing, mobility, etc.) for use in training professional assessors how to conduct interviews to determine one’s level of functioning. The system is implemented on the MindMeld platform for conversational AI and features a Bidirectional Long Short-Term Memory topic tracker that allows the agent to navigate conversations spanning 18 different ADL domains, a dialogue manager that interfaces with a database of over 10,000 historical ADL assessments, a rule-based Natural Language Generation (NLG) module, and a pre-trained open-domain conversational sub-agent (based on GPT-2) for handling conversation turns outside of the 18 ADL domains. CADLAC is delivered via state-of-the-art web frameworks to handle multiple conversations and users simultaneously and is enabled with voice interface. The paper includes a description of the system design and evaluation of individual components followed by a brief discussion of current limitations and next steps.

pdf
Everyday Living Artificial Intelligence Hub
Raymond Finzel | Esha Singh | Martin Michalowski | Maria Gini | Serguei Pakhomov
Proceedings of the Second Workshop on Data Science with Human in the Loop: Language Advances

We present the Everyday Living Artificial Intelligence (AI) Hub, a novel proof-of-concept framework for enhancing human health and wellbeing via a combination of tailored wear-able and Conversational Agent (CA) solutions for non-invasive monitoring of physiological signals, assessment of behaviors through unobtrusive wearable devices, and the provision of personalized interventions to reduce stress and anxiety. We utilize recent advancements and industry standards in the Internet of Things (IoT)and AI technologies to develop this proof-of-concept framework.

2020

pdf
A Tale of Two Perplexities: Sensitivity of Neural Language Models to Lexical Retrieval Deficits in Dementia of the Alzheimer’s Type
Trevor Cohen | Serguei Pakhomov
Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics

In recent years there has been a burgeoning interest in the use of computational methods to distinguish between elicited speech samples produced by patients with dementia, and those from healthy controls. The difference between perplexity estimates from two neural language models (LMs) - one trained on transcripts of speech produced by healthy participants and one trained on those with dementia - as a single feature for diagnostic classification of unseen transcripts has been shown to produce state-of-the-art performance. However, little is known about why this approach is effective, and on account of the lack of case/control matching in the most widely-used evaluation set of transcripts (DementiaBank), it is unclear if these approaches are truly diagnostic, or are sensitive to other variables. In this paper, we interrogate neural LMs trained on participants with and without dementia by using synthetic narratives previously developed to simulate progressive semantic dementia by manipulating lexical frequency. We find that perplexity of neural LMs is strongly and differentially associated with lexical frequency, and that using a mixture model resulting from interpolating control and dementia LMs improves upon the current state-of-the-art for models trained on transcript text exclusively.

2017

pdf bib
What Analogies Reveal about Word Vectors and their Compositionality
Gregory Finley | Stephanie Farmer | Serguei Pakhomov
Proceedings of the 6th Joint Conference on Lexical and Computational Semantics (*SEM 2017)

Analogy completion via vector arithmetic has become a common means of demonstrating the compositionality of word embeddings. Previous work have shown that this strategy works more reliably for certain types of analogical word relationships than for others, but these studies have not offered a convincing account for why this is the case. We arrive at such an account through an experiment that targets a wide variety of analogy questions and defines a baseline condition to more accurately measure the efficacy of our system. We find that the most reliably solvable analogy categories involve either 1) the application of a morpheme with clear syntactic effects, 2) male–female alternations, or 3) named entities. These broader types do not pattern cleanly along a syntactic–semantic divide. We suggest instead that their commonality is distributional, in that the difference between the distributions of two words in any given pair encompasses a relatively small number of word types. Our study offers a needed explanation for why analogy tests succeed and fail where they do and provides nuanced insight into the relationship between word distributions and the theoretical linguistic domains of syntax and semantics.

2013

pdf
UMLS::Similarity: Measuring the Relatedness and Similarity of Biomedical Concepts
Bridget McInnes | Ted Pedersen | Serguei Pakhomov | Ying Liu | Genevieve Melton-Meaux
Proceedings of the 2013 NAACL HLT Demonstration Session

pdf
Computerized Analysis of a Verbal Fluency Test
James O. Ryan | Serguei Pakhomov | Susan Marino | Charles Bernick | Sarah Banks
Proceedings of the 51st Annual Meeting of the Association for Computational Linguistics (Volume 2: Short Papers)

2011

pdf
Using Second-order Vectors in a Knowledge-based Method for Acronym Disambiguation
Bridget T. McInnes | Ted Pedersen | Ying Liu | Serguei V. Pakhomov | Genevieve B. Melton
Proceedings of the Fifteenth Conference on Computational Natural Language Learning

2010

pdf
Automated Identification of Synonyms in Biomedical Acronym Sense Inventories
Genevieve B. Melton | SungRim Moon | Bridget McInnes | Serguei Pakhomov
Proceedings of the NAACL HLT 2010 Second Louhi Workshop on Text and Data Mining of Health Documents

2007

pdf bib
Determining the Syntactic Structure of Medical Terms in Clinical Notes
Bridget McInnes | Ted Pedersen | Serguei Pakhomov
Biological, translational, and clinical language processing

2005

pdf
High Throughput Modularized NLP System for Clinical Text
Serguei Pakhomov | James Buntrock | Patrick Duffy
Proceedings of the ACL Interactive Poster and Demonstration Sessions

2004

pdf
Creating a Test Corpus of Clinical Notes Manually Tagged for Part-of-Speech Information
Serguei Pakhomov | Anni Coden | Christopher Chute
Proceedings of the International Joint Workshop on Natural Language Processing in Biomedicine and its Applications (NLPBA/BioNLP)

2003

pdf
Identification of Patients with Congestive Heart Failure using a Binary Classifier: A Case Study.
Serguei V. Pakhomov | James Buntrock | Christopher G. Chute
Proceedings of the ACL 2003 Workshop on Natural Language Processing in Biomedicine

pdf
Exploring Adjectival Modification in Biomedical Discourse Across Two Genres
Olivier Bodenreider | Serguei V. Pakhomov
Proceedings of the ACL 2003 Workshop on Natural Language Processing in Biomedicine

2002

pdf
Semi-Supervised Maximum Entropy Based Approach to Acronym and Abbreviation Normalization in Medical Texts
Serguei Pakhomov
Proceedings of the 40th Annual Meeting of the Association for Computational Linguistics