Factors Affecting the Performance of Automated Speaker Verification in Alzheimer’s Disease Clinical Trials

Malikeh Ehghaghi, Marija Stanojevic, Ali Akram, Jekaterina Novikova

Abstract
Detecting duplicate patient participation in clinical trials is a major challenge because repeated patients can undermine the credibility and accuracy of the trial’s findings and result in significant health and financial risks. Developing accurate automated speaker verification (ASV) models is crucial to verify the identity of enrolled individuals and remove duplicates, but the size and quality of data influence ASV performance. However, there has been limited investigation into the factors that can affect ASV capabilities in clinical environments. In this paper, we bridge the gap by conducting analysis of how participant demographic characteristics, audio quality criteria, and severity level of Alzheimer’s disease (AD) impact the performance of ASV utilizing a dataset of speech recordings from 659 participants with varying levels of AD, obtained through multiple speech tasks. Our results indicate that ASV performance: 1) is slightly better on male speakers than on female speakers; 2) degrades for individuals who are above 70 years old; 3) is comparatively better for non-native English speakers than for native English speakers; 4) is negatively affected by clinician interference, noisy background, and unclear participant speech; 5) tends to decrease with an increase in the severity level of AD. Our study finds that voice biometrics raise fairness concerns as certain subgroups exhibit different ASV performances owing to their inherent voice characteristics. Moreover, the performance of ASV is influenced by the quality of speech recordings, which underscores the importance of improving the data collection settings in clinical trials.
Anthology ID:
2023.clinicalnlp-1.27
Volume:
Proceedings of the 5th Clinical Natural Language Processing Workshop
Month:
July
Year:
2023
Address:
Toronto, Canada
Editors:
Tristan Naumann, Asma Ben Abacha, Steven Bethard, Kirk Roberts, Anna Rumshisky
Venue:
ClinicalNLP
SIG:
Publisher:
Association for Computational Linguistics
Note:
Pages:
218–227
Language:
URL:
https://aclanthology.org/2023.clinicalnlp-1.27
DOI:
10.18653/v1/2023.clinicalnlp-1.27
Bibkey:
Cite (ACL):
Malikeh Ehghaghi, Marija Stanojevic, Ali Akram, and Jekaterina Novikova. 2023. Factors Affecting the Performance of Automated Speaker Verification in Alzheimer’s Disease Clinical Trials. In Proceedings of the 5th Clinical Natural Language Processing Workshop, pages 218–227, Toronto, Canada. Association for Computational Linguistics.
Cite (Informal):
Factors Affecting the Performance of Automated Speaker Verification in Alzheimer’s Disease Clinical Trials (Ehghaghi et al., ClinicalNLP 2023)
Copy Citation:
PDF:
https://preview.aclanthology.org/dois-2013-emnlp/2023.clinicalnlp-1.27.pdf
PDF Search