2022
pdf
abs
Reflections on 30 Years of Language Resource Development and Sharing
Christopher Cieri
|
Mark Liberman
|
Sunghye Cho
|
Stephanie Strassel
|
James Fiumara
|
Jonathan Wright
Proceedings of the Thirteenth Language Resources and Evaluation Conference
The Linguistic Data Consortium was founded in 1992 to solve the problem that limitations in access to shareable data was impeding progress in Human Language Technology research and development. At the time, DARPA had adopted the common task research management paradigm to impose additional rigor on their programs by also providing shared objectives, data and evaluation methods. Early successes underscored the promise of this paradigm but also the need for a standing infrastructure to host and distribute the shared data. During LDC’s initial five year grant, it became clear that the demand for linguistic data could not easily be met by the existing providers and that a dedicated data center could add capacity first for data collection and shortly thereafter for annotation. The expanding purview required expansions of LDC’s technical infrastructure including systems support and software development. An open question for the center would be its role in other kinds of research beyond data development. Over its 30 years history, LDC has performed multiple roles ranging from neutral, independent data provider to multisite programs, to creator of exploratory data in tight collaboration with system developers, to research group focused on data intensive investigations.
pdf
bib
Proceedings of the 2nd Workshop on Novel Incentives in Data Collection from People: models, implementations, challenges and results within LREC 2022
Chris Callison-Burch
|
Christopher Cieri
|
James Fiumara
|
Mark Liberman
Proceedings of the 2nd Workshop on Novel Incentives in Data Collection from People: models, implementations, challenges and results within LREC 2022
pdf
bib
abs
The NIEUW Project: Developing Language Resources through Novel Incentives
James Fiumara
|
Christopher Cieri
|
Mark Liberman
|
Chris Callison-Burch
|
Jonathan Wright
|
Robert Parker
Proceedings of the 2nd Workshop on Novel Incentives in Data Collection from People: models, implementations, challenges and results within LREC 2022
This paper provides an overview and update on the Linguistic Data Consortium’s (LDC) NIEUW (Novel Incentives and Workflows) project supported by the National Science Foundation and part of LDC’s larger goal of improving the cost, variety, scale, and quality of language resources available for education, research, and technology development. NIEUW leverages the power of novel incentives to elicit linguistic data and annotations from a wide variety of contributors including citizen scientists, game players, and language students and professionals. In order to align appropriate incentives with the various contributors, LDC has created three distinct web portals to bring together researchers and other language professionals with participants best suited to their project needs. These portals include LanguageARC designed for citizen scientists, Machina Pro Linguistica designed for students and language professionals, and LingoBoingo designed for game players. The design, interface, and underlying tools for each web portal were developed to appeal to the different incentives and motivations of their respective target audiences.
pdf
abs
Identifying stable speech-language markers of autism in children: Preliminary evidence from a longitudinal telephony-based study
Sunghye Cho
|
Riccardo Fusaroli
|
Maggie Rose Pelella
|
Kimberly Tena
|
Azia Knox
|
Aili Hauptmann
|
Maxine Covello
|
Alison Russell
|
Judith Miller
|
Alison Hulink
|
Jennifer Uzokwe
|
Kevin Walker
|
James Fiumara
|
Juhi Pandey
|
Christopher Chatham
|
Christopher Cieri
|
Robert Schultz
|
Mark Liberman
|
Julia Parish-morris
Proceedings of the Eighth Workshop on Computational Linguistics and Clinical Psychology
This study examined differences in linguistic features produced by autistic and neurotypical (NT) children during brief picture descriptions, and assessed feature stability over time. Weekly speech samples from well-characterized participants were collected using a telephony system designed to improve access for geographically isolated and historically marginalized communities. Results showed stable group differences in certain acoustic features, some of which may potentially serve as key outcome measures in future treatment studies. These results highlight the importance of eliciting semi-structured speech samples in a variety of contexts over time, and adds to a growing body of research showing that fine-grained naturalistic communication features hold promise for intervention research.
2020
pdf
bib
Proceedings of the LREC 2020 Workshop on "Citizen Linguistics in Language Resource Development"
James Fiumara
|
Christopher Cieri
|
Mark Liberman
|
Chris Callison-Burch
Proceedings of the LREC 2020 Workshop on "Citizen Linguistics in Language Resource Development"
pdf
bib
abs
LanguageARC: Developing Language Resources Through Citizen Linguistics
James Fiumara
|
Christopher Cieri
|
Jonathan Wright
|
Mark Liberman
Proceedings of the LREC 2020 Workshop on "Citizen Linguistics in Language Resource Development"
This paper introduces the citizen science platform, LanguageARC, developed within the NIEUW (Novel Incentives and Workflows) project supported by the National Science Foundation under Grant No. 1730377. LanguageARC is a community-oriented online platform bringing together researchers and “citizen linguists” with the shared goal of contributing to linguistic research and language technology development. Like other Citizen Science platforms and projects, LanguageARC harnesses the power and efforts of volunteers who are motivated by the incentives of contributing to science, learning and discovery, and belonging to a community dedicated to social improvement. Citizen linguists contribute language data and judgments by participating in research tasks such as classifying regional accents from audio clips, recording audio of picture descriptions and answering personality questionnaires to create baseline data for NLP research into autism and neurodegenerative conditions. Researchers can create projects on Language ARC without any coding or HTML required using our Project Builder Toolkit.
pdf
abs
LanguageARC - a tutorial
Christopher Cieri
|
James Fiumara
Proceedings of the LREC 2020 Workshop on "Citizen Linguistics in Language Resource Development"
LanguageARC is a portal that offers citizen linguists opportunities to contribute to language related research. It also provides researchers with infrastructure for easily creating data collection and annotation tasks on the portal and potentially connecting with contributors. This document describes LanguageARC’s main features and operation for researchers interested in creating new projects and or using the resulting data.
pdf
abs
A Progress Report on Activities at the Linguistic Data Consortium Benefitting the LREC Community
Christopher Cieri
|
James Fiumara
|
Stephanie Strassel
|
Jonathan Wright
|
Denise DiPersio
|
Mark Liberman
Proceedings of the Twelfth Language Resources and Evaluation Conference
This latest in a series of Linguistic Data Consortium (LDC) progress reports to the LREC community does not describe any single language resource, evaluation campaign or technology but sketches the activities, since the last report, of a data center devoted to supporting the work of LREC attendees among other research communities. Specifically, we describe 96 new corpora released in 2018-2020 to date, a new technology evaluation campaign, ongoing activities to support multiple common task human language technology programs, and innovations to advance the methodology of language data collection and annotation.
2018
pdf
Introducing NIEUW: Novel Incentives and Workflows for Eliciting Linguistic Data
Christopher Cieri
|
James Fiumara
|
Mark Liberman
|
Chris Callison-Burch
|
Jonathan Wright
Proceedings of the Eleventh International Conference on Language Resources and Evaluation (LREC 2018)
2012
pdf
abs
Creating HAVIC: Heterogeneous Audio Visual Internet Collection
Stephanie Strassel
|
Amanda Morris
|
Jonathan Fiscus
|
Christopher Caruso
|
Haejoong Lee
|
Paul Over
|
James Fiumara
|
Barbara Shaw
|
Brian Antonishek
|
Martial Michel
Proceedings of the Eighth International Conference on Language Resources and Evaluation (LREC'12)
Linguistic Data Consortium and the National Institute of Standards and Technology are collaborating to create a large, heterogeneous annotated multimodal corpus to support research in multimodal event detection and related technologies. The HAVIC (Heterogeneous Audio Visual Internet Collection) Corpus will ultimately consist of several thousands of hours of unconstrained user-generated multimedia content. HAVIC has been designed with an eye toward providing increased challenges for both acoustic and video processing technologies, focusing on multi-dimensional variation inherent in user-generated multimedia content. To date the HAVIC corpus has been used to support the NIST 2010 and 2011 TRECVID Multimedia Event Detection (MED) Evaluations. Portions of the corpus are expected to be released in LDC's catalog in the coming year, with the remaining segments being published over time after their use in the ongoing MED evaluations.