Developing and Evaluating Annotation Procedures for Twitter Data during Hazard Events
Kevin Stowe, Martha Palmer, Jennings Anderson, Marina Kogan, Leysia Palen, Kenneth M. Anderson, Rebecca Morss, Julie Demuth, Heather Lazrus
Abstract
When a hazard such as a hurricane threatens, people are forced to make a wide variety of decisions, and the information they receive and produce can influence their own and others’ actions. As social media grows more popular, an increasing number of people are using social media platforms to obtain and share information about approaching threats and discuss their interpretations of the threat and their protective decisions. This work aims to improve understanding of natural disasters through social media and provide an annotation scheme to identify themes in user’s social media behavior and facilitate efforts in supervised machine learning. To that end, this work has three contributions: (1) the creation of an annotation scheme to consistently identify hazard-related themes in Twitter, (2) an overview of agreement rates and difficulties in identifying annotation categories, and (3) a public release of both the dataset and guidelines developed from this scheme.- Anthology ID:
- W18-4915
- Volume:
- Proceedings of the Joint Workshop on Linguistic Annotation, Multiword Expressions and Constructions (LAW-MWE-CxG-2018)
- Month:
- August
- Year:
- 2018
- Address:
- Santa Fe, New Mexico, USA
- Editors:
- Agata Savary, Carlos Ramisch, Jena D. Hwang, Nathan Schneider, Melanie Andresen, Sameer Pradhan, Miriam R. L. Petruck
- Venues:
- LAW | MWE
- SIGs:
- SIGLEX | SIGANN
- Publisher:
- Association for Computational Linguistics
- Note:
- Pages:
- 133–143
- Language:
- URL:
- https://aclanthology.org/W18-4915
- DOI:
- Cite (ACL):
- Kevin Stowe, Martha Palmer, Jennings Anderson, Marina Kogan, Leysia Palen, Kenneth M. Anderson, Rebecca Morss, Julie Demuth, and Heather Lazrus. 2018. Developing and Evaluating Annotation Procedures for Twitter Data during Hazard Events. In Proceedings of the Joint Workshop on Linguistic Annotation, Multiword Expressions and Constructions (LAW-MWE-CxG-2018), pages 133–143, Santa Fe, New Mexico, USA. Association for Computational Linguistics.
- Cite (Informal):
- Developing and Evaluating Annotation Procedures for Twitter Data during Hazard Events (Stowe et al., LAW-MWE 2018)
- PDF:
- https://preview.aclanthology.org/naacl24-info/W18-4915.pdf