ConfliBERT: A Pre-trained Language Model for Political Conflict and Violence
Yibo Hu, MohammadSaleh Hosseini, Erick Skorupa Parolin, Javier Osorio, Latifur Khan, Patrick Brandt, Vito D’Orazio
Abstract
Analyzing conflicts and political violence around the world is a persistent challenge in the political science and policy communities due in large part to the vast volumes of specialized text needed to monitor conflict and violence on a global scale. To help advance research in political science, we introduce ConfliBERT, a domain-specific pre-trained language model for conflict and political violence. We first gather a large domain-specific text corpus for language modeling from various sources. We then build ConfliBERT using two approaches: pre-training from scratch and continual pre-training. To evaluate ConfliBERT, we collect 12 datasets and implement 18 tasks to assess the models’ practical application in conflict research. Finally, we evaluate several versions of ConfliBERT in multiple experiments. Results consistently show that ConfliBERT outperforms BERT when analyzing political violence and conflict.- Anthology ID:
- 2022.naacl-main.400
- Volume:
- Proceedings of the 2022 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies
- Month:
- July
- Year:
- 2022
- Address:
- Seattle, United States
- Editors:
- Marine Carpuat, Marie-Catherine de Marneffe, Ivan Vladimir Meza Ruiz
- Venue:
- NAACL
- SIG:
- Publisher:
- Association for Computational Linguistics
- Note:
- Pages:
- 5469–5482
- Language:
- URL:
- https://preview.aclanthology.org/icon-24-ingestion/2022.naacl-main.400/
- DOI:
- 10.18653/v1/2022.naacl-main.400
- Cite (ACL):
- Yibo Hu, MohammadSaleh Hosseini, Erick Skorupa Parolin, Javier Osorio, Latifur Khan, Patrick Brandt, and Vito D’Orazio. 2022. ConfliBERT: A Pre-trained Language Model for Political Conflict and Violence. In Proceedings of the 2022 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, pages 5469–5482, Seattle, United States. Association for Computational Linguistics.
- Cite (Informal):
- ConfliBERT: A Pre-trained Language Model for Political Conflict and Violence (Hu et al., NAACL 2022)
- PDF:
- https://preview.aclanthology.org/icon-24-ingestion/2022.naacl-main.400.pdf
- Code
- eventdata/conflibert
- Data
- BookCorpus, WebText