Using LLMs to Build a Database of Climate Extreme Impacts
Ni Li, Shorouq Zahra, Mariana Brito, Clare Flynn, Olof Görnerup, Koffi Worou, Murathan Kurfali, Chanjuan Meng, Wim Thiery, Jakob Zscheischler, Gabriele Messori, Joakim Nivre
Abstract
To better understand how extreme climate events impact society, we need to increase the availability of accurate and comprehensive information about these impacts. We propose a method for building large-scale databases of climate extreme impacts from online textual sources, using LLMs for information extraction in combination with more traditional NLP techniques to improve accuracy and consistency. We evaluate the method against a small benchmark database created by human experts and find that extraction accuracy varies for different types of information. We compare three different LLMs and find that, while the commercial GPT-4 model gives the best performance overall, the open-source models Mistral and Mixtral are competitive for some types of information.- Anthology ID:
- 2024.climatenlp-1.7
- Volume:
- Proceedings of the 1st Workshop on Natural Language Processing Meets Climate Change (ClimateNLP 2024)
- Month:
- August
- Year:
- 2024
- Address:
- Bangkok, Thailand
- Editors:
- Dominik Stammbach, Jingwei Ni, Tobias Schimanski, Kalyan Dutia, Alok Singh, Julia Bingler, Christophe Christiaen, Neetu Kushwaha, Veruska Muccione, Saeid A. Vaghefi, Markus Leippold
- Venues:
- ClimateNLP | WS
- SIG:
- Publisher:
- Association for Computational Linguistics
- Note:
- Pages:
- 93–110
- Language:
- URL:
- https://aclanthology.org/2024.climatenlp-1.7
- DOI:
- 10.18653/v1/2024.climatenlp-1.7
- Cite (ACL):
- Ni Li, Shorouq Zahra, Mariana Brito, Clare Flynn, Olof Görnerup, Koffi Worou, Murathan Kurfali, Chanjuan Meng, Wim Thiery, Jakob Zscheischler, Gabriele Messori, and Joakim Nivre. 2024. Using LLMs to Build a Database of Climate Extreme Impacts. In Proceedings of the 1st Workshop on Natural Language Processing Meets Climate Change (ClimateNLP 2024), pages 93–110, Bangkok, Thailand. Association for Computational Linguistics.
- Cite (Informal):
- Using LLMs to Build a Database of Climate Extreme Impacts (Li et al., ClimateNLP-WS 2024)
- PDF:
- https://preview.aclanthology.org/autopr/2024.climatenlp-1.7.pdf