One Year of Continuous and Automatic Data Gathering from Parliaments of European Union Member States

Ota Mikušek

Abstract
This paper provides insight into automatic parliamentary corpora development. One year ago, I created a simple set of tools designed to continuously and automatically download, process, and create corpora from speeches in the parliaments of European Union member states. Despite the existence of numerous corpora providing speeches from European Union parliaments, the tools are more focused on collecting and building such corpora with minimal human interaction. These tools have been operating continuously for over a year, gathering parliamentary data and extending corpora, which together have more than one billion words. However, the process of maintaining these tools has brought unforeseen challenges, including issues such as being blocked by some parliaments due to overloading the parliament with requests, the inability to access the most recent data of a parliament, and effectively managing interrupted connections. Additionally, potential problems that may arise in the future are provided, along with possible solutions. These include problems with data loss prevention and adaptation to changes in the sources from which speeches are downloaded.
Anthology ID:
2024.parlaclarin-1.22
Volume:
Proceedings of the IV Workshop on Creating, Analysing, and Increasing Accessibility of Parliamentary Corpora (ParlaCLARIN) @ LREC-COLING 2024
Month:
May
Year:
2024
Address:
Torino, Italia
Editors:
Darja Fiser, Maria Eskevich, David Bordon
Venues:
ParlaCLARIN | WS
SIG:
Publisher:
ELRA and ICCL
Note:
Pages:
149–153
Language:
URL:
https://aclanthology.org/2024.parlaclarin-1.22
DOI:
Bibkey:
Cite (ACL):
Ota Mikušek. 2024. One Year of Continuous and Automatic Data Gathering from Parliaments of European Union Member States. In Proceedings of the IV Workshop on Creating, Analysing, and Increasing Accessibility of Parliamentary Corpora (ParlaCLARIN) @ LREC-COLING 2024, pages 149–153, Torino, Italia. ELRA and ICCL.
Cite (Informal):
One Year of Continuous and Automatic Data Gathering from Parliaments of European Union Member States (Mikušek, ParlaCLARIN-WS 2024)
Copy Citation:
PDF:
https://preview.aclanthology.org/add_acl24_videos/2024.parlaclarin-1.22.pdf
PDF Search