Langforia: Language Pipelines for Annotating Large Collections of Documents

Marcus Klang, Pierre Nugues

Abstract
In this paper, we describe Langforia, a multilingual processing pipeline to annotate texts with multiple layers: formatting, parts of speech, named entities, dependencies, semantic roles, and entity links. Langforia works as a web service, where the server hosts the language processing components and the client, the input and result visualization. To annotate a text or a Wikipedia page, the user chooses an NLP pipeline and enters the text in the interface or selects the page URL. Once processed, the results are returned to the client, where the user can select the annotation layers s/he wants to visualize. We designed Langforia with a specific focus for Wikipedia, although it can process any type of text. Wikipedia has become an essential encyclopedic corpus used in many NLP projects. However, processing articles and visualizing the annotations are nontrivial tasks that require dealing with multiple markup variants, encodings issues, and tool incompatibilities across the language versions. This motivated the development of a new architecture. A demonstration of Langforia is available for six languages: English, French, German, Spanish, Russian, and Swedish at http://vilde.cs.lth.se:9000/ as well as a web API: http://vilde.cs.lth.se:9000/api. Langforia is also provided as a standalone library and is compatible with cluster computing.
Anthology ID:
C16-2016
Volume:
Proceedings of COLING 2016, the 26th International Conference on Computational Linguistics: System Demonstrations
Month:
December
Year:
2016
Address:
Osaka, Japan
Editor:
Hideo Watanabe
Venue:
COLING
SIG:
Publisher:
The COLING 2016 Organizing Committee
Note:
Pages:
74–78
Language:
URL:
https://aclanthology.org/C16-2016
DOI:
Bibkey:
Cite (ACL):
Marcus Klang and Pierre Nugues. 2016. Langforia: Language Pipelines for Annotating Large Collections of Documents. In Proceedings of COLING 2016, the 26th International Conference on Computational Linguistics: System Demonstrations, pages 74–78, Osaka, Japan. The COLING 2016 Organizing Committee.
Cite (Informal):
Langforia: Language Pipelines for Annotating Large Collections of Documents (Klang & Nugues, COLING 2016)
Copy Citation:
PDF:
https://preview.aclanthology.org/nschneid-patch-2/C16-2016.pdf
PDF Search