2016
pdf
abs
Is all that Glitters in Machine Translation Quality Estimation really Gold?
Yvette Graham
|
Timothy Baldwin
|
Meghan Dowling
|
Maria Eskevich
|
Teresa Lynn
|
Lamia Tounsi
Proceedings of COLING 2016, the 26th International Conference on Computational Linguistics: Technical Papers
Human-targeted metrics provide a compromise between human evaluation of machine translation, where high inter-annotator agreement is difficult to achieve, and fully automatic metrics, such as BLEU or TER, that lack the validity of human assessment. Human-targeted translation edit rate (HTER) is by far the most widely employed human-targeted metric in machine translation, commonly employed, for example, as a gold standard in evaluation of quality estimation. Original experiments justifying the design of HTER, as opposed to other possible formulations, were limited to a small sample of translations and a single language pair, however, and this motivates our re-evaluation of a range of human-targeted metrics on a substantially larger scale. Results show significantly stronger correlation with human judgment for HBLEU over HTER for two of the nine language pairs we include and no significant difference between correlations achieved by HTER and HBLEU for the remaining language pairs. Finally, we evaluate a range of quality estimation systems employing HTER and direct assessment (DA) of translation adequacy as gold labels, resulting in a divergence in system rankings, and propose employment of DA for future quality estimation evaluations.
2014
pdf
bib
Proceedings of COLING 2014, the 25th International Conference on Computational Linguistics: System Demonstrations
Lamia Tounsi
|
Rafal Rak
Proceedings of COLING 2014, the 25th International Conference on Computational Linguistics: System Demonstrations
pdf
DCU: Aspect-based Polarity Classification for SemEval Task 4
Joachim Wagner
|
Piyush Arora
|
Santiago Cortes
|
Utsab Barman
|
Dasha Bogdanova
|
Jennifer Foster
|
Lamia Tounsi
Proceedings of the 8th International Workshop on Semantic Evaluation (SemEval 2014)
pdf
Cross-lingual Transfer Parsing for Low-Resourced Languages: An Irish Case Study
Teresa Lynn
|
Jennifer Foster
|
Mark Dras
|
Lamia Tounsi
Proceedings of the First Celtic Language Technology Workshop
2013
pdf
Sentiment Analysis of Political Tweets: Towards an Accurate Classifier
Akshat Bakliwal
|
Jennifer Foster
|
Jennifer van der Puil
|
Ron O’Brien
|
Lamia Tounsi
|
Mark Hughes
Proceedings of the Workshop on Language Analysis in Social Media
2012
pdf
abs
Automatic Extraction and Evaluation of Arabic LFG Resources
Mohammed Attia
|
Khaled Shaalan
|
Lamia Tounsi
|
Josef van Genabith
Proceedings of the Eighth International Conference on Language Resources and Evaluation (LREC'12)
This paper presents the results of an approach to automatically acquire large-scale, probabilistic Lexical-Functional Grammar (LFG) resources for Arabic from the Penn Arabic Treebank (ATB). Our starting point is the earlier, work of (Tounsi et al., 2009) on automatic LFG f(eature)-structure annotation for Arabic using the ATB. They exploit tree configuration, POS categories, functional tags, local heads and trace information to annotate nodes with LFG feature-structure equations. We utilize this annotation to automatically acquire grammatical function (dependency) based subcategorization frames and paths linking long-distance dependencies (LDDs). Many state-of-the-art treebank-based probabilistic parsing approaches are scalable and robust but often also shallow: they do not capture LDDs and represent only local information. Subcategorization frames and LDD paths can be used to recover LDDs from such parser output to capture deep linguistic information. Automatic acquisition of language resources from existing treebanks saves time and effort involved in creating such resources by hand. Moreover, data-driven automatic acquisition naturally associates probabilistic information with subcategorization frames and LDD paths. Finally, based on the statistical distribution of LDD path types, we propose empirical bounds on traditional regular expression based functional uncertainty equations used to handle LDDs in LFG.
2011
pdf
bib
Morphological Features for Parsing Morphologically-rich Languages: A Case of Arabic
Jon Dehdari
|
Lamia Tounsi
|
Josef van Genabith
Proceedings of the Second Workshop on Statistical Parsing of Morphologically Rich Languages
pdf
An Open-Source Finite State Morphological Transducer for Modern Standard Arabic
Mohammed Attia
|
Pavel Pecina
|
Antonio Toral
|
Lamia Tounsi
|
Josef van Genabith
Proceedings of the 9th International Workshop on Finite State Methods and Natural Language Processing
2010
pdf
bib
Statistical Parsing of Morphologically Rich Languages (SPMRL) What, How and Whither
Reut Tsarfaty
|
Djamé Seddah
|
Yoav Goldberg
|
Sandra Kuebler
|
Yannick Versley
|
Marie Candito
|
Jennifer Foster
|
Ines Rehbein
|
Lamia Tounsi
Proceedings of the NAACL HLT 2010 First Workshop on Statistical Parsing of Morphologically-Rich Languages
pdf
Handling Unknown Words in Statistical Latent-Variable Parsing Models for Arabic, English and French
Mohammed Attia
|
Jennifer Foster
|
Deirdre Hogan
|
Joseph Le Roux
|
Lamia Tounsi
|
Josef van Genabith
Proceedings of the NAACL HLT 2010 First Workshop on Statistical Parsing of Morphologically-Rich Languages
pdf
Automatic Extraction of Arabic Multiword Expressions
Mohammed Attia
|
Antonio Toral
|
Lamia Tounsi
|
Pavel Pecina
|
Josef van Genabith
Proceedings of the 2010 Workshop on Multiword Expressions: from Theory to Applications
pdf
abs
An Automatically Built Named Entity Lexicon for Arabic
Mohammed Attia
|
Antonio Toral
|
Lamia Tounsi
|
Monica Monachini
|
Josef van Genabith
Proceedings of the Seventh International Conference on Language Resources and Evaluation (LREC'10)
We have adapted and extended the automatic Multilingual, Interoperable Named Entity Lexicon approach to Arabic, using Arabic WordNet (AWN) and Arabic Wikipedia (AWK). First, we extract AWNs instantiable nouns and identify the corresponding categories and hyponym subcategories in AWK. Then, we exploit Wikipedia inter-lingual links to locate correspondences between articles in ten different languages in order to identify Named Entities (NEs). We apply keyword search on AWK abstracts to provide for Arabic articles that do not have a correspondence in any of the other languages. In addition, we perform a post-processing step to fetch further NEs from AWK not reachable through AWN. Finally, we investigate diacritization using matching with geonames databases, MADA-TOKAN tools and different heuristics for restoring vowel marks of Arabic NEs. Using this methodology, we have extracted approximately 45,000 Arabic NEs and built, to the best of our knowledge, the largest, most mature and well-structured Arabic NE lexical resource to date. We have stored and organised this lexicon following the LMF ISO standard. We conduct a quantitative and qualitative evaluation against a manually annotated gold standard and achieve precision scores from 95.83% (with 66.13% recall) to 99.31% (with 61.45% recall) according to different values of a threshold.
pdf
abs
Arabic Parsing Using Grammar Transforms
Lamia Tounsi
|
Josef van Genabith
Proceedings of the Seventh International Conference on Language Resources and Evaluation (LREC'10)
We investigate Arabic Context Free Grammar parsing with dependency annotation comparing lexicalised and unlexicalised parsers. We study how morphosyntactic as well as function tag information percolation in the form of grammar transforms (Johnson, 1998, Kulick et al., 2006) affects the performance of a parser and helps dependency assignment. We focus on the three most frequent functional tags in the Arabic Penn Treebank: subjects, direct objects and predicates . We merge these functional tags with their phrasal categories and (where appropriate) percolate case information to the non-terminal (POS) category to train the parsers. We then automatically enrich the output of these parsers with full dependency information in order to annotate trees with Lexical Functional Grammar (LFG) f-structure equations with produce f-structures, i.e. attribute-value matrices approximating to basic predicate-argument-adjunct structure representations. We present a series of experiments evaluating how well lexicalized, history-based, generative (Bikel) as well as latent variable PCFG (Berkeley) parsers cope with the enriched Arabic data. We measure quality and coverage of both the output trees and the generated LFG f-structures. We show that joint functional and morphological information percolation improves both the recovery of trees as well as dependency results in the form of LFG f-structures.
2009
pdf
Automatic Treebank-Based Acquisition of Arabic LFG Dependency Structures
Lamia Tounsi
|
Mohammed Attia
|
Josef van Genabith
Proceedings of the EACL 2009 Workshop on Computational Approaches to Semitic Languages