README for the EMNLP-2012 submission supplementary files

This file explains the format of the data files used in the targeted
human evaluation results quoted in the EMNLP-2012 submission entitled
"Minimal Dependency Length in Realization Ranking". The files
themselves are being submitted as supplementary material along with
the paper. We employed 2 raters and had 3 test conditions. Each file
contains the judgments of 25 items presented to each rater for a given
test condition. The reference sentence (along with the WSJ id) was
provided and judges were requested to choose the version that best
expressed the meaning of the reference sentence in a grammatical and
fluent way. The judges had to choose either 'a' or 'b'.  The responses
are contained in the following files:

System-a=DEPLEN
System-b=DEPORD-NF
judge1.test1.xml  
judge2.test1.xml  

System-a=DEPLEN-NODIST
System-b=DEPORD-NODIST
judge1.test2.xml  
judge2.test2.xml  

System-a=DEPLEN-GLOBAL
System-b=GLOBAL
judge1.test3.xml  
judge2.test3.xml

The feature sets contained in these models are described in Table 3 of
the paper.