annotation task 0.00306377
annotation tasks 0.002428098
annotation quality 0.002398034
language annotation 0.0023721799999999998
standard annotation 0.0023524039999999998
gold annotation 0.002278806
annotation this 0.002133971
linguistic annotation 0.002123522
scale annotation 0.002104796
event annotation 0.002067249
online annotation 0.002059629
labels task 0.002020474
task labels 0.002020474
annotation projects 0.002017371
temporal annotation 0.002011529
entailment task 0.001890672
data quality 0.0018171440000000001
learning data 0.001813092
annotation 0.00179565
expert data 0.001782718
data set 0.001738895
same data 0.001711954
rte task 0.001638412
similar task 0.001619044
general task 0.0016173989999999998
this task 0.001606441
task design 0.001589634
emotion task 0.00158936
task results 0.001580473
ing data 0.001578987
sample task 0.00156676
previous task 0.001563266
recognition task 0.001562612
similarity task 0.00153454
sense labels 0.001532886
ment task 0.00153213
affect task 0.00151853
notation task 0.001505854
labels labels 0.001504708
semeval task 0.0014982069999999999
labeled data 0.0014869050000000002
gorical task 0.001471437
task designer 0.001471437
task descriptions 0.001471437
numeric data 0.001470759
data source 0.001452289
amt data 0.001452156
beler data 0.0014223320000000001
categorical data 0.0014222590000000002
sense word 0.0013893030000000002
word sense 0.0013893030000000002
standard labels 0.001309108
language annotations 0.0012830580000000001
expert annotations 0.001274486
time labels 0.001269321
standard annotations 0.0012632820000000001
noise model 0.001257874
label quality 0.00125663
cost time 0.001237153
gold labels 0.00123551
labels figure 0.001223127
expert label 0.0012222040000000002
such tasks 0.00120554
word pairs 0.001191126
sentence pairs 0.0011356
average expert 0.001111526
expert annotator 0.0011057950000000001
product model 0.001094764
single expert 0.00106742
overall quality 0.001064364
average system 0.001045268
true labels 0.0010443779999999999
gold standard 0.00103991
standard test 0.001035318
many tasks 0.001033923
expert annotators 0.0010329570000000001
unigram model 0.001032909
dataset expert 0.001026797
single set 0.001023597
noisy labels 0.001023105
same set 0.001021329
independent annotations 0.001021021
small tasks 0.001018277
standard dataset 0.001015593
standard examples 0.001009359
labeling tasks 0.001005871
affect labels 0.001002764
test set 0.0010026990000000001
sense disambiguation 0.001000878
unique annotations 0.001000681
multinomial model 9.994070000000001E-4
individual expert 9.99005E-4
training set 9.87438E-4
ilar model 9.858150000000001E-4
sense disam 9.84881E-4
other workers 9.77557E-4
raw labels 9.68763E-4
total annotations 9.61894E-4
financial cost 9.57147E-4
system performance 9.56963E-4
