human evaluation 0.00327888
summarization evaluation 0.00301124
evaluation score 0.00291668
human scores 0.002671575
human metrics 0.002630195
automatic evaluation 0.002486438
automatic summarization 0.002476798
different summaries 0.00238338
evaluation metrics 0.002372195
summarization system 0.002255562
content evaluation 0.0022421200000000002
summarization systems 0.002196157
pyramid evaluation 0.002194959
human assessment 0.002165138
same summarization 0.002163038
human judgment 0.002160187
human assessors 0.002156481
human feedback 0.002116607
pyramid score 0.002090759
human mistakes 0.002081619
human ratings 0.002066705
evaluation consistency 0.002036898
ranking summarization 0.001994586
evaluation techniques 0.001940331
overall evaluation 0.001937418
score differences 0.0019265200000000001
tomatic evaluation 0.001914738
other summaries 0.001871242
automatic metrics 0.001837753
different content 0.00182783
summarization conditions 0.001822681
summarization track 0.001813514
summarization sys 0.0018042140000000002
tac summarization 0.001801291
final score 0.0017904589999999999
human 0.00176844
main summaries 0.00175471
summarizer score 0.001746993
autopyramid score 0.001729829
same topic 0.0017077379999999999
cal score 0.001705672
summary weight 0.001699318
summary level 0.0016964620000000001
summary pairs 0.00169536
identical summaries 0.0016888060000000002
total summary 0.001670231
automatic pyramid 0.0016605169999999998
individual summaries 0.00164929
training data 0.001622521
identical summary 0.001620066
tive summaries 0.001606152
pyramid scores 0.001587654
different sum 0.001562186
evaluation 0.00151044
summarization 0.0015008
different versions 0.001470129
different skills 0.001409297
different subset 0.001407874
score 0.00140624
different picture 0.001403898
different settings 0.001401027
different processes 0.001399619
inconsistent topic 0.001391799
content features 0.001363856
manual scores 0.001363029
readability scores 0.00135399
consistency data 0.00134798
ular topic 0.001343031
responsiveness scores 0.001336125
manual metrics 0.001321649
automatic summarizers 0.001311889
automatic evalua 0.001306216
summaries 0.00128723
valid information 0.0012831
automatic evalu 0.00127958
model 0.00124089
whole sentences 0.001231769
autopyramid scores 0.001226724
man metrics 0.001222402
summary 0.00121849
pyramid consistency 0.001210977
mid scores 0.001209754
tion metrics 0.001208217
scores sug 0.001201326
dict scores 0.001201326
many topics 0.001200701
same scus 0.001167446
ual metrics 0.001163946
text strings 0.001155883
conference data 0.001151694
content units 0.001144867
manual pyramid 0.001144413
same weight 0.0011430659999999999
marked data 0.00112158
responsiveness pyramid 0.001117509
regression methods 0.00109524
scu selection 0.001089487
pyramid process 0.001079956
mean correlation 0.001075833
final pyramid 0.001068738
