pdfCrowdsourcing Document Relevance Assessment with Mechanical TurkCatherine Grady | Matthew LeaseProceedings of the NAACL HLT 2010 Workshop on Creating Speech and Language Data with Amazon’s Mechanical Turk