<?xml version="1.0" encoding="UTF-8"?>
<modsCollection xmlns="http://www.loc.gov/mods/v3">
<mods ID="schwartz-etal-2017-assessing">
    <titleInfo>
        <title>Assessing Objective Recommendation Quality through Political Forecasting</title>
    </titleInfo>
    <name type="personal">
        <namePart type="given">H</namePart>
        <namePart type="given">Andrew</namePart>
        <namePart type="family">Schwartz</namePart>
        <role>
            <roleTerm authority="marcrelator" type="text">author</roleTerm>
        </role>
    </name>
    <name type="personal">
        <namePart type="given">Masoud</namePart>
        <namePart type="family">Rouhizadeh</namePart>
        <role>
            <roleTerm authority="marcrelator" type="text">author</roleTerm>
        </role>
    </name>
    <name type="personal">
        <namePart type="given">Michael</namePart>
        <namePart type="family">Bishop</namePart>
        <role>
            <roleTerm authority="marcrelator" type="text">author</roleTerm>
        </role>
    </name>
    <name type="personal">
        <namePart type="given">Philip</namePart>
        <namePart type="family">Tetlock</namePart>
        <role>
            <roleTerm authority="marcrelator" type="text">author</roleTerm>
        </role>
    </name>
    <name type="personal">
        <namePart type="given">Barbara</namePart>
        <namePart type="family">Mellers</namePart>
        <role>
            <roleTerm authority="marcrelator" type="text">author</roleTerm>
        </role>
    </name>
    <name type="personal">
        <namePart type="given">Lyle</namePart>
        <namePart type="family">Ungar</namePart>
        <role>
            <roleTerm authority="marcrelator" type="text">author</roleTerm>
        </role>
    </name>
    <originInfo>
        <dateIssued>2017-sep</dateIssued>
    </originInfo>
    <typeOfResource>text</typeOfResource>
    <relatedItem type="host">
        <titleInfo>
            <title>Proceedings of the 2017 Conference on Empirical Methods in Natural Language Processing</title>
        </titleInfo>
        <originInfo>
            <publisher>Association for Computational Linguistics</publisher>
            <place>
                <placeTerm type="text">Copenhagen, Denmark</placeTerm>
            </place>
        </originInfo>
        <genre authority="marcgt">conference publication</genre>
    </relatedItem>
    <abstract>Recommendations are often rated for their subjective quality, but few researchers have studied comment quality in terms of objective utility. We explore recommendation quality assessment with respect to both subjective (i.e. users’ ratings) and objective (i.e., did it influence? did it improve decisions?) metrics in a massive online geopolitical forecasting system, ultimately comparing linguistic characteristics of each quality metric. Using a variety of features, we predict all types of quality with better accuracy than the simple yet strong baseline of comment length. Looking at the most predictive content illustrates rater biases; for example, forecasters are subjectively biased in favor of comments mentioning business transactions or dealings as well as material things, even though such comments do not indeed prove any more useful objectively. Additionally, more complex sentence constructions, as evidenced by subordinate conjunctions, are characteristic of comments leading to objective improvements in forecasting.</abstract>
    <identifier type="citekey">schwartz-etal-2017-assessing</identifier>
    <identifier type="doi">10.18653/v1/D17-1250</identifier>
    <location>
        <url>https://aclanthology.org/D17-1250</url>
    </location>
    <part>
        <date>2017-sep</date>
        <extent unit="page">
            <start>2348</start>
            <end>2357</end>
        </extent>
    </part>
</mods>
</modsCollection>

%0 Conference Proceedings
%T Assessing Objective Recommendation Quality through Political Forecasting
%A Schwartz, H. Andrew
%A Rouhizadeh, Masoud
%A Bishop, Michael
%A Tetlock, Philip
%A Mellers, Barbara
%A Ungar, Lyle
%S Proceedings of the 2017 Conference on Empirical Methods in Natural Language Processing
%D 2017
%8 sep
%I Association for Computational Linguistics
%C Copenhagen, Denmark
%F schwartz-etal-2017-assessing
%X Recommendations are often rated for their subjective quality, but few researchers have studied comment quality in terms of objective utility. We explore recommendation quality assessment with respect to both subjective (i.e. users’ ratings) and objective (i.e., did it influence? did it improve decisions?) metrics in a massive online geopolitical forecasting system, ultimately comparing linguistic characteristics of each quality metric. Using a variety of features, we predict all types of quality with better accuracy than the simple yet strong baseline of comment length. Looking at the most predictive content illustrates rater biases; for example, forecasters are subjectively biased in favor of comments mentioning business transactions or dealings as well as material things, even though such comments do not indeed prove any more useful objectively. Additionally, more complex sentence constructions, as evidenced by subordinate conjunctions, are characteristic of comments leading to objective improvements in forecasting.
%R 10.18653/v1/D17-1250
%U https://aclanthology.org/D17-1250
%U https://doi.org/10.18653/v1/D17-1250
%P 2348-2357

Markdown (Informal)

[Assessing Objective Recommendation Quality through Political Forecasting](https://aclanthology.org/D17-1250) (Schwartz et al., EMNLP 2017)

• Assessing Objective Recommendation Quality through Political Forecasting (Schwartz et al., EMNLP 2017)

ACL

• H. Andrew Schwartz, Masoud Rouhizadeh, Michael Bishop, Philip Tetlock, Barbara Mellers, and Lyle Ungar. 2017. Assessing Objective Recommendation Quality through Political Forecasting. In Proceedings of the 2017 Conference on Empirical Methods in Natural Language Processing, pages 2348–2357, Copenhagen, Denmark. Association for Computational Linguistics.