@inproceedings{barros-etal-2026-superficiality,
    title = "The Superficiality Bias: Community Votes and Answer Utility in {P}ortuguese Health Question Answering",
    author = "Barros, Carlos Henrique Santos  and
      Sousa, Gustavo Figueredo Rodrigues de  and
      Sousa, Rog{\'e}rio Figueredo de",
    editor = "Souza, Marlo  and
      de-Dios-Flores, Iria  and
      Santos, Diana  and
      Freitas, Larissa  and
      Souza, Jackson Wilke da Cruz  and
      Ribeiro, Eug{\'e}nio",
    booktitle = "Proceedings of the 17th International Conference on Computational Processing of {P}ortuguese ({PROPOR} 2026) - Vol. 1",
    month = apr,
    year = "2026",
    address = "Salvador, Brazil",
    publisher = "Association for Computational Linguistics",
    url = "https://preview.aclanthology.org/ingest-dnd/2026.propor-1.113/",
    pages = "1074--1078",
    ISBN = "979-8-89176-387-6",
    abstract = "Supervised models trained on community-labeled data have shown promise in Health Question Answering (HQA), but relying on ``likes'' as a proxy for clinical usefulness remains controversial. This work investigates the alignment between automated predictions and human perception in Portuguese HQA. Using a subset of the SaudeBR-QA corpus, we compare a Random Forest classifier against a controlled evaluation conducted by laypeople and healthcare professionals. Our results reveal a recurring divergence that we term \textbf{Superficiality Bias}: human evaluators frequently validate very brief answers, whereas the classifier often labels these cases as non-useful under its learned criteria. Rather than indicating that the model is inherently more clinically accurate, this pattern suggests a misalignment between community feedback and feature-driven utility judgments. We argue that crowd-based labels in medical domains should be treated cautiously and complemented with more rigorous annotation protocols."
}