Enkelejda Kasneci
2026
MemeScouts@LT-EDI 2026: Asking the Right Questions - Prompted Weak Supervision for Meme Hate Speech Detection
Ivo Bueno | Lea Hirlimann | Enkelejda Kasneci
Proceedings of the Sixth Workshop on Language Technology for Equality, Diversity, Inclusion
Ivo Bueno | Lea Hirlimann | Enkelejda Kasneci
Proceedings of the Sixth Workshop on Language Technology for Equality, Diversity, Inclusion
Detecting hate speech in memes is challenging due to their multimodal nature and subtle, culturally grounded cues such as sarcasm and context. While recent vision-language models (VLMs) enable joint reasoning over text and images, end-to-end prompting can be brittle, as a single prediction must resolve target, stance, implicitness, and irony. These challenges are amplified in multilingual settings. We propose a prompted weak supervision (PWS) approach that decomposes meme understanding into targeted, question-based labeling functions with constrained answer options for homophobia and transphobia detection in the LT-EDI 2026 shared task. Using a quantized Qwen3-VLM to extract features by answering targeted questions, our method outperforms direct VLM classification, with substantial gains for Chinese and Hindi, ranking 1st in English, 2nd in Chinese, and 3rd in Hindi. Iterative refinement via error-driven LF expansion and feature pruning reduces redundancy and improves generalization. Our results highlight the effectiveness of prompted weak supervision for multilingual multimodal hate speech detection.
From Scoring to Explanations: Evaluating SHAP and LLM Rationales for Rubric-based Teaching Quality Assessment
Ivo Bueno | Babette Bühler | Philipp Stark | Tim Fütterer | Ulrich Trautwein | Dorottya Demszky | Heather Hill | Enkelejda Kasneci
Findings of the Association for Computational Linguistics: ACL 2026
Ivo Bueno | Babette Bühler | Philipp Stark | Tim Fütterer | Ulrich Trautwein | Dorottya Demszky | Heather Hill | Enkelejda Kasneci
Findings of the Association for Computational Linguistics: ACL 2026
Automated scoring models are increasingly used to assign rubric-based quality ratings to complex language performances, including classroom transcripts, yet they typically provide little insight into why a particular score is produced. We propose a general framework for sentence-level interpretability of rubric-based scoring that combines model-agnostic Shapley-value attributions with rationales generated by large language models (LLMs). Instantiated on the Quality of Feedback dimension of the CLASS framework using the NCTE corpus, the framework enables systematic comparison of fine-tuned pretrained language models (PLMs) and prompted LLMs on both scoring performance and explanation faithfulness. Across 6k annotated transcript segments, fine-tuned PLMs outperform LLMs in prediction accuracy but exhibit label compression toward mid-scale scores. Deletion-based tests show that SHAP identifies sentences that reliably drive model predictions, producing typically larger and more coherent prediction shifts than LLM-generated rationales. Cross-model analyses further reveal that SHAP attributions transfer robustly across architectures, whereas LLM rationales exert limited and inconsistent influence. Overall, the findings demonstrate that SHAP provides more faithful and transferable explanations for rubric-based scoring, and that the proposed framework offers a principled basis for evaluating both scoring models and their explanations in high-stakes educational settings and other rubric-based language assessment tasks.
2025
LLM-Human Alignment in Evaluating Teacher Questioning Practices: Beyond Ratings to Explanation
Ruikun Hou | Tim Fütterer | Babette Bühler | Patrick Schreyer | Peter Gerjets | Ulrich Trautwein | Enkelejda Kasneci
Proceedings of the Artificial Intelligence in Measurement and Education Conference (AIME-Con): Full Papers
Ruikun Hou | Tim Fütterer | Babette Bühler | Patrick Schreyer | Peter Gerjets | Ulrich Trautwein | Enkelejda Kasneci
Proceedings of the Artificial Intelligence in Measurement and Education Conference (AIME-Con): Full Papers
This study investigates the alignment between large language models (LLMs) and human raters in assessing teacher questioning practices, moving beyond rating agreement to the evidence selected to justify their decisions. Findings highlight LLMs’ potential to support large-scale classroom observation through interpretable, evidence-based scoring, with possible implications for concrete teacher feedback.