Sentiment analysis models are increasingly deployed to analyze political discourse, yet strong in-domain performance does not guarantee robustness under domain shift. We study cross-domain generalization in Hinglish (Hindi–English code-mixed) sentiment analysis by evaluating a fine-tuned XLM-RoBERTa classifier, trained on 29,000 general-domain Hinglish sentences, on a curated benchmark of politically oriented Hinglish text. While the model achieves 92.02% accuracy in-domain, performance drops to 71.83% under political domain shift. Error analysis reveals a pronounced directional bias with 48.9% of neutral political statements misclassified as negative, indicating a systematic neutrality-to-negative shift. In addition, 87.5% of incorrect predictions are assigned confidence scores above 95%, pointing to severe miscalibration under distribution shift. We further compare these results against an instruction-tuned large language model (Llama 3.3), which achieves 90.85% zero-shot accuracy and 94.37% accuracy with contextual prompting, while substantially reducing neutrality bias. Our findings indicate the need for domain-aware evaluation, calibration diagnostics, and explicit reporting of failure modes when deploying sentiment models in politically sensitive settings.

As large language models (LLMs) increasingly mediate political information across linguistic contexts, concerns emerge regarding cross-lingual consistency in political framing. We investigate whether multilingual LLMs generate systematically different rhetorical and semantic frames when prompted in English versus Arabic on the politically salient issue of migration. Focusing on two widely used models, i.e., GPT-4o and Jais-13B, we implement a controlled prompt design (N = 800 generations; 400 per language), to isolate language as the primary experimental variable. We introduce a mixed method evaluation framework that combines lexical frame analysis, statistical association testing, and qualitative discourse analysis. Our results show a significant association between language and framing distribution (χ2 = 43.32, p = 2.11 × 10−9). While security-oriented framing is prominent in both languages, English generations exhibit substantially higher rates of institutional and legislative framing, whereas Arabic generations show greater concentration in security and communitarian discourse. These findings indicate that input language acts as a conditioning signal that systematically modulates political framing within multilingual LLMs, even under controlled semantic prompts. We conceptualize this phenomenon as cross-lingual framing drift and discuss its implications for multilingual alignment, political bias evaluation, and global information ecosystems. We conclude by outlining an evaluative protocol for detecting language-conditioned asymmetries in generative models. We make all data, code, and experimental settings publicly available at: https://github.com/NRAwwad/-A-Cross-Lingual-Analysis-of-Political-Framing-in-English-and-Arabic.git.

In the digital age, cyberbullying (CB) poses a significant concern, impacting individuals as early as primary school and leading to severe or lasting consequences, including an increased risk of self-harm. CB incidents, are not limited to bullies and victims, but include bystanders with various roles, and usually have numerous sub-categories and variations of online harms. This position paper emphasises the complexity of CB incidents by drawing on insights from psychology, social sciences, and computational linguistics. While awareness of CB complexities is growing, existing computational techniques tend to oversimplify CB as a binary classification task, often relying on training datasets that capture peripheries of CB behaviours. Inconsistent definitions and categories of CB-related online harms across various platforms further complicates the issue. Ethical concerns arise when CB research involves children to role-play CB incidents to curate datasets. Through multi-disciplinary collaboration, we propose strategies for consideration when developing CB detection systems. We present our position on leveraging large language models (LLMs) such as Claude-2 and Llama2-Chat as an alternative approach to generate CB-related role-playing datasets. Our goal is to assist researchers, policymakers, and online platforms in making informed decisions regarding the automation of CB incident detection and intervention. By addressing these complexities, our research contributes to a more nuanced and effective approach to combating CB especially in young people.

In this paper, we present our approach to the GEM Shared Task at the INLG’24 Generation Challenges, which focuses on generating data-to-text in multiple languages, including low-resource languages, from WebNLG triples. We employ a combination of end-to-end and pipeline neural architectures for English text generation. To extend our methodology to Hindi, Korean, Arabic, and Swahili, we leverage a neural machine translation model. Our results demonstrate that our approach achieves competitive performance in the given task.