Existing large language model (LLM) evaluation benchmarks primarily focus on English, while current multilingual tasks lack parallel questions that specifically assess cross-lingual reasoning abilities. This dual limitation makes it challenging to assess LLMs’ performance in the multilingual setting comprehensively. To fill this gap, we introduce MMLU-ProX, a comprehensive benchmark covering 29 languages, built on an English benchmark. Each language version consists of 11,829 identical questions, enabling direct cross-lingual comparisons. Additionally, to meet efficient evaluation needs, we provide a lite version containing 658 questions per language. To ensure the high quality of MMLU-ProX, we employ a rigorous development process that involves multiple powerful LLMs for translation, followed by expert review to ensure accurate expression, consistent terminology, and cultural relevance. Building on this, we systematically evaluate 36 state-of-the-art LLMs, including reasoning-enhanced and multilingual-optimized LLMs. The results reveal significant disparities in the multilingual capabilities of LLMs: While they perform well in high-resource languages, their performance declines markedly in low-resource languages, particularly for African languages. Through MMLU-ProX, we aim to advance the development of more inclusive AI systems and promote equitable access to technology across global contexts.

In this paper, we describe the participation of team ”CLP” in SemEval-2019 Task 3 “Con- textual Emotion Detection in Text” that aims to classify emotion of user utterance in tex- tual conversation. The submitted system is a deep learning architecture based on Hier- archical Attention Networks (HAN) and Em- bedding from Language Model (ELMo). The core of the architecture contains two represen- tation layers. The first one combines the out- puts of ELMo, hand-craft features and Bidi- rectional Long Short-Term Memory with At- tention (Bi-LSTM-Attention) to represent user utterance. The second layer use a Bi-LSTM- Attention encoder to represent the conversa- tion. Our system achieved F1 score of 0.7524 which outperformed the baseline model of the organizers by 0.1656.