📄 MRS at SemEval-2025 Task 11: A Hybrid Approach for Bridging the Gap in Text-Based Emotion Detection

Summary

The paper addresses SemEval-2025 Task 11, Track A: multi-label emotion recognition in short English texts. For each text, the task is to determine which of five emotions most people would perceive in the speaker's words: anger, fear, joy, sadness, or surprise. The authors propose RoEmo, a system that combines models of two types. The first, a discriminative model, is RoBERTa-large with a two-layer MLP on top. It solves a separate binary task for each emotion. The second, a generative model, is EmoT5 from the EmoLLMs family, which was previously tuned on affective instructions. It was further fine-tuned on the competition data in a text-to-text format with a dedicated prompt. The models' outputs are combined with a logical OR: an emotion is considered present if at least one of them detects it. This raises recall at the cost of a slight drop in precision, but overall macro-F1 improves. On the test set, the ensemble outperformed both individual models and placed among the top systems in the competition. The result shows that even a very simple combination of models from different paradigms yields a noticeable gain on a subjective and ambiguous task.

Key points

• Track A data (English only): 2,768 training, 116 dev, and 2,767 test examples. Annotation was done via Amazon Mechanical Turk, and text length ranges from 4 to 121 tokens. Besides the five emotions, there are neutral examples, and a single text may contain several emotions at once.
• Both models were trained for 15 epochs with the AdamW optimizer (learning rate 2×10⁻⁵) on a single NVIDIA RTX A6000.
• Test: RoEmo achieves a macro-F1 of 0.7659, compared with 0.7552 for RoBERTa-large and 0.747 for EmoT5; RoEmo's micro-F1 is 0.7913.
• Dev: macro-F1 is 0.7821 for RoEmo, 0.7514 for RoBERTa-large, and 0.7625 for EmoT5.
• On the test set, texts with anger are the hardest (F1 ≈ 0.67) and texts with fear the easiest (≈ 0.86). The ensemble improved results for all emotions except joy, where RoEmo slightly underperformed both models.