Elamur Research⌕
☰
home / papers / 2025 / EmoHopeSpeech: An Annotated Dataset of Emotions and Hope Speech in English and Arabic

EmoHopeSpeech: An Annotated Dataset of Emotions and Hope Speech in English and Arabic

Paper · 2025 · Wajdi Zaghouani and Md. Rafiul Biswas

Summary

EmoHopeSpeech addresses the lack of corpora that label both emotions and hope speech, especially for Arabic. The authors reused existing emotion-labelled data: three Arabic sources (a poetry emotion set, Emotional Tone, and a multilabel hate speech corpus) and one large English emotion collection. They kept texts of 6–79 words and added several new annotation layers: emotion intensity, emotion complexity, emotion cause, a five-way hope speech category, and hope subcategories. Ten native Arabic annotators from several Arab countries and five English annotators worked from bilingual guidelines with examples, which were refined through training sessions and discussions to resolve disagreements. Chi-square tests show that every emotion layer is significantly associated with the hope labels. Baselines for predicting hope categories (LR, NB, AraBERT, BERT-base-uncased) reach a top F1 of about 0.45–0.47. The abstract claims Fleiss' Kappa of 0.75–0.85 and F1 of 0.69, but the tables in the body report roughly 0.31–0.52 and lower F1, so treat the reliability claims with caution. The main value is a first joint emotion-plus-hope resource for Arabic and for cross-cultural comparison.

Key points

  • Corpus size: 27,456 Arabic rows were kept from 49,517. The English set was filtered from 393,822 rows down to 10,036, but only 4,036 English rows received the new annotations.
  • Label scheme:
    • Intensity: Low / Medium / High / N/A.
    • Complexity: Simple / Medium / Complex / N/A.
    • Cause: Internal Reflection, Relationship, External Event, Achievement/Failure, Unclear.
    • Hope category: Hope, Not Hope, Counter Speech, Neutral, Hate/Negativity.
    • Hope subcategory: Inspirational, Solidarity, Spiritual, Resilience.
  • Label distributions differ by language. Arabic texts are mostly of medium complexity (52%), while English texts are mostly simple (49%). Arabic causes are dominated by Relationship (40%) and Internal Reflection (39%).
  • The strongest association is between the emotion label and the hope category (χ² 1502.6 for Arabic, 160.5 for English).
  • Agreement was measured on 200 shared items and is higher for Arabic:
    • Intensity: 0.52 Arabic / 0.44 English.
    • Cause: 0.43 / 0.39.
    • Hope subcategory: 0.36 / 0.31.
  • Best baselines: AraBERT reaches F1 0.47 (accuracy 0.49) on Arabic; BERT reaches F1 0.45 on English.

Interesting ideas

  • I1. The intensity guidelines rewrite one situation (worry about a test) at low, medium, high and no-emotion levels, giving annotators minimal pairs that sharpen the boundaries between grades.
  • I2. Emotion cause is annotated as its own layer, separating what triggers a feeling from the feeling itself.
  • I3. A dedicated "complex" label covers texts that hold several emotions at once (excited about a job yet nervous about leaving).
  • I4. The authors use chi-square tests between annotation layers as a sanity check that related labels actually co-vary.
  • I5. Reporting agreement per label shows which layers are inherently subjective, such as cause and subcategory. The gap between the abstract's numbers and the tables also shows why headline figures need checking.

For Mova

How these ideas could serve Mova, our small open language model — its files, stages and tests:

  • I1 (world): Build similar graded contrast sets to check that emotional states in the text world get the right degree value, not just the right emotion type.
  • I2 (frames): Fill a Stimulus role in emotion frames and type it (relationship, external event, achievement), so answers to "why is X sad?" point to an explicit cause node.
  • I3 (world): Allow one experiencer to hold several concurrent emotional states, each linked to its own cause, instead of collapsing them into a single polarity.
  • I4 (learn): Test cross-layer dependencies in silver data, for example emotion frames against speech-act labels; if layers that should be linked look independent, treat that as a noise signal or a negative control.
  • I5 (learn): Report per-layer agreement among witness parsers and LLM teachers, and down-weight or re-specify layers that stay low.

Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.

 Fields, topics and concepts

📚 Shelves 5The field of science this work belongs to (OpenAlex fields).Computer Science · 16,667 worksArtificial Intelligence · 16,194 worksPsychology · 1,552 worksExperimental and Cognitive Psychology · 870 worksSocial Psychology · 306 works

🧭 Topics 3Research subjects and languages the work is about.English · 2,263 worksSentiment & emotion · 972 worksSemitic languages · 2,908 works

Cite

Cite as Wajdi Zaghouani and Md. Rafiul Biswas. 2025. EmoHopeSpeech: An Annotated Dataset of Emotions and Hope Speech in English and Arabic. arXiv:2505.11959.

The paper itself, not this page. BibTeX from the library's record of the work. DOI 10.48550/arxiv.2505.11959 · arXiv 2505.11959

@misc{zaghouani2025emohopespeech,
  title = {{EmoHopeSpeech: An Annotated Dataset of Emotions and Hope Speech in English and Arabic}},
  author = {Wajdi Zaghouani and Md. Rafiul Biswas},
  year = {2025},
  doi = {10.48550/arxiv.2505.11959},
  eprint = {2505.11959},
  archivePrefix = {arXiv},
  url = {https://doi.org/10.48550/arxiv.2505.11959}
}

References

11 references, 4 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

  1. Al-Khatib, A.; and El-Beltagy, S. R. 2018. Emotional tone detection in arabic tweets. In Computational Linguistics and Intelligent Text Processing: 18th International Conference, CICLing 2017, Budapest, Hungary, April 17–23, 2017, Revised Selected Papers, Part II 18, 105–114. Springer. find
  2. Alrefaie, M. T. 2019. Arabic Stop Words. find
  3. Anjali, S. 2024. Emotion analysis based on text. find
  4. Antoun, W.; Baly, F.; and Hajj, H. 2020. Arabert: Transformer-based model for arabic language understanding. arXiv preprint arXiv:2003.00104. ↗
  5. Babanne, V.; Borgaonkar, M.; Katta, M.; Kudale, P.; and Deshpande, V. 2020. Emotion based personalized recommendation system. Int. Res. J. Eng. Technol.(IRJET), 7: 701–705. find
  6. Balouchzahi, F.; Sidorov, G.; and Gelbukh, A. 2023. Polyhope: Two-level hope speech detection from tweets. Expert Systems with Applications, 225: 120078. ↗
  7. Chakravarthi, B. R. 2020. HopeEDI: A multilingual hope speech detection dataset for equality, diversity, and inclusion. In Proceedings of the Third Workshop on Computational Modeling of People’s Opinions, Personality, and Emotion’s in Social Media, 41–53.
  8. Chatterjee, A.; Narahari, K. N.; Joshi, M.; and Agrawal, P. 2019. SemEval-2019 Task 3: EmoContext Contextual Emotion Detection in Text. In Proceedings of the 13th International Workshop on Semantic Evaluation, 39–48. Minneapolis, Minnesota, USA: Association for Computational Linguistics.
  9. Chintala, S. 2024. Emotion AI in Business Intelligence: Under-standing Customer Sentiments and Behaviors. Central Asian Journal of Mathematical Theory and Com-puter Sciences, 5(3): 205–212. find
  10. Devlin, J.; Chang, M.; Lee, K.; and Toutanova, K. 2018. BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding. CoRR, abs/1810.04805. ↗
  11. Dheeraj, K.; and Ramakrishnudu, T. 2021. Negative emotions detection on online mental-health related patients texts *Table 12: Inter-Annotator Agreement (IAA) - Fleiss’ Kappa for Arabic and English Data* find

Updated

↑ back to top

This page in Markdown → https://md.research.elamur.ai/free/en/papers/zaghouani-2025-emohopespeech-annotated-dataset-emotions-hope/