Elamur Research⌕
☰
home / papers / 2018 / Enabling Deep Learning of Emotion With First-Person Seed Expressions

Enabling Deep Learning of Emotion With First-Person Seed Expressions

Paper · 2018 · Hassan Alhuzali, Muhammad Abdul-Mageed and Lyle Ungar

Summary

Emotion detection lags behind plain sentiment analysis mainly because labelled data is scarce, and Arabic has very few emotion resources. The authors gather Arabic tweets in Modern Standard Arabic and Dialectal Arabic. A tweet is collected only if it contains a first-person seed phrase, the pronoun "I" followed by an emotion word, covering Plutchik's eight primary emotions. Four native speakers check a sample of 7,268 tweets (LAMA). Instead of judging emotion presence and intensity together, they first decide whether the tweet is relevant and then grade intensity, and this split raises inter-rater agreement. The remaining automatically labelled tweets, after cleaning, form LAMA-DIST. GRU models trained on a hybrid of gold and distant data reach 70% F-score, 11 points above a strong baseline. The approach also helps on the external SemEval 2018 Arabic set. Arabic training data produced by machine translation from English performs badly, which suggests emotion labels may not survive translation. The work gives a cheap, validated way to bootstrap emotion corpora.

Key points

  • The seed list grows from 23 to 48 expressions, kept only when two native speakers fully agree. Coverage expands from Ekman's 6 emotions to Plutchik's 8 by adding anticipation and trust.
  • The datasets are LAMA (7,268 human-checked tweets, more than twice DINA's 2,984), LAMA-DINA (10,252 combined) and LAMA-DIST (182,690 tweets left from 405,588 after cleaning).
  • The best hybrid-supervised GRU reaches 70% F-score, a gain of 11% over the baseline (p < 0.05).
  • On SE-18 (4 emotions, 4,037 tweets), GRU F-score is:
    • 36% when trained on SE-18;
    • 41% on Lama-D2;
    • 46% on Lama-D2+SE-18, against a 26% majority baseline.
  • MT-DIST (756,663 English tweets machine-translated into Arabic) yields only 10% F-score. Merging it with gold data reaches just 20%, while gold LAMA-DINA alone gives 59%.

Interesting ideas

  • I1. Pairing a first-person pronoun with an emotion word gives a narrow, high-precision trigger for distant labels, because the writer is explicitly naming their own state.
  • I2. Splitting annotation into a binary relevance decision followed by an intensity grade for relevant items lowers cognitive load and improves agreement.
  • I3. A small gold set merged with a large distantly labelled pool beats either source alone, and checking on an independently built dataset shows the gain is not an artefact of one's own collection method.
  • I4. Before deduplicating, collapse runs of more than two repeated characters to two and normalise user mentions, then drop retweets and very short texts.
  • I5. Labels carried across languages by MT can actively hurt, possibly because of translation errors, unhandled dialects and culture-specific emotion expression.

For Mova

How these ideas could serve Mova, our small open language model — its files, stages and tests:

  • I1 (frames): Treat "I + emotion predicate" as a strong cue that the speaker is the experiencer, and use such matches to harvest silver examples for experiencer/emotion frames.
  • I2 (learn): LLM teacher tiers could ask the yes/no question first and a graded follow-up only on positives, giving cleaner and more consistent silver labels.
  • I3 (learn): Combine verified gold with multi-parser silver data, and always report scores on an external, differently sourced test set as a control.
  • I4 (tokens): Use similar normalisation in the tokenizer's preprocessing of social-media input, so near-identical noisy strings map to one form before tagging.
  • I5 (back): When translating through the meta-world, do not assume affective or pragmatic labels transfer unchanged, and verify them on target-language data.

Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.

 Institutions

🎓University of PennsylvaniaPhiladelphia · United States233 works · 65,583 citations🎓University of British ColumbiaVancouver · Canada104 works · 37,403 citations

 Fields, topics and concepts

📚 Shelves 5The field of science this work belongs to (OpenAlex fields).Computer Science · 18,614 worksArtificial Intelligence · 17,965 worksPsychology · 1,649 worksExperimental and Cognitive Psychology · 930 worksSocial Psychology · 330 works

🧭 Topics 2Research subjects and languages the work is about.Sentiment & emotion · 1,035 worksSemitic languages · 3,066 works

Cite

Cite as Hassan Alhuzali, Muhammad Abdul-Mageed and Lyle Ungar. 2018. Enabling Deep Learning of Emotion With First-Person Seed Expressions. Proceedings of the Second Workshop on Computational Modeling of People's Opinions, Personality, and Emotions in Social Media.

The paper itself, not this page. BibTeX from the library's record of the work. DOI 10.18653/v1/w18-1104

@inproceedings{alhuzali2018enabling,
  title = {{Enabling Deep Learning of Emotion With First-Person Seed Expressions}},
  author = {Hassan Alhuzali, Muhammad Abdul-Mageed and Lyle Ungar},
  booktitle = {Proceedings of the Second Workshop on Computational Modeling of People's Opinions, Personality, and Emotions in Social Media},
  year = {2018},
  doi = {10.18653/v1/w18-1104},
  url = {https://doi.org/10.18653/v1/w18-1104}
}

References

51 references, 19 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

  1. Julian Brooke, Milan Tofiloski, and Maite Taboada. 2009. Cross-linguistic sentiment analysis: From english to spanish. In RANLP. pages 50–54. find
  2. Muhammad Abdul-Mageed. 2017. Modeling arabic subjectivity and sentiment in lexical space. Information Processing & Management . ↗
  3. Kyunghyun Cho, Bart Van Merriënboer, Caglar Gulcehre, Dzmitry Bahdanau, Fethi Bougares, Holger Schwenk, and Yoshua Bengio. 2014. Learning phrase representations using rnn encoder-decoder for statistical machine translation. arXiv preprint arXiv:1406.1078 . ↗
  4. Muhammad Abdul-Mageed. 2018. Learning subjective language: Feature engineerd vs. deep models. In The 3rd Workshop on Open-Source Arabic Corpora and Processing Tools (OSACT 2018), LREC. find
  5. Muhammad Abdul-Mageed, Hassan Alhuzali, and Mohamed Elaraby. 2018. You tweet what you speak: A city-level dataset of arabic dialects. In LREC.
  6. Junyoung Chung, Caglar Gülçehre, Kyunghyun Cho, and Yoshua Bengio. 2015. Gated feedback recurrent neural networks. In ICML. pages 2067–2075. ↗
  7. Muhammad Abdul-Mageed, Hassan AlHuzli, and Mona Diab DuaaAbu Elhija. 2016. Dina: A multi-dialect dataset for arabic emotion analysis. In The 2nd Workshop on Arabic Corpora and Processing Tools 2016 Theme: Social Media. page 29. find
  8. J. Cohen. 1960. A coefficient of agreement for nominal scales. Educational and Psychological Measurement 20:37–46. ↗
  9. Andrew Cotter, Ohad Shamir, Nati Srebro, and Karthik Sridharan. 2011. Better mini-batch algorithms via accelerated gradient methods. In Advances in neural information processing systems. pages 1647–1655. ↗
  10. Muhammad Abdul-Mageed, Mona Diab, and Mohamed Korayem. 2011. Subjectivity and sentiment analysis of modern standard arabic. In Proceedings of the 49th Annual Meeting of the Association for Computational Linguistics: Human Language Technologies. Association for Computational Linguistics, Portland, Oregon, USA, pages 587–591. http://www.aclweb.org/ anthology/P11-2103. ↗
  11. Munmun De Choudhury, Scott Counts, and Michael Gamon. 2012. Not all moods are created equal! exploring human emotional states in social media. find
  12. P. Ekman. 1972. Universal and cultural differences in facial expression of emotion. Nebraska Symposium on Motivation pages 207–283. find
  13. Muhammad Abdul-Mageed, Mona Diab, and Sandra Kübler. 2014. Samar: Subjectivity and sentiment analysis for arabic social media. Computer Speech & Language 28(1):20–37. ↗
  14. Paul Ekman. 1992. An argument for basic emotions. Cognition & emotion 6(3-4):169–200. ↗
  15. Muhammad Abdul-Mageed and Lyle Ungar. 2017. Emonet: Fine-grained emotion detection with gated recurrent neural networks. In Proceedings of the 55th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers). volume 1, pages 718–728.
  16. Geoffrey E Hinton, Nitish Srivastava, Alex Krizhevsky, Ilya Sutskever, and Ruslan R Salakhutdinov. 2012. Improving neural networks by preventing co-adaptation of feature detectors. arXiv preprint arXiv:1207.0580 . ↗
  17. Ali Alshehri, AlMoetazbillah Nagoudi, Alhuzali Hassan, and Muhammad Abdul-Mageed. 2018. Think before your click: Data and models for adult content in arabic twitter. In The 2nd Text Analytics for Cybersecurity and Online Safety (TA-COS-2018), LREC. find
  18. Kanayama Hiroshi, Nasukawa Tetsuya, and Watanabe Hideo. 2004. Deeper sentiment analysis using machine translation technology. In Proceedings of the 20th international conference on Computational Linguistics. Association for Computational Linguistics, page 494. find
  19. Saima Aman and Stan Szpakowicz. 2007. Identifying expressions of emotion in text. In Text, Speech and Dialogue. Springer, pages 196–205. find
  20. Sepp Hochreiter and Jürgen Schmidhuber. 1997. Long short-term memory. Neural computation 9(8):1735–1780. ↗
  21. A. Balahur and R. Steinberger. 2009. Rethinking Sentiment Analysis in the News: from Theory to Practice and back. Proceeding of WOMSA . find
  22. Diederik Kingma and Jimmy Ba. 2014. Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 . ↗
  23. Mahmoud Nabil, Mohamed Aly, and Amir F Atiya. 2015. Astd: Arabic sentiment tweets dataset. In Proceedings of the 2015 Conference on Empirical Methods in Natural Language Processing. pages 2515–2519. ↗
  24. Svetlana Kiritchenko and Saif M Mohammad. 2016. Capturing reliable fine-grained sentiment associations by crowdsourcing and best-worst scaling. In HLT-NAACL. pages 811–817.
  25. J.R. Landis and G.G. Koch. 1977. The measurement of observer agreement for categorical data. Biometrics 33(1):159. ↗
  26. Thin Nguyen. 2010. Mood patterns and affective lexicon access in weblogs. In Proceedings of the ACL 2010 Student Research Workshop. Association for Computational Linguistics, pages 43–48. find
  27. Bing Liu. 2012. Sentiment analysis and opinion mining. Synthesis lectures on human language technologies 5(1):1–167. ↗
  28. B. Pang and L. Lee. 2004. A sentimental education: Sentimental analysis using subjectivity summarization based on minimum cuts. In Proceedings of the 42nd Annual Meeting of the Association for Computational Linguistics. pages 271–278. find
  29. Batja Mesquita, Michael Boiger, and Jozefien De Leersnyder. 2017. Doing emotions: The role of culture in everyday emotions. European Review of Social Psychology 28(1):95–133. ↗
  30. Razvan Pascanu, Tomas Mikolov, and Yoshua Bengio. 2013. On the difficulty of training recurrent neural networks. ICML (3) 28:1310–1318. ↗
  31. Rada Mihalcea, Carmen Banea, and Janyce Wiebe. 2007. Learning multilingual subjective language via cross-lingual projections. In Annual meeting-association for computational linguistics. volume 45, page 976.
  32. Robert Plutchik. 1985. On emotion: The chicken-and-egg problem revisited. Motivation and Emotion 9(2):197–200. ↗
  33. Mike Mintz, Steven Bills, Rion Snow, and Dan Jurafsky. 2009. Distant supervision for relation extraction without labeled data. In Proceedings of the Joint Conference of the 47th Annual Meeting of the ACL and the 4th International Joint Conference on Natural Language Processing of the AFNLP: Volume 2-Volume 2. Association for Computational Linguistics, pages 1003–1011.
  34. Robert Plutchik. 1994. The psychology and biology of emotion.. HarperCollins College Publishers. find
  35. Eshrag Refaee and Verena Rieser. 2014. An arabic twitter corpus for subjectivity and sentiment analysis. In LREC. pages 2268–2273.
  36. Sara Rosenthal, Noura Farra, and Preslav Nakov. 2017. Semeval-2017 task 4: Sentiment analysis in twitter. In Proceedings of the 11th International Workshop on Semantic Evaluation (SemEval-2017). pages 502–518.
  37. S. Bravo-Marquez F. Salameh M. Mohammad and S. Kiritchenko. 2018. Semeval-2018 task 1: Affect in tweets. In Proceedings of International Workshop on Semantic Evaluation (SemEval-2018). Association for Computational Linguistics.
  38. Mohammad Salameh, Saif Mohammad, and Svetlana Kiritchenko. 2015. Sentiment after translation: A case-study on arabic social media posts. In HLT-NAACL. pages 767–777.
  39. Saif M Mohammad. 2012. #emotional tweets. In Proceedings of the First Joint Conference on Lexical and Computational Semantics-Volume 1: Proceedings of the main conference and the shared task, and Volume 2: Proceedings of the Sixth International Workshop on Semantic Evaluation. Association for Computational Linguistics, pages 246–255.
  40. Laura Smith, Salvatore Giorgi, Rishi Solanki, Johannes C. Eichstaedt, Hansen Andrew Schwartz, Muhammad Abdul-Mageed, Anneke Buffone, and Lyle H. Ungar. 2016. Does ’well-being’ translate on twitter? In EMNLP. find
  41. Saif M. Mohammad and Felipe Bravo-Marquez. 2017. Emotion intensities in tweets. In Proceedings of the sixth joint conference on lexical and computational semantics (*Sem). Vancouver, Canada.
  42. Carlo Strapparava and Rada Mihalcea. 2007. Semeval-2007 task 14: Affective text. In Proceedings of the 4th International Workshop on Semantic Evaluations. Association for Computational Linguistics, pages 70–74.
  43. Saif M Mohammad and Svetlana Kiritchenko. 2015. Using hashtags to capture fine emotion categories from tweets. Computational Intelligence 31(2):301–326. ↗
  44. Svitlana Volkova and Yoram Bachrach. 2016. Inferring perceived demographics from user emotional tone and user-environment emotional contrast. In Proceedings of the 54th Annual Meeting of the Association for Computational Linguistics, ACL. find
  45. Saif M Mohammad, Mohammad Salameh, and Svetlana Kiritchenko. 2016. How translation alters sentiment. J. Artif. Intell. Res.(JAIR) 55:95–130. ↗
  46. Xiaojun Wan. 2008. Using bilingual knowledge and ensemble techniques for unsupervised chinese sentiment analysis. In EMNLP. Association for Computational Linguistics, pages 553–561. find
  47. Saif M. Mohammad and Peter D. Turney. 2013. Crowdsourcing a word-emotion association lexicon 29(3):436–465. ↗
  48. Wenbo Wang, Lu Chen, Krishnaprasad Thirunarayan, and Amit P Sheth. 2012. Harnessing twitter” big data” for automatic emotion identification. In Privacy, Security, Risk and Trust (PASSAT), 2012 International Conference on and 2012 International Confernece on Social Computing (Social-Com). IEEE, pages 587–592. ↗
  49. J. M. Wiebe, T. Wilson, R. Bruce, M. Bell, and M. Martin. 2004. Learning subjective language. Computational Linguistics 30:227–308. ↗
  50. Jasy Liew Suet Yan and Howard R Turtle. 2016. Exploring fine-grained emotion detection in tweets. In Proceedings of NAACL-HLT. pages 73–80. find
  51. Yi Yang and Jacob Eisenstein. 2017. Overcoming language variation in sentiment analysis with social attention. TACL 5:295–307. ↗

Updated

↑ back to top

This page in Markdown → https://md.research.elamur.ai/free/en/papers/alhuzali-2018-enabling-deep-learning-emotion-first/