Elamur Research⌕
☰
home / papers / 2012 / #Emotional Tweets

#Emotional Tweets

Paper · 2012 · Saif Mohammad

Summary

Supervised emotion detection needs labelled training text, but in 2012 there was no microblog corpus annotated for emotions, and classifiers trained on other domains transfer poorly to tweets. Mohammad addresses this by treating emotion-word hashtags as labels that the authors of the tweets supplied themselves. Tweets carrying hashtags for the six Ekman emotions were gathered through the Twitter Search API, using the Archivist service. After filtering, about 21,000 tweets remained, forming the Twitter Emotion Corpus (TEC). Experiments show that these self-assigned labels are consistent and agree with trained annotators. Adding TEC to the training data also improves emotion classification of newspaper headlines from SemEval-2007 Affective Text, a very different domain. From TEC, the author derives a large word–emotion lexicon using a PMI-based association score. As classifier features, this lexicon beats the hand-built WordNet Affect, though it stays below the crowdsourced NRC emotion lexicon. The work shows a cheap and scalable route to emotion-labelled data and lexicons for social media.

Key points

  • Collection: about 50,000 tweets were posted between 15 November and 6 December 2011 under six emotion hashtags (anger, disgust, fear, joy/happy, sadness, surprise). After filtering, roughly 21,000 remained as TEC. Joy was the most frequent label.

  • Why existing corpora failed: collections of 380,000 tweets (Eisenstein et al.) and 1.6 million tweets (Go et al.) together contained fewer than 50 emotion-hashtag tweets. This forced the author to use targeted hashtag search instead.

  • Filtering rules:

    • drop tweets with fewer than three valid English words, using the Roget Thesaurus as the word list;
    • drop retweets (the "RT" prefix);
    • keep only tweets where the target hashtag appears at the end.

  • Lexicon construction: association strength is defined as SoA = PMI(n,e) − PMI(n,¬e), computed for unigrams and bigrams. N-grams seen fewer than five times are ignored. Pairs with a positive SoA enter the TEC lexicon.

  • Lexicon sizes (word types):

    LexiconWord types
    1000-headlines (auto-derived)152
    TEC11,418
    WordNet Affect1,536
    NRC emotion lexicon14,000

Interesting ideas

  • I1. User markup as free labels. Hashtags written by users can serve as distant labels if a few simple heuristics clean them, such as keeping only end-of-message tags and dropping reposts. The same trick could bootstrap labelled social-media data in languages and domains that lack annotated corpora.
  • I2. Checking noisy labels against a gold domain. The author tests whether the noisy labels help a classifier on a separate, expert-annotated domain. If they do, that indirectly shows the labels match human judgement. This is a practical way to validate any weakly supervised dataset.
  • I3. A contrastive association score. Subtracting PMI with "not e" from PMI with e rewards n-grams that are specific to one emotion, not just frequent. Combined with a minimum-frequency cutoff, this is a general recipe for building class-association lexicons from any labelled corpus.
  • I4. Lexicon counts as features. Instead of a large n-gram feature space, the classifier can use one integer per emotion: the number of tokens from the lexicon. Normalising by sentence length did not help. This compact form makes lexicons easy to plug into classifiers for other tasks.
  • I5. A cheap language filter. Requiring at least a few dictionary words in each message removes foreign-language and junk posts without needing a dedicated language identifier.

Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.

📚 Shelves 2The field of science this work belongs to (OpenAlex fields).Computer Science · 9,086 worksArtificial Intelligence · 8,889 works

🧭 Topics 1Research subjects and languages the work is about.Sentiment & emotion · 608 works

Cite

Cite as Saif Mohammad. 2012. #Emotional Tweets. *SEM 2012: The First Joint Conference on Lexical and Computational Semantics – Volume 1: Proceedings of the main conference and the shared task, and Volume 2: Proceedings of the Sixth International Workshop on Semantic Evaluation (SemEval 2012).

The paper itself, not this page. BibTeX from the library's record of the work.

@inproceedings{mohammad2012emotional,
  title = {{\#Emotional Tweets}},
  author = {Saif Mohammad},
  booktitle = {*SEM 2012: The First Joint Conference on Lexical and Computational Semantics – Volume 1: Proceedings of the main conference and the shared task, and Volume 2: Proceedings of the Sixth International Workshop on Semantic Evaluation (SemEval 2012)},
  year = {2012},
  url = {https://aclanthology.org/S12-1033/}
}

References

38 references, 7 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

  1. Cecilia O. Alm and Richard Sproat, 2005. Emotional sequencing and development in fairy tales, pages 668– 674. Springer. find
  2. Cecilia Ovesdotter Alm, Dan Roth, and Richard Sproat. 2005. Emotions from text: Machine learning for text-based emotion prediction. In Proceedings of the Joint Conference on HLT–EMNLP, Vancouver, Canada.
  3. Saima Aman and Stan Szpakowicz. 2007. Identifying expressions of emotion in text. In Vclav Matoušek and Pavel Mautner, editors, Text, Speech and Dialogue, volume 4629 of Lecture Notes in Computer Science, pages 196–205. Springer Berlin / Heidelberg. find
  4. Johan Bollen, Alberto Pepe, and Huina Mao. 2009. Modeling public mood and emotion: Twitter sentiment and socio-economic phenomena. CoRR. find
  5. Anthony C. Boucouvalas. 2002. Real time text-to-emotion engine for expressive internet communication. Emerging Communication: Studies on New Technologies and Practices in Communication, 5:305–318. find
  6. J. R. G. Bougie, R. Pieters, and M. Zeelenberg. 2003. Angry customers don’t come back, they get back: The experience and behavioral implications of anger and dissatisfaction in services. Open access publications from tilburg university, Tilburg University. find
  7. Danah Boyd, Scott Golder, and Gilad Lotan. 2010. Tweet, Tweet, Retweet: Conversational Aspects of Retweeting on Twitter. volume 0, pages 1–10, Los Alamitos, CA, USA. IEEE Computer Society. find
  8. Soumaya Chaffar and Diana Inkpen. 2011. Using a heterogeneous dataset for emotion analysis in text. In Canadian Conference on AI, pages 62–67. find
  9. Hal Daumé. 2007. Frustratingly easy domain adaptation. In Conference of the Association for Computational Linguistics (ACL), Prague, Czech Republic.
  10. Jacob Eisenstein, Brendan O’Connor, Noah A. Smith, and Eric P. Xing. 2010. A latent variable model for geographic lexical variation. In Proceedings of the 2010 Conference on Empirical Methods in Natural Language Processing, pages 1277–1287, Stroudsburg, PA. Association for Computational Linguistics.
  11. Paul Ekman. 1992. An argument for basic emotions. Cognition and Emotion, 6(3):169–200.
  12. Virginia Francisco and Pablo Gervás. 2006. Automated mark up of affective information in english texts. In Petr Sojka, Ivan Kopecek, and Karel Pala, editors, Text, Speech and Dialogue, volume 4188 of Lecture Notes in Computer Science, pages 375–382. Springer Berlin / Heidelberg. find
  13. Michel Genereux and Roger P. Evans. 2006. Distinguishing affective states in weblogs. In AAAI-2006 Spring Symposium on Computational Approaches to Analysing Weblogs, pages 27–29, Stanford, California. find
  14. Alec Go, Richa Bhayani, and Lei Huang. 2009. Twitter sentiment classification using distant supervision. In Final Projects from CS224N for Spring 2008–2009 at The Stanford Natural Language Processing Group. find
  15. Roberto González-Ibáñez, Smaranda Muresan, and Nina Wacholder. 2011. Identifying sarcasm in twitter: a closer look. In Proceedings of the 49th Annual Meeting of the Association for Computational Linguistics: Human Language Technologies: short papers - Volume 2, pages 581–586, Portland, Oregon.
  16. Mark Hall, Eibe Frank, Geoffrey Holmes, Bernhard Pfahringer, Peter Reutemann, and Ian H. Witten. 2009. The weka data mining software: an update. SIGKDD, 11:10–18. find
  17. Lars E. Holzman and William M. Pottenger. 2003. Classification of emotions in internet chat: An application of machine learning using speech phonemes. Technical report, Leigh University. find
  18. David John, Anthony C. Boucouvalas, and Zhe Xu. 2006. Representing emotional momentum within expressive internet communication. In Proceedings of the 24th IASTED international conference on Internet and multimedia systems and applications, pages 183– 188, Anaheim, CA. ACTA Press. find
  19. Elsa Kim, Sam Gilbert, Michael J. Edwards, and Erhardt Graeff. 2009. Detecting sadness in 140 characters: Sentiment analysis of mourning michael jackson on twitter. find
  20. Hugo Liu, Henry Lieberman, and Ted Selker. 2003. A model of textual affect sensing using real-world knowledge. In Proceedings of the 8th international conference on Intelligent user interfaces, IUI ’03, pages 125–132, New York, NY. ACM. find
  21. Chunling Ma, Helmut Prendinger, and Mitsuru Ishizuka. 2005. Emotion estimation and reasoning based on affective textual interaction. In J. Tao and R. W. Picard, editors, First International Conference on Affective Computing and Intelligent Interaction (ACII-2005), pages 622–628, Beijing, China. find
  22. Rada Mihalcea and Hugo Liu. 2006. A corpus-based approach to finding happiness. In AAAI-2006 Spring Symposium on Computational Approaches to Analysing Weblogs, pages 139–144. AAAI Press. find
  23. Saif M. Mohammad and Peter D. Turney. 2012. Crowdsourcing a word–emotion association lexicon. To Appear in Computational Intelligence. find
  24. Saif M. Mohammad and Tony Yang. 2011. Tracking Sentiment in Mail: How Genders Differ on Emotional Axes. In Proceedings of the 2nd Workshop on Computational Approaches to Subjectivity and Sentiment Analysis (WASSA 2.011), pages 70–79, Portland, Oregon. Association for Computational Linguistics. find
  25. Saif M. Mohammad. 2012a. From once upon a time to happily ever after: Tracking emotions in mail and books. To Appear in Decision Support Systems. find
  26. Saif M. Mohammad. 2012b. Portable features for emotion classification. In Proceedings of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies (NAACL-HLT 2012), Montreal, Canada. Association for Computational Linguistics. find
  27. Alena Neviarouskaya, Helmut Prendinger, and Mitsuru Ishizuka. 2009. Compositionality principle in recognition of fine-grained emotions from text. In Proceedings of the Proceedings of the Third International Conference on Weblogs and Social Media (ICWSM-09), pages 278–281, San Jose, California. find
  28. Lisa Pearl and Mark Steyvers. 2010. Identifying emotions, intentions, and attitudes in text using a game with a purpose. In Proceedings of the NAACL-HLT 2010 Workshop on Computational Approaches to Analysis and Generation of Emotion in Text, Los Angeles, California. find
  29. John Platt. 1999. Using analytic qp and sparseness to speed training of support vector machines. In In Neural Info. Processing Systems 11, pages 557–563. MIT Press. find
  30. Robert Plutchik. 1985. On emotion: The chicken-and-egg problem revisited. Motivation and Emotion, 9(2):197–200. find
  31. Niklas Ravaja, Timo Saari, Marko Turpeinen, Jari Laarni, Mikko Salminen, and Matias Kivikangas. 2006. Spatial presence and emotions during video game playing: Does it matter with whom you play? Presence: Teleoperators and Virtual Environments, 15(4):381–392. find
  32. Tapas Ray. 2011. The ’story’ of digital excess in revolutions of the arab spring. Journal of Media Practice, 12(2):189–196. find
  33. Julia Skinner. 2011. Social media and revolution: The arab spring and the occupy movement as seen through three information studies paradigms. Sprouts: Working Papers on Information Systems, 11(169). find
  34. Carlo Strapparava and Rada Mihalcea. 2007. Semeval-2007 task 14: Affective text. In Proceedings of SemEval-2007, pages 70–74, Prague, Czech Republic.
  35. Carlo Strapparava and Alessandro Valitutti. 2004. Wordnet-Affect: An affective extension of WordNet. In Proceedings of the 4th International Conference on Language Resources and Evaluation (LREC-2004), pages 1083–1086, Lisbon, Portugal.
  36. Andranik Tumasjan, Timm O Sprenger, Philipp G Sandner, and Isabell M Welpe. 2010. Predicting elections with twitter : What 140 characters reveal about political sentiment. Word Journal Of The International Linguistic Association, pages 178–185. find
  37. Juan D. Velásquez. 1997. Modeling emotions and other motivations in synthetic agents. In Proceedings of the fourteenth national conference on artificial intelligence and ninth conference on Innovative applications of artificial intelligence, AAAI’97/IAAI’97, pages 10–15. AAAI Press. find
  38. Xu Zhe and A Boucouvalas, 2002. Text-to-Emotion Engine for Real Time Internet CommunicationText-to-Emotion Engine for Real Time Internet Communication, pages 164–168. find

Updated

↑ back to top

This page in Markdown → https://md.research.elamur.ai/free/en/papers/mohammad-2012-emotional-tweets/