Elamur Research⌕
☰
home / papers / 2005 / Emotions from Text: Machine Learning for Text-based Emotion Prediction

Emotions from Text: Machine Learning for Text-based Emotion Prediction

Summary

The paper addresses text-based emotion prediction (TEP): deciding which emotion, if any, a sentence conveys. The target application is expressive text-to-speech for children's stories, where prosody should match the emotional content of the text. The authors are building a corpus of about 185 children's stories by Grimm, H.C. Andersen and B. Potter. Pairs of annotators label each sentence independently, and the first author breaks ties.

The experiments use a preliminary set of 22 Grimms' tales (1580 sentences). The learner is SNoW, a Winnow-based linear classifier, evaluated with 10-fold cross-validation on two flat tasks:

  • neutral vs. emotional;
  • neutral vs. positive vs. negative.

Features are boolean and cover several groups:

  • sentence-level cues such as punctuation, direct speech and length;
  • part-of-speech proportions;
  • how far the sentence is into the story, and the story type;
  • counts of positive and negative words, plus WordNet emotion words;
  • conjunctions of these features.

The feature-rich model beats a naive majority baseline and a bag-of-words approach, although results depend partly on parameter tuning. This work is an early step toward treating emotion as a learnable sentence-level property rather than relying only on lexicons and hand-written rules.

Key points

  • Annotators chose from seven emotion classes plus neutral, an extended Ekman set in which surprise is split into positive and negative.
  • Annotation is taken from the feeler's perspective.
  • Inter-annotator agreement was low: κ ranged from .24 to .51, with 45–64% raw overlap depending on the annotator pair.
  • Neutral sentences made up 59.94% of the data. After merging classes, the valence task had 9.87% positive and 30.19% negative sentences.
  • SNoW received only active features, typically around 30 per sentence. Perceptron was also tested and performed worse.
  • In cumulative ablation, accuracy dropped steadily from roughly 63% with all features as groups were removed. The removed groups were word lists, syntactic, story-related, orthographic, conjunction and content-word features.

Interesting ideas

  • I1. Overlapping ranges for continuous values. Sentence length and similar values are encoded as several overlapping boolean bins. This gives a sparse linear learner smoother generalization and suits any binary-feature setup.
  • I2. Discourse position as a signal. The model uses how far a sentence lies into the narrative. Emotional intensity tends to cluster near a story's climax and ending, and similar positional cues could help in other long-form genres.
  • I3. Collapsing fine labels into coarse tasks. With little data, the authors merge many emotions into emotional vs. neutral, or into positive vs. negative. This lets them study a problem before scaling to fine labels, with a planned coarse-to-fine sequential cascade later.
  • I4. Independent annotation to measure real difficulty. Annotators are not trained jointly, so the agreement scores reflect genuine subjectivity rather than a shared bias. This is a useful choice whenever you need to know the true ceiling of a subjective task.
  • I5. Qualitative contrast with a baseline. The authors show example sentences where their model succeeds and bag-of-words fails, and name the features responsible. This makes claims about feature contributions concrete.

Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.

Fields: Natural language processing

Cited by

160 papers of our library cite it (of the 140884 whose bibliographies we read); 9 of them have a page here, 151 not yet.

Cite

The paper itself, not this page. BibTeX from the library's record of the work.

@inproceedings{alm2005emotions,
  title = {{Emotions from Text: Machine Learning for Text-based Emotion Prediction}},
  author = {Alm, Cecilia Ovesdotter and Roth, Dan and Sproat, Richard},
  booktitle = {Proceedings of Human Language Technology Conference and Conference on Empirical Methods in Natural Language Processing},
  year = {2005},
  url = {https://aclanthology.org/H05-1073/}
}

References

26 references, 3 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

  1. Antti Aarne. 1964. The Types of the Folk-Tale: a Classification and Bibliography. Helsinki: Suomalainen Tiedeakatemia. find
  2. Cecilia O. Alm, and Richard Sproat. 2005. Perceptions of emotions in expressive storytelling. find
  3. Xue Bai, Rema Padman, and Edoardo Airoldi. 2004. Sentiment extraction from unstructured text using tabu search-enhanced Markov blankets. In MSW2004, Seattle. find
  4. Philip Beineke, Trevor Hastie, and Shivakumar Vaithyanathan. 2004. The sentimental factor: improving review classification via human-provided information. In Proceedings of ACL, 263–270. find
  5. Janet Cahn. 1990. The generation of affect in synthesized Speech. Journal of the American Voice I/O Society, 8:1–19. find
  6. Andrew Carlson, Chad Cumby, Nicholas Rizzolo, Jeff Rosen, and Dan Roth. 1999. The SNoW Learning Architecture. Technical Report UIUCDCS-R-99-2101, UIUC Comp. Sci. find
  7. Stacey Di Cicco et al. General Inquirer Pos./Neg. lists http://www.webuse.umd.edu:9090/ ↗
  8. Paul Ekman. 1993. Facial expression and emotion. American Psychologist, 48(4), 384–392. find
  9. Christiane Fellbaum, Ed. 1998. WordNet: An Electronic Lexical Database. MIT Press, Cambridge, Mass. find
  10. Vasileios Hatzivassiloglou, and Kathleen McKeown. 1997. Predicting the semantic orientation of adjectives. In Proceedings of ACL, 174–181. find
  11. Philip Johnson-Laird, and Keith Oatley. 1989. The language of emotions: an analysis of a semantic field. Cognition and Emotion, 3:81–123. find
  12. Peter Koomen, Vasin Punyakanok, Dan Roth, and Wen-tau Yih. 2005. Generalized inference with multiple semantic role labeling systems. In Proceedings of the Annual Conference on Computational Language Learning (CoNLL), 181–184. find
  13. Diane Litman, and Kate Forbes-Riley. 2004. Predicting student emotions in computer-human tutoring dialogues. In Proceedings of ACL, 351–358. find
  14. Xin Li, and Dan Roth. 2002. Learning question classifiers: the role of semantic information. In Proc. International Conference on Computational Linguistics (COLING), 556–562. find
  15. Hugo Liu, Henry Lieberman, and Ted Selker. 2003. A model of textual affect sensing using real-world knowledge. In ACM Conference on Intelligent User Interfaces, 125–132. find
  16. Tony Mullen, and Nigel Collier. 2004. Sentiment analysis using support vector machines with diverse information sources. In Proceedings of EMNLP, 412–418. find
  17. Bo Pang, and Lillian Lee. 2004. A sentimental education: sentiment analysis using subjectivity summarization based on minimum cuts. In Proceedings of ACL, 271–278. find our card
  18. Rosalind Picard. 1997. Affective computing. MIT Press, Cambridge, Mass. find
  19. Dan Roth. 1998. Learning to resolve natural language ambiguities: a unified approach. In AAAI, 806–813. find
  20. Klaus Scherer. 2003. Vocal communication of emotion: a review of research paradigms. Speech Commununication, 40(1-2):227–256. find
  21. Greg Siegle. The Balanced Affective Word http://www.sci.sdsu.edu/CAL/wordlist/words.prn List Oliver Steele et al. http://osteele.com/projects/pywordnet/Py-WordNet ↗
  22. Futoshi Sugimoto et al. 2004. A method to classify emotional expressions of text and synthesize speech. In IEEE, 611– 614. find
  23. Peter Turney. 2002. Thumbs up or thumbs down? Semantic orientation applied to unsupervised classification of reviews. In Proceedings of ACL, 417–424. find our card
  24. Jan van Santen et al. 2003. Applications of computer generated expressive speech for communication disorders. In EUROSPEECH 2003, 1657–1660. find
  25. Janyce Wiebe et al. 2004. Learning subjective language. Journal of Computational Linguistics, 30(3):277–308. find our card
  26. Theresa Wilson, Janyce Wiebe, and Rebecca Hwa. 2004. Just how mad are you? Finding strong and weak opinion clauses. In Proceedings of the Nineteenth National Conference on Artificial Intelligence (AAAI), 761–769. find

↑ back to top

This page in Markdown → https://md.research.elamur.ai/free/en/papers/alm-2005-emotions-text-machine-learning-text/