Elamur Research⌕
☰
home / papers / 2016 / Inducing Domain-Specific Sentiment Lexicons from Unlabeled Corpora

Inducing Domain-Specific Sentiment Lexicons from Unlabeled Corpora

Summary

Word polarity depends on context. Soft reads differently in a sports forum than in a toy-pony community, and terrific used to be negative. Generic sentiment lexicons can therefore mislead computational social science (CSS) studies, and hand-built domain lexicons are costly to make. The authors propose SentProp, which needs only a modest unlabeled domain corpus (around 10^7 tokens) and a small list of seed words.

SentProp works in three steps:

  • It trains word vectors with SVD on a PPMI matrix.
  • It links each word to its nearest neighbours to form a weighted graph.
  • It spreads polarity from positive and negative seeds by random walks, and uses bootstrap resampling to attach a confidence to each score.

The method reaches state-of-the-art results, competitive even with approaches that rely on curated dictionaries such as WordNet. The authors then build two lexicon collections: one for 250 Reddit communities and one for 150 years of English. These show large differences in sentiment between communities and over time. Everything is released as the SocialSent package, which matters for anyone analysing niche, historical or non-standard text where no curated resource exists.

Key points

  • Embeddings: 300-dimensional vectors from truncated SVD of a smoothed PPMI matrix, built with a window of 4 words per side and context-distribution smoothing c = 0.75. The singular-value weights are dropped. In preliminary tests on the domain data, this beat word2vec and GloVe.
  • Graph and propagation: each word connects to its k nearest neighbours by cosine similarity, and edges are weighted with an arccos-based formula. Propagation is a random walk with restart (Zhou et al., 2004), where the parameter β trades off agreement among neighbours against fidelity to the seeds.
  • Seed sets: about ten positive and ten negative words per domain (Standard English, Finance, Twitter). They were chosen to be context-insensitive, without looking at the test lexicons.
  • Historical lexicons from COHA: comparing 1850–1880 with 1970–2000, more than 5% of polar words fully flipped polarity, and more than 25% changed label if moves to or from neutral are counted. Examples include lean (amelioration), pathetic (pejoration), sorry and worldly.
  • Released resources: the SocialSent package contains lexicons for the 250 largest subreddits, the historical lexicons, and code for SentProp. It also includes implementations, not previously available, of the baseline methods from Velikovich and colleagues (2010) and from Rothe and colleagues (2016).

Interesting ideas

  • I1. Bootstrap confidence scores: rerunning propagation on resampled seed sets gives a standard deviation for every word's score. Downstream analyses can then separate reliable scores from noisy ones. The same trick fits any seed-driven lexicon or label-spreading method.
  • I2. Small, stable, hand-picked seeds: for the historical work, the authors used a few words whose sentiment stayed constant over 150 years, checked against the OED. This avoids assuming that an entire modern lexicon is still valid in older text. It is a cheap, transparent way to anchor induction when moving into a new domain, era or low-resource language.
  • I3. Count-based embeddings for small corpora: with domain corpora of moderate size, SVD over PPMI outperformed neural embeddings in the authors' preliminary experiments. For niche corpora it is therefore worth testing classic count-based vectors before defaulting to word2vec or GloVe.
  • I4. Calibrated three-way labels: polar-versus-neutral thresholds were set with class-mass normalization, and the averages of early and late periods were compared. This turns continuous scores into discrete labels in a consistent way, which helps when counting polarity switches across time slices or communities.
  • I5. Explaining shifts with neighbours: nearest neighbours in the period-specific vectors were used to show why a word's polarity moved (lean drifting toward muscular and away from weak). This makes automatically detected changes easier for humans to verify.

Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.

Fields: Natural language processing

Cited by

76 papers of our library cite it (of the 140884 whose bibliographies we read); 1 of them have a page here, 75 not yet.

Cite

The paper itself, not this page. BibTeX from the library's record of the work. DOI 10.18653/v1/d16-1057

@inproceedings{hamilton2016inducing,
  title = {{Inducing Domain-Specific Sentiment Lexicons from Unlabeled Corpora}},
  author = {Hamilton, William L. and Clark, Kevin and Leskovec, Jure and Jurafsky, Dan},
  booktitle = {Proceedings of the 2016 Conference on Empirical Methods in Natural Language Processing},
  year = {2016},
  doi = {10.18653/v1/d16-1057},
  url = {https://doi.org/10.18653/v1/d16-1057}
}

References

57 references, 8 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

  1. Muhammad Zubair Asghar, Aurangzeb Khan, Shakeel Ahmad, Imran Ali Khan, and Fazal Masud Kundi. 2015. A Unified Framework for Creating Domain Dependent Polarity Lexicons from User Generated Reviews. PLOS ONE, 10(10):e0140204, October. find
  2. Sasha Blair-Goldensohn, Kerry Hannan, Ryan McDonald, Tyler Neylon, George A. Reis, and Jeff Reynar. 2008. Building a sentiment summarizer for local service reviews. In WWW Workshop on NLP in the Information Explosion Era. find
  3. Felipe Bravo-Marquez, Eibe Frank, and Bernhard Pfahringer. 2015. From Unlabelled Tweets to Twitter-specific Opinion Words. In SIGIR. find
  4. John A. Bullinaria and Joseph P. Levy. 2012. Extracting semantic representations from word co-occurrence statistics: stop-lists, stemming, and SVD. Behavior Research Methods, 44(3):890–907, September. find
  5. Yejin Choi and Claire Cardie. 2009. Adapting a polarity lexicon using integer linear programming for domain-specific sentiment classification. In EMNLP. find
  6. Paul Cook and Suzanne Stevenson. 2010. Automatically Identifying Changes in the Semantic Orientation of Words. In LREC. find
  7. Mark Davies. 2010. The Corpus of Historical American English: 400 million words, 1810-2009. http://corpus.byu.edu/coha/. ↗
  8. Lingjia Deng, Janyce Wiebe, and Yoonjung Choi. 2014. Joint inference and disambiguation of implicit sentiments via implicature constraints. In COLING. find our card
  9. Andrea Esuli and Fabrizio Sebastiani. 2006. Sentiwordnet: A publicly available lexical resource for opinion mining. In LREC. find our card
  10. Ethan Fast, Binbin Chen, and Michael S. Bernstein. 2016. Empath: Understanding Topic Signals in Large-Scale Text. In CHI. find
  11. Christiane Fellbaum. 1998. find
  12. WordNet. Wiley Online Library. William L. Hamilton, Jure Leskovec, and Dan Jurafsky. 2016. Diachronic Word Embeddings Reveal Statistical Laws of Semantic Change. In ACL. find our card
  13. Vasileios Hatzivassiloglou and Kathleen R. McKeown. 1997. Predicting the semantic orientation of adjectives. In EACL. find
  14. Iris Hendrickx, Michel Gnreux, and Rita Marquilhas. 2011. Automatic pragmatic text segmentation of historical letters. In Language Technology for Cultural Heritage, pages 135–153. find
  15. Springer. Dirk Hovy. 2015. Demographic factors improve classification performance. In ACL. find our card
  16. Minqing Hu and Bing Liu. 2004. Mining and summarizing customer reviews. In KDD. find
  17. Sheng Huang, Zhendong Niu, and Chongyang Shi. 2014. Automatic construction of domain-specific sentiment lexicon based on constrained label propagation. Knowledge-Based Systems, 56:191–200. find
  18. Sean P. Igo and Ellen Riloff. 2009. Corpus-based semantic lexicon induction with web-based corroboration. In ACL Workshop on Unsupervised and Minimally Supervised Learning of Lexical Semantics. find
  19. Valentin Jijkoun, Maarten de Rijke, and Wouter Weerkamp. 2010. Generating focused topic-specific sentiment lexicons. In ACL. find
  20. Jaap Kamps, M. J. Marx, Robert J. Mokken, and M. de Rijke. 2004. Using wordnet to measure semantic orientations of adjectives. In LREC. find
  21. William Labov. 2006. The social stratification of English in New York City. find
  22. Cambridge University Press. Thomas K. Landauer and Susan T. Dumais. 1997. A solution to Plato’s problem: The latent semantic analysis theory of acquisition, induction, and representation of knowledge. Psychol. Rev., 104(2):211. find
  23. Heeyoung Lee, Mihai Surdeanu, Bill MacCartney, and Dan Jurafsky. 2014. On the Importance of Text Analysis for Stock Price Prediction. In LREC. find
  24. Omer Levy, Yoav Goldberg, and Ido Dagan. 2015. Improving distributional similarity with lessons learned from word embeddings. Trans. Assoc. Comput. Ling., 3. find
  25. Tim Loughran and Bill McDonald. 2011. When is a liability not a liability? Textual analysis, dictionaries, and 10-Ks. The Journal of Finance, 66(1):35–65. find
  26. Tomas Mikolov, Ilya Sutskever, Kai Chen, Greg S. Corrado, and Jeff Dean. 2013. Distributed representations of words and phrases and their compositionality. In NIPS. find
  27. Saif M. Mohammad and Peter D. Turney. 2010. Emotions evoked by common words and phrases: Using Mechanical Turk to create an emotion lexicon. In NAACL. find our card
  28. Aditi Muralidharan and Marti A Hearst. 2012. Supporting exploratory text analysis in literature study. find
  29. Literary and Linguistic Computing. Eitan Adam Pechenick, Christopher M. Danforth, and Peter Sheridan Dodds. 2015. Characterizing the Google Books corpus: Strong limits to inferences of socio-cultural and linguistic evolution. PLoS ONE, 10(10). find
  30. Jeffrey Pennington, Richard Socher, and Christopher D. Manning. 2014. Glove: Global vectors for word representation. In EMNLP. find
  31. Michael Piotrowski. 2012. Natural language processing for historical texts. Synthesis Lectures on Human Language Technologies, 5(2):1–157. find
  32. Guang Qiu, Bing Liu, Jiajun Bu, and Chun Chen. 2009. Expanding Domain Sentiment Lexicon through Double Propagation. In IJCAI. find
  33. Delip Rao and Deepak Ravichandran. 2009. Semi-supervised polarity lexicon induction. In EACL. find
  34. Ellen Riloff and Jessica Shepherd. 1997. A corpus-based approach for building semantic lexicons. arXiv preprint cmp-lg/9706013. find
  35. Mats Rooth, Stefan Riezler, Detlef Prescher, Glenn Carroll, and Franz Beil. 1999. Inducing a semantically annotated lexicon via EM-based clustering. In ACL. find
  36. Sara Rosenthal, Preslav Nakov, Svetlana Kiritchenko, Saif M. Mohammad, Alan Ritter, and Veselin Stoyanov. 2015. SemEval-2015 task 10: Sentiment analysis in Twitter. SemEval-2015. find our card
  37. Sascha Rothe, Sebastian Ebert, and Hinrich Schutze. 2016. Ultradense Word Embeddings by Orthogonal Transformation. In NAACL-HLT. find
  38. Inaki San Vicente, Rodrigo Agerri, German Rigau, and Donostia-San Sebastin. 2014. Simple, Robust and (almost) Unsupervised Generation of Polarity Lexicons for Multiple Languages. In EACL. find
  39. Aliaksei Severyn and Alessandro Moschitti. 2015. On the automatic learning of sentiment lexicons. In NAACL-HLT. find
  40. John Andrew Simpson, Edmund SC Weiner, et al. 1989. The Oxford English Dictionary, volume 2. find
  41. Clarendon Press Oxford, Oxford, UK. Michael Speriosu, Nikita Sudan, Sid Upadhyay, and Jason Baldridge. 2011. Twitter polarity classification with label propagation over lexical links and the follower graph. In ACL Workshop on Unsupervised Learning in NLP. find
  42. Philip J Stone, Dexter C Dunphy, and Marshall S Smith. 1966. The General Inquirer: A Computer Approach to Content Analysis. find
  43. Maite Taboada, Julian Brooke, Milan Tofiloski, Kimberly Voll, and Manfred Stede. 2011. Lexicon-based methods for sentiment analysis. Comput. Ling., 37(2):267– 307. find our card
  44. Yen-Jen Tai and Hung-Yu Kao. 2013. Automatic domain-specific sentiment lexicon generation with label propagation. In Proceedings of International Conference on Information Integration and Web-based Applications & Services, page 53. find
  45. ACM. Hiroya Takamura, Takashi Inui, and Manabu Okumura. 2005. Extracting semantic orientations of words using spin model. In ACL. find
  46. Duyu Tang, Furu Wei, Bing Qin, Ming Zhou, and Ting Liu. 2014. Building Large-Scale Twitter-Specific Sentiment Lexicon: A Representation Learning Approach. In COLING. find
  47. Michael Thelen and Ellen Riloff. 2002. A bootstrapping method for learning semantic lexicons using extraction pattern contexts. In EMNLP. find our card
  48. Elizabeth Closs Traugott and Richard B Dasher. 2001. Regularity in Semantic Change. find
  49. Cambridge University Press, Cambridge, UK. Peter Trudgill. 1974. Linguistic change and diffusion: Description and explanation in sociolinguistic dialect geography. Language in Society, 3(2):215–246. find
  50. Peter D. Turney and Michael L. Littman. 2003. Measuring praise and criticism: Inference of semantic orientation from association. ACM Trans. Inf. Sys., 21(4):315–346. find
  51. Peter D. Turney and Patrick Pantel. 2010. From frequency to meaning: Vector space models of semantics. J. Artif. Intell. Res., 37(1):141–188. find
  52. Leonid Velikovich, Sasha Blair-Goldensohn, Kerry Hannan, and Ryan McDonald. 2010. The viability of web-derived polarity lexicons. In NAACL-HLT. find
  53. Amy Beth Warriner, Victor Kuperman, and Marc Brysbaert. 2013. Norms of valence, arousal, and dominance for 13,915 English lemmas. Behavior research methods, 45(4):1191–1207. find
  54. Dominic Widdows and Beate Dorow. 2002. A graph model for unsupervised lexical acquisition. In COL-ING. find
  55. Yi Yang and Jacob Eisenstein. 2015. Putting Things in Context: Community-specific Embedding Projections for Sentiment Analysis. arXiv preprint arXiv:1511.06052. — [arXiv:1511.06052](https://arxiv.org/abs/1511.06052) find
  56. Dengyong Zhou, Olivier Bousquet, Thomas Navin Lal, Jason Weston, and Bernhard Scholkopf. 2004. Learning with local and global consistency. In NIPS. find
  57. Xiaojin Zhu and Zoubin Ghahramani. 2002. Learning from labeled and unlabeled data with label propagation. Technical report. Xiaojin Zhu, Zoubin Ghahramani, John Lafferty, and others. 2003. Semi-supervised learning using Gaussian fields and harmonic functions. In ICML. find

↑ back to top

This page in Markdown → https://md.research.elamur.ai/free/en/papers/hamilton-2016-inducing-domain-specific-sentiment-lexicons/