Elamur Research⌕
☰
home / papers / 2005 / Extracting Semantic Orientations of Words using Spin Model

Extracting Semantic Orientations of Words using Spin Model

Paper · 2005 · Hiroya Takamura, Takashi Inui and Manabu Okumura

Summary

The paper tackles automatic construction of a polarity lexicon, which labels English words as desirable (positive) or undesirable (negative), starting from only a handful of labelled seed words. The authors treat each word as an electron in an Ising spin model, where a word's polarity is its spin. They build a lexical network in which words are connected when one appears in the other's dictionary gloss. Links are marked as opposite-polarity when a negation word is involved. Two richer variants add thesaurus relations (synonyms, antonyms, hypernyms) and corpus "and"/"but" adjective conjunctions. Exact inference is intractable, so the mean field approximation is used to compute each word's average spin, and a penalty term keeps seed words close to their known labels. To choose the inverse-temperature β without much labelled data, they monitor magnetization and pick the value just before the phase transition. With as few as 2, 4 or 14 seeds, the method reaches accuracy comparable to the best previously reported. It also beats shortest-path and bootstrapping baselines, showing that global optimization copes well with noisy resources such as glosses and corpora.

Key points

  • Three networks are compared: the gloss network (G), the gloss-thesaurus network (GT) and the gloss-thesaurus-corpus network (GTC). Antonyms, "but" conjunctions and negated gloss words create different-orientation links. Edge weights are normalized by the degrees of both words, 1/√(d(i)d(j)).
  • Experiments use 14, 4 or 2 seed words. In the magnetization plot, the sharp change happens around β ≈ 1.0, which is roughly where classification accuracy peaks.
  • Ranking words by the absolute value of their average spin works as a confidence score. With 14 seeds, the top 1000 words reach over 92% precision.
  • On a synonym-only network (697 adjectives), the spin model beats the Kamps et al. (2004) shortest-path method:
    • 14 seeds: 73.4 vs 70.8
    • 4 seeds: 71.0 vs 64.9
    • 2 seeds: 68.2 vs 66.0
  • On a synonym+antonym network (1470 adjectives), it beats Hu and Liu (2004)-style bootstrapping by about 10 points (e.g., 83.6 vs 72.8 with 14 seeds). Remaining errors come from word-sense ambiguity, missing syntactic structure in glosses, and idioms (e.g., "brag" = "show off").

Interesting ideas

  • I1. Signed graphs for label propagation. Encoding "same label" and "opposite label" evidence as positive and negative edge weights lets antonyms and contrastive conjunctions inform the result. This could suit any binary lexicon induction task, such as building a Ukrainian sentiment dictionary from a monolingual dictionary.
  • I2. Seeds as a soft prior. Labelled seeds enter as a quadratic penalty rather than fixed values. This keeps semi-supervised inference smooth and could tolerate imperfect seeds.
  • I3. Unsupervised hyper-parameter selection via phase transition. Sweeping the temperature and stopping just before global magnetization jumps gives a principled way to tune a smoothing parameter when no development set exists. The same trick may apply to other graph-based propagation methods.
  • I4. Cheap leave-one-out when labels are plentiful. One mean-field update for each seed, with its own penalty term dropped, gives a pseudo leave-one-out error. This is an inexpensive alternative to full cross-validation.
  • I5. Magnitude as confidence. The absolute value of a word's average spin serves as a confidence score. This allows a high-precision subset to be released first, which is useful when bootstrapping a resource that people will review manually.

Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.

 Institutions

🎓Tokyo Institute of TechnologyTokyo · Japan14 works · 1,759 citations

📚 Shelves 4The field of science this work belongs to (OpenAlex fields).Computer Science · 9,085 worksArtificial Intelligence · 8,888 worksArts and Humanities · 9,296 worksLanguage and Linguistics · 9,206 works

Cite

Cite as Hiroya Takamura, Takashi Inui and Manabu Okumura. 2005. Extracting Semantic Orientations of Words using Spin Model. Proceedings of the 43rd Annual Meeting of the Association for Computational Linguistics (ACL'05).

The paper itself, not this page. BibTeX from the library's record of the work. DOI 10.3115/1219840.1219857

@inproceedings{takamura2005extracting,
  title = {{Extracting Semantic Orientations of Words using Spin Model}},
  author = {Hiroya Takamura, Takashi Inui and Manabu Okumura},
  booktitle = {Proceedings of the 43rd Annual Meeting of the Association for Computational Linguistics (ACL'05)},
  year = {2005},
  doi = {10.3115/1219840.1219857},
  url = {https://doi.org/10.3115/1219840.1219857}
}

References

20 references, 2 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

  1. Adam L. Berger, Stephen Della Pietra, and Vincent J. Della Pietra. 1996. A maximum entropy approach to natural language processing. Computational Linguistics, 22(1):39–71.
  2. David Chandler. 1987. Introduction to Modern Statistical Mechanics. Oxford University Press. find
  3. Jim Cowie, Joe Guthrie, and Louise Guthrie. 1992. Lexical disambiguation using simulated annealing. In Proceedings of the 14th conference on Computational linguistics, volume 1, pages 359–365. find
  4. Christiane Fellbaum. 1998. WordNet: An Electronic Lexical Database, Language, Speech, and Communication Series. MIT Press. find
  5. Stuart Geman and Donald Geman. 1984. Stochastic relaxation, gibbs distributions, and the bayesian restoration of images. IEEE Transactions on Pattern Analysis and Machine Intelligence, 6:721–741. find
  6. Vasileios Hatzivassiloglou and Kathleen R. McKeown. 1997. Predicting the semantic orientation of adjectives. In Proceedings of the Thirty-Fifth Annual Meeting of the Association for Computational Linguistics and the Eighth Conference of the European Chapter of the Association for Computational Lingu… find
  7. Minqing Hu and Bing Liu. 2004. Mining and summarizing customer reviews. In Proceedings of the 2004 ACM SIGKDD international conference on Knowledge discovery and data mining (KDD-2004), pages 168–177. find
  8. Yukito Iba. 1999. The nishimori line and bayesian statistics. Journal of Physics A: Mathematical and General, pages 3875–3888. find
  9. Junichi Inoue and Domenico M. Carlucci. 2001. Image restoration using the q-ising spin glass. Physical Review E, 64:036121–1 – 036121–18. find
  10. Jaap Kamps, Maarten Marx, Robert J. Mokken, and Maarten de Rijke. 2004. Using wordnet to measure semantic orientation of adjectives. In Proceedings of the 4th International Conference on Language Resources and Evaluation (LREC 2004), volume IV, pages 1115–1118. find
  11. Nozomi Kobayashi, Takashi Inui, and Kentaro Inui. 2001. Dictionary-based acquisition of the lexical knowledge for p/n analysis (in Japanese). In Proceedings of Japanese Society for Artificial Intelligence, SLUD-33, pages 45–50. find
  12. David J. C. Mackay. 2003. Information Theory, Inference and Learning Algorithms. Cambridge University Press. find
  13. Jose L. Marroquin. 1985. Optimal bayesian estimators for image segmentation and surface reconstruction. Technical Report A.I. Memo 839, Massachusetts Institute of Technology. find
  14. Ellen Riloff, Janyce Wiebe, and Theresa Wilson. 2003. Learning subjective nouns using extraction pattern bootstrapping. In Proceedings of the Seventh Conference on Natural Language Learning (CoNLL-03), pages 25–32.
  15. Helmut Schmid. 1994. Probabilistic part-of-speech tagging using decision trees. In Proceedings of International Conference on New Methods in Language Processing, pages 44–49. find
  16. Philip J. Stone, Dexter C. Dunphy, Marshall S. Smith, and Daniel M. Ogilvie. 1966. The General Inquirer: A Computer Approach to Content Analysis. The MIT Press. find
  17. Kazuyuki Tanaka, Junichi Inoue, and Mike Titterington. 2003. Probabilistic image processing by means of the bethe approximation for the q-ising model. Journal of Physics A: Mathematical and General, 36:11023– 11035. find
  18. Peter D. Turney and Michael L. Littman. 2003. Measuring praise and criticism: Inference of semantic orientation from association. ACM Transactions on Information Systems, 21(4):315–346. find
  19. Jean Veronis and Nancy M. Ide. 1990. Word sense disambiguation with very large neural networks extracted from machine readable dictionaries. In Proceedings of the 13th Conference on Computational Linguistics, volume 2, pages 389–394. find
  20. Janyce M. Wiebe. 2000. Learning subjective adjectives from corpora. In Proceedings of the 17th National Conference on Artificial Intelligence (AAAI-2000), pages 735–740. find

Updated

↑ back to top

This page in Markdown → https://md.research.elamur.ai/free/en/papers/takamura-2005-extracting-semantic-orientations-words-spin/