Elamur Research⌕
☰
home / papers / 2004 / A Sentimental Education: Sentiment Analysis Using Subjectivity Summarization Based on Minimum Cuts

A Sentimental Education: Sentiment Analysis Using Subjectivity Summarization Based on Minimum Cuts

Summary

The paper tackles document-level sentiment polarity classification: deciding whether a movie review is positive or negative. Pang and Lee argue that much of a review, especially plot description, is objective and can mislead a bag-of-words classifier. They propose a two-stage pipeline. First, a sentence-level subjectivity detector keeps only the opinionated sentences. Then a standard polarity classifier (Naive Bayes or SVM) labels the resulting extract.

To train the detector, they automatically build a subjectivity dataset from the Web. Snippets from Rotten Tomatoes serve as subjective examples and IMDb plot-summary sentences as objective ones. Detection is framed as a minimum-cut problem on a graph. Each sentence's own classifier score is combined with pairwise proximity scores that push nearby sentences toward the same label, and max-flow algorithms solve this exactly and efficiently.

On the polarity dataset, the extracts reach about 60% of the original word count while matching or beating full-review accuracy. Depending on the downstream classifier, accuracy is either preserved or rises significantly (82.8% to 86.4%). The extracts also double as compact opinion summaries.

Key points

  • The polarity dataset contains 1000 positive and 1000 negative movie reviews written before 2002. It is capped at 20 reviews per author per category and covers 312 authors.
  • The subjectivity dataset has 5000 Rotten Tomatoes review snippets and 5000 IMDb plot-summary sentences. All items are at least ten words long and come from post-2001 films, so they do not overlap with the polarity data.
  • The default polarity classifiers are NB and SVM using binary unigram-presence features, with length-normalized vectors for SVM.
  • For NB, subjectivity extracts work better than full reviews. Proximity-aware min-cut extraction gives further gains, which are statistically significant only with SVM subjectivity detectors.
  • When paragraph boundaries are added, the min-cut approach significantly beats treating whole paragraphs as classification units (NB: 86.4% vs 85.2%; SVM: 86.15% vs 85.45%).

Interesting ideas

  • I1. Filter first, then classify. Removing off-topic or objective sentences before the main classifier can both shrink the input and denoise it. This could help any document-level task where only some passages carry the signal.
  • I2. Cheap labels from naturally separated sources. Pairing short review blurbs with plot synopses yields a large labeled sentence corpus with no manual annotation. The same trick could work for low-resource languages wherever genres separate cleanly online.
  • I3. Min-cut as a glue layer. Per-item scores from any classifier, even a knowledge-rich one, can be merged with pairwise "same label" preferences. The optimum is found exactly in polynomial time, avoiding hand-crafted context features.
  • I4. Soft rather than hard structural constraints. Paragraph boundaries are handled by down-weighting cross-paragraph links instead of forcing a whole paragraph to share one label. This makes the system more robust to noisy segmentation.
  • I5. Position-based baselines as a sanity check. Comparing extracts built from the first N and last N sentences showed that reviewers tend to put opinions at the end. Such cheap baselines can reveal genre structure in other corpora as well.

Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.

Fields: Natural language processing

Cited by

541 papers of our library cite it (of the 140884 whose bibliographies we read); 56 of them have a page here, 485 not yet.

Cite

The paper itself, not this page. BibTeX from the library's record of the work. DOI 10.3115/1218955.1218990

@inproceedings{pang2004sentimental,
  title = {{A Sentimental Education: Sentiment Analysis Using Subjectivity Summarization Based on Minimum Cuts}},
  author = {Pang, Bo and Lee, Lillian},
  booktitle = {Proceedings of the 42nd Annual Meeting of the Association for Computational Linguistics (ACL-04)},
  year = {2004},
  doi = {10.3115/1218955.1218990},
  url = {https://doi.org/10.3115/1218955.1218990}
}

References

26 references, 5 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

  1. Agrawal, Rakesh, Sridhar Rajagopalan, Ramakrishnan Srikant, and Yirong Xu. 2003. Mining newsgroups using networks arising from social behavior. In WWW, pages 529–535. find
  2. Ahuja, Ravindra, Thomas L. Magnanti, and James B. Orlin. 1993. Network Flows: Theory, Algorithms, and Applications. Prentice Hall. find
  3. Beineke, Philip, Trevor Hastie, Christopher Manning, and Shivakumar Vaithyanathan. 2004. Exploring sentiment summarization. In AAAI Spring Symposium on Exploring Attitude and Affect in Text: Theories and Applications (AAAI tech report SS-04-07). find
  4. Blum, Avrim and Shuchi Chawla. 2001. Learning from labeled and unlabeled data using graph min-cuts. In Intl. Conf. on Machine Learning (ICML), pages 19–26. find
  5. Boykov, Yuri, Olga Veksler, and Ramin Zabih. 1999. Fast approximate energy minimization via graph cuts. In Intl. Conf. on Computer Vision (ICCV), pages 377–384. Journal version in IEEE Trans. Pattern Analysis and Machine Intelligence (PAMI) 23(11):1222–1239, 2001. find
  6. Cardie, Claire, Janyce Wiebe, Theresa Wilson, and Diane Litman. 2003. Combining low-level and summary representations of opinions for multi-perspective question answering. In AAAI Spring Symposium on New Directions in Question Answering, pages 20–27. find
  7. Cormen, Thomas H., Charles E. Leiserson, and Ronald L. Rivest. 1990. Introduction to Algorithms. MIT Press. find
  8. Das, Sanjiv and Mike Chen. 2001. Yahoo! for Amazon: Extracting market sentiment from stock message boards. In Asia Pacific Finance Association Annual Conf. (APFA). find
  9. Dave, Kushal, Steve Lawrence, and David M. Pennock. 2003. Mining the peanut gallery: Opinion extraction and semantic classification of product reviews. In WWW, pages 519–528. find
  10. Dini, Luca and Giampaolo Mazzini. 2002. Opinion classification through information extraction. In Intl. Conf. on Data Mining Methods and Databases for Engineering, Finance and Other Fields, pages 299–310. find
  11. Durbin, Stephen D., J. Neal Richter, and Doug Warner. 2003. A system for affective rating of texts. In KDD Wksp. on Operational Text Classification Systems (OTC-3). find
  12. Hatzivassiloglou, Vasileios and Kathleen Mc-Keown. 1997. Predicting the semantic orientation of adjectives. In 35th ACL/8th EACL, pages 174–181. find
  13. Joachims, Thorsten. 2003. Transductive learning via spectral graph partitioning. In Intl. Conf. on Machine Learning (ICML). find
  14. Liu, Hugo, Henry Lieberman, and Ted Selker. 2003. A model of textual affect sensing using real-world knowledge. In Intelligent User Interfaces (IUI), pages 125–132. find
  15. Montes-y-Gómez, Manuel, Aurelio López-López, and Alexander Gelbukh. 1999. Text mining as a social thermometer. In IJCAI Wksp. on Text Mining, pages 103–107. find
  16. Morinaga, Satoshi, Kenji Yamanishi, Kenji Tateishi, and Toshikazu Fukushima. 2002. Mining product reputations on the web. In KDD, pages 341– 349. Industry track. find
  17. Pang, Bo, Lillian Lee, and Shivakumar Vaithyanathan. 2002. Thumbs up? Sentiment classification using machine learning techniques. In EMNLP, pages 79–86. find our card
  18. Qu, Yan, James Shanahan, and Janyce Wiebe, editors. 2004. AAAI Spring Symposium on Exploring Attitude and Affect in Text: Theories and Applications. AAAI technical report SS-04-07. find
  19. Riloff, Ellen and Janyce Wiebe. 2003. Learning extraction patterns for subjective expressions. In EMNLP. find our card
  20. Riloff, Ellen, Janyce Wiebe, and Theresa Wilson. 2003. Learning subjective nouns using extraction pattern bootstrapping. In Conf. on Natural Language Learning (CoNLL), pages 25–32. find our card
  21. Subasic, Pero and Alison Huettner. 2001. Affect analysis of text using fuzzy semantic typing. IEEE Trans. Fuzzy Systems, 9(4):483–496. find
  22. Tong, Richard M. 2001. An operational system for detecting and tracking opinions in on-line discussion. SIGIR Wksp. on Operational Text Classification. find
  23. Turney, Peter. 2002. Thumbs up or thumbs down? Semantic orientation applied to unsupervised classification of reviews. In ACL, pages 417–424. find our card
  24. Wiebe, Janyce M. 1994. Tracking point of view in narrative. Computational Linguistics, 20(2):233– 287. find
  25. Yi, Jeonghee, Tetsuya Nasukawa, Razvan Bunescu, and Wayne Niblack. 2003. Sentiment analyzer: Extracting sentiments about a given topic using natural language processing techniques. In IEEE Intl. Conf. on Data Mining (ICDM). find
  26. Yu, Hong and Vasileios Hatzivassiloglou. 2003. Towards answering opinion questions: Separating facts from opinions and identifying the polarity of opinion sentences. In EMNLP. find our card

↑ back to top

This page in Markdown → https://md.research.elamur.ai/free/en/papers/pang-2004-sentimental-education-sentiment-analysis-subjectivity/