A Sentimental Education: Sentiment Analysis Using Subjectivity Summarization Based on Minimum Cuts
Summary
The paper tackles document-level sentiment polarity classification: deciding whether a movie review is positive or negative. Pang and Lee argue that much of a review, especially plot description, is objective and can mislead a bag-of-words classifier. They propose a two-stage pipeline. First, a sentence-level subjectivity detector keeps only the opinionated sentences. Then a standard polarity classifier (Naive Bayes or SVM) labels the resulting extract.
To train the detector, they automatically build a subjectivity dataset from the Web. Snippets from Rotten Tomatoes serve as subjective examples and IMDb plot-summary sentences as objective ones. Detection is framed as a minimum-cut problem on a graph. Each sentence's own classifier score is combined with pairwise proximity scores that push nearby sentences toward the same label, and max-flow algorithms solve this exactly and efficiently.
On the polarity dataset, the extracts reach about 60% of the original word count while matching or beating full-review accuracy. Depending on the downstream classifier, accuracy is either preserved or rises significantly (82.8% to 86.4%). The extracts also double as compact opinion summaries.
Key points
- The polarity dataset contains 1000 positive and 1000 negative movie reviews written before 2002. It is capped at 20 reviews per author per category and covers 312 authors.
- The subjectivity dataset has 5000 Rotten Tomatoes review snippets and 5000 IMDb plot-summary sentences. All items are at least ten words long and come from post-2001 films, so they do not overlap with the polarity data.
- The default polarity classifiers are NB and SVM using binary unigram-presence features, with length-normalized vectors for SVM.
- For NB, subjectivity extracts work better than full reviews. Proximity-aware min-cut extraction gives further gains, which are statistically significant only with SVM subjectivity detectors.
- When paragraph boundaries are added, the min-cut approach significantly beats treating whole paragraphs as classification units (NB: 86.4% vs 85.2%; SVM: 86.15% vs 85.45%).
Interesting ideas
- I1. Filter first, then classify. Removing off-topic or objective sentences before the main classifier can both shrink the input and denoise it. This could help any document-level task where only some passages carry the signal.
- I2. Cheap labels from naturally separated sources. Pairing short review blurbs with plot synopses yields a large labeled sentence corpus with no manual annotation. The same trick could work for low-resource languages wherever genres separate cleanly online.
- I3. Min-cut as a glue layer. Per-item scores from any classifier, even a knowledge-rich one, can be merged with pairwise "same label" preferences. The optimum is found exactly in polynomial time, avoiding hand-crafted context features.
- I4. Soft rather than hard structural constraints. Paragraph boundaries are handled by down-weighting cross-paragraph links instead of forcing a whole paragraph to share one label. This makes the system more robust to noisy segmentation.
- I5. Position-based baselines as a sanity check. Comparing extracts built from the first N and last N sentences showed that reviewers tend to put opinions at the end. Such cheap baselines can reveal genre structure in other corpora as well.
Cited by
- A Comparative Study of Feature Selection Methods for Dialectal Arabic Sentiment Classification Using Support Vector Machine
- Negation Handling in Machine Learning-Based Sentiment Classification for Colloquial Arabic
- Emotions from Text: Machine Learning for Text-based Emotion Prediction
- A Corpus for Sentence-Level Subjectivity Detection on English News Articles
- Interpreting Neural Networks to Improve Politeness Comprehension
- Layer Normalization
- Robust Sentiment Detection on Twitter from Biased and Noisy Data
- Better Document-level Sentiment Analysis from RST Discourse Parsing
- Universal Sentence Encoder for English
- Universal Sentence Encoder
- DiffCSE: Difference-based Contrastive Learning for Sentence Embeddings
- SentEval: An Evaluation Toolkit for Universal Sentence Representations
- What you can cram into a single $&!#* vector: Probing sentence embeddings for linguistic properties
- Why Can GPT Learn In-Context? Language Models Secretly Perform Gradient Descent as Meta-Optimizers
- Semi-Supervised Recognition of Sarcasm in Twitter and Amazon
- RLPrompt: Optimizing Discrete Text Prompts with Reinforcement Learning
- ERASER: A Benchmark to Evaluate Rationalized NLP Models
- SENTIWORDNET: A Publicly Available Lexical Resource for Opinion Mining
- Language-agnostic BERT Sentence Embedding
- Making Pre-trained Language Models Better Few-shot Learners
- SimCSE: Simple Contrastive Learning of Sentence Embeddings
- DeCLUTR: Deep Contrastive Learning for Unsupervised Textual Representations
- Why are You Taking this Stance? Identifying and Classifying Reasons in Ideological Debates
- Learning Distributed Representations of Sentences from Unlabelled Data
- CODRA: A Novel Discriminative Framework for Rhetorical Analysis
- Your Sentiment Precedes You: Using an author's historical tweets to predict sarcasm
- Convolutional Neural Networks for Sentence Classification
- Contextual Augmentation: Data Augmentation by Words with Paradigmatic Relations
- Unified Demonstration Retriever for In-Context Learning
- An efficient framework for learning sentence representations
- Fantastically Ordered Prompts and Where to Find Them: Overcoming Few-Shot Prompt Order Sensitivity
- Learning Word Vectors for Sentiment Analysis
- Collective Classification for Fine-grained Information Status
- Learning Multilingual Subjective Language via Cross-Lingual Projections
- Noisy Channel Language Model Prompting for Few-Shot Text Classification
- Harnessing Cognitive Features for Sarcasm Detection
- Graph-Cut-Based Anaphoricity Determination for Coreference Resolution
- Representation Learning with Contrastive Predictive Coding
- Seeing Stars: Exploiting Class Relationships for Sentiment Categorization with Respect to Rating Scales
- Rumor has it: Identifying Misinformation in Microblogs
- Learning to Generate Reviews and Discovering Sentiment
- Semi-Supervised Recursive Autoencoders for Predicting Sentiment Distributions
- Supervised and Unsupervised Methods in Employing Discourse Relations for Improving Opinion Polarity Classification
- Partially Supervised Coreference Resolution for Opinion Summarization through Structured Rule Learning
- SemEval-2007 Task 14: Affective Text
- Lexicon-Based Methods for Sentiment Analysis
- Get out the vote: Determining support or opposition from Congressional floor-debate transcripts
- Co-Training for Cross-Lingual Sentiment Classification
- Baselines and Bigrams: Simple, Good Sentiment and Topic Classification
- EDA: Easy Data Augmentation Techniques for Boosting Performance on Text Classification Tasks
- Recognizing Contextual Polarity in Phrase-Level Sentiment Analysis
- Learning from Bullying Traces in Social Media
- GPT3Mix: Leveraging Large-scale Language Models for Text Augmentation
- Using “Annotator Rationales” to Improve Machine Learning for Text Categorization
- An Unsupervised Sentence Embedding Method by Mutual Information Maximization
- Multi-VALUE: A Framework for Cross-Dialectal English NLP
Cite
@inproceedings{pang2004sentimental,
title = {{A Sentimental Education: Sentiment Analysis Using Subjectivity Summarization Based on Minimum Cuts}},
author = {Pang, Bo and Lee, Lillian},
booktitle = {Proceedings of the 42nd Annual Meeting of the Association for Computational Linguistics (ACL-04)},
year = {2004},
doi = {10.3115/1218955.1218990},
url = {https://doi.org/10.3115/1218955.1218990}
}References
- Agrawal, Rakesh, Sridhar Rajagopalan, Ramakrishnan Srikant, and Yirong Xu. 2003. Mining newsgroups using networks arising from social behavior. In WWW, pages 529–535. find
- Ahuja, Ravindra, Thomas L. Magnanti, and James B. Orlin. 1993. Network Flows: Theory, Algorithms, and Applications. Prentice Hall. find
- Beineke, Philip, Trevor Hastie, Christopher Manning, and Shivakumar Vaithyanathan. 2004. Exploring sentiment summarization. In AAAI Spring Symposium on Exploring Attitude and Affect in Text: Theories and Applications (AAAI tech report SS-04-07). find
- Blum, Avrim and Shuchi Chawla. 2001. Learning from labeled and unlabeled data using graph min-cuts. In Intl. Conf. on Machine Learning (ICML), pages 19–26. find
- Boykov, Yuri, Olga Veksler, and Ramin Zabih. 1999. Fast approximate energy minimization via graph cuts. In Intl. Conf. on Computer Vision (ICCV), pages 377–384. Journal version in IEEE Trans. Pattern Analysis and Machine Intelligence (PAMI) 23(11):1222–1239, 2001. find
- Cardie, Claire, Janyce Wiebe, Theresa Wilson, and Diane Litman. 2003. Combining low-level and summary representations of opinions for multi-perspective question answering. In AAAI Spring Symposium on New Directions in Question Answering, pages 20–27. find
- Cormen, Thomas H., Charles E. Leiserson, and Ronald L. Rivest. 1990. Introduction to Algorithms. MIT Press. find
- Das, Sanjiv and Mike Chen. 2001. Yahoo! for Amazon: Extracting market sentiment from stock message boards. In Asia Pacific Finance Association Annual Conf. (APFA). find
- Dave, Kushal, Steve Lawrence, and David M. Pennock. 2003. Mining the peanut gallery: Opinion extraction and semantic classification of product reviews. In WWW, pages 519–528. find
- Dini, Luca and Giampaolo Mazzini. 2002. Opinion classification through information extraction. In Intl. Conf. on Data Mining Methods and Databases for Engineering, Finance and Other Fields, pages 299–310. find
- Durbin, Stephen D., J. Neal Richter, and Doug Warner. 2003. A system for affective rating of texts. In KDD Wksp. on Operational Text Classification Systems (OTC-3). find
- Hatzivassiloglou, Vasileios and Kathleen Mc-Keown. 1997. Predicting the semantic orientation of adjectives. In 35th ACL/8th EACL, pages 174–181. find
- Joachims, Thorsten. 2003. Transductive learning via spectral graph partitioning. In Intl. Conf. on Machine Learning (ICML). find
- Liu, Hugo, Henry Lieberman, and Ted Selker. 2003. A model of textual affect sensing using real-world knowledge. In Intelligent User Interfaces (IUI), pages 125–132. find
- Montes-y-Gómez, Manuel, Aurelio López-López, and Alexander Gelbukh. 1999. Text mining as a social thermometer. In IJCAI Wksp. on Text Mining, pages 103–107. find
- Morinaga, Satoshi, Kenji Yamanishi, Kenji Tateishi, and Toshikazu Fukushima. 2002. Mining product reputations on the web. In KDD, pages 341– 349. Industry track. find
- Pang, Bo, Lillian Lee, and Shivakumar Vaithyanathan. 2002. Thumbs up? Sentiment classification using machine learning techniques. In EMNLP, pages 79–86. find our card
- Qu, Yan, James Shanahan, and Janyce Wiebe, editors. 2004. AAAI Spring Symposium on Exploring Attitude and Affect in Text: Theories and Applications. AAAI technical report SS-04-07. find
- Riloff, Ellen and Janyce Wiebe. 2003. Learning extraction patterns for subjective expressions. In EMNLP. find our card
- Riloff, Ellen, Janyce Wiebe, and Theresa Wilson. 2003. Learning subjective nouns using extraction pattern bootstrapping. In Conf. on Natural Language Learning (CoNLL), pages 25–32. find our card
- Subasic, Pero and Alison Huettner. 2001. Affect analysis of text using fuzzy semantic typing. IEEE Trans. Fuzzy Systems, 9(4):483–496. find
- Tong, Richard M. 2001. An operational system for detecting and tracking opinions in on-line discussion. SIGIR Wksp. on Operational Text Classification. find
- Turney, Peter. 2002. Thumbs up or thumbs down? Semantic orientation applied to unsupervised classification of reviews. In ACL, pages 417–424. find our card
- Wiebe, Janyce M. 1994. Tracking point of view in narrative. Computational Linguistics, 20(2):233– 287. find
- Yi, Jeonghee, Tetsuya Nasukawa, Razvan Bunescu, and Wayne Niblack. 2003. Sentiment analyzer: Extracting sentiments about a given topic using natural language processing techniques. In IEEE Intl. Conf. on Data Mining (ICDM). find
- Yu, Hong and Vasileios Hatzivassiloglou. 2003. Towards answering opinion questions: Separating facts from opinions and identifying the polarity of opinion sentences. In EMNLP. find our card
This page in Markdown → https://md.research.elamur.ai/free/en/papers/pang-2004-sentimental-education-sentiment-analysis-subjectivity/