📄 Biographies, Bollywood, Boom-boxes and Blenders: Domain Adaptation for Sentiment Classification

Summary

Sentiment classifiers trained on one kind of review often lose accuracy on others, and labeling data for every new product category is costly. The authors study cross-domain transfer for binary review polarity. They build a new Amazon corpus with reviews from four categories: books, DVDs, electronics and kitchen appliances. They then extend structural correspondence learning (SCL) in two ways. First, pivot features are chosen not only because they are frequent in both domains but also because of their mutual information with the source labels (SCL-MI). Second, a small number of labeled target examples is used to fix cases where SCL links features incorrectly. On average, these changes cut the relative error caused by domain shift by 30% compared with original SCL and by 46% compared with a non-adapted baseline. The authors also show that the A-distance, which needs only unlabeled data, correlates well with adaptation loss. This makes it a practical tool for deciding which few domains are worth annotating as sources.

Key points

• Dataset. The Amazon reviews corpus has four domains, each with 1000 positive and 1000 negative reviews.
  • Reviews above 3 stars count as positive and below 3 as negative; 3-star reviews are discarded.
  • Each domain is split 1600 for training and 400 for testing.
  • Each domain also has 3685 (DVDs) to 5945 (kitchen) unlabeled reviews.
• Classifier setup. The model is a linear classifier over unigram and bigram features, trained with Huber loss and stochastic gradient descent.
  • SCL uses k = 50 singular vectors.
  • Pivots must appear in more than five documents per domain.
• Example result (DVDs → books). The baseline reaches 72.8%, SCL-MI reaches 79.7%, and the in-domain classifier reaches 80.4%. SCL-MI therefore removes about 90.8% of the adaptation loss.
• Domain grouping. The domains form two clusters: books/DVDs and electronics/kitchen. For kitchen → electronics, SCL-MI even beats the in-domain classifier, apparently thanks to information in the unlabeled data.
• Feature overlap. For books versus kitchen, 85–90% of each domain's top 1000 informative features are not among the other domain's most informative ones.

Interesting ideas

• I1. Task-aware pivot selection. Pick bridge features that are both shared across domains and strongly tied to the label, not just frequent. Frequent shared words suit tasks like POS tagging, but for semantic tasks they are mostly noise. This filter could carry over to any representation-alignment method for new text genres, such as Ukrainian news versus social media.
• I2. Correspondence via shared co-occurrence. Domain-specific phrases can be aligned without any labels if they correlate with the same pivot words in unlabeled text. For example, a phone-specific phrase and a laptop-specific phrase may both co-occur with "excellent". This is a cheap way to transfer lexical knowledge when target labels are scarce.
• I3. Limited use of target labels. When a few labeled target examples exist, the authors update only the part of the model tied to the learned projection rather than retraining everything. This fits small budgets because it corrects misaligned features without overfitting the full weight vector.
• I4. Unlabeled domain distance to guide annotation. The A-distance, measured from unlabeled data alone, can rank candidate source domains before any annotation money is spent. This is useful for planning which corpora or regions to label first.
• I5. Similar sources beat a diverse mix. In the authors' experiments, a source close to the target worked better than mixing several sources for the same amount of labeled data. This challenges the habit of simply pooling all available data when building training sets.