Elamur Research⌕
☰
home / papers / 2009 / Co-Training for Cross-Lingual Sentiment Classification

Co-Training for Cross-Lingual Sentiment Classification

Summary

The paper addresses polarity classification of Chinese product reviews when labeled Chinese training data is unavailable but annotated English reviews exist. Wan uses Google Translate in both directions. English training reviews are rendered into Chinese, and unlabeled and test Chinese reviews are rendered into English. Each review therefore has two versions. The English-side features and the Chinese-side features serve as two separate views, which fits the co-training framework of Blum and Mitchell. Two SVM classifiers, one per view, are bootstrapped on unlabeled Chinese reviews (about 1000, kept separate from the test data). In each round, each classifier adds its most confident positive and negative examples to the shared labeled pool. The two final classifiers are merged by averaging their predictions. The method was compared with eight baselines: inductive SVM and transductive SVM, each using Chinese-only, English-only, or both views. Co-training reached 0.813 accuracy, compared with 0.790 for the strongest baseline. This shows that machine translation combined with unlabeled target-language text can partly compensate for missing sentiment corpora in a new language.

Key points

  • Setup: labeled English reviews (a sentiment dataset hosted by M. Dredze at UPenn) plus unlabeled Chinese reviews (source: it168.com). Translation in both directions uses Google Translate, and SVM is the base learner.
  • Baselines are SVM and TSVM in four variants each: CN, EN, ENCN1 (a single model on both feature sets), and ENCN2 (the averaged outputs of the EN and CN models).
  • Total accuracy: CoTrain (I=40, p=n=5) 0.813, TSVM(ENCN2) 0.790, SVM(ENCN2) 0.775, SVM(CN) 0.771, SVM(EN) 0.738. CoTrain beats every baseline on all reported precision, recall, and F-measure values.
  • Inside co-training, the separate English and Chinese classifiers score 0.775 and 0.790. Each is above its TSVM counterpart, and the Chinese one alone surpasses the best baseline for iteration counts of roughly 40–70.
  • Accuracy rises after about 20 iterations and later falls slightly as noisy examples enter the pool. It then plateaus once the unlabeled pool is exhausted. Imbalanced growth, such as (1,5) or (5,1), makes accuracy decline steadily.

Interesting ideas

  • I1. Translation as view creation: translating everything both ways turns a single-language problem into a two-view one. This makes co-training usable even without natural multi-view data, and it could transfer to Ukrainian NLP tasks that rely on English-labeled corpora.
  • I2. Keep the views separate, then combine: training per-language classifiers and averaging their scores worked better than merging all features into one model (ENCN2 > ENCN1). This is a cheap trick worth trying whenever two noisy feature sources exist.
  • I3. Balanced self-labeling: adding equal numbers of confident positives and negatives each round was essential, because skewed additions quickly degraded the model. Any bootstrapping or pseudo-labeling pipeline should respect class balance in the same way.
  • I4. Blind test set during bootstrapping: the Chinese reviews used for evaluation were excluded from the unlabeled pool. This protocol avoids optimistic transductive leakage and is worth copying in semi-supervised evaluations.
  • I5. Report the full iteration curve: plotting accuracy against the number of iterations exposes both the gain and the later drift caused by noisy examples. A curve gives a more honest picture than a single tuned operating point.

Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.

Fields: Natural language processing

Cited by

107 papers of our library cite it (of the 140884 whose bibliographies we read); 2 of them have a page here, 105 not yet.

Cite

The paper itself, not this page. BibTeX from the library's record of the work.

@inproceedings{wan2009co,
  title = {{Co-Training for Cross-Lingual Sentiment Classification}},
  author = {Wan, Xiaojun},
  booktitle = {Proceedings of the Joint Conference of the 47th Annual Meeting of the ACL and the 4th International Joint Conference on Natural Language Processing of the AFNLP},
  year = {2009},
  doi = {10.3115/1687878.1687913},
  url = {https://doi.org/10.3115/1687878.1687913}
}

References

43 references, 8 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

  1. A. Andreevskaia and S. Bergler. 2008. When specialists and generalists work together: overcoming domain dependence in sentiment tagging. In Proceedings of ACL-08: HLT. find
  2. C. Banea, R. Mihalcea, J. Wiebe and S. Hassan. 2008. Multilingual subjectivity analysis using machine translation. In Proceedings of EMNLP-2008. find
  3. N. Bel, C. H. A. Koster, and M. Villegas. 2003. Cross-lingual text categorization. In Proceedings of ECDL-03. find
  4. J. Blitzer, M. Dredze and F. Pereira. 2007. Biographies, bollywood, boom-boxes and blenders: domain adaptation for sentiment classification. In Proceedings of ACL-07. find our card
  5. A. Blum and T. Mitchell. 1998. Combining labeled and unlabeled data with cotraining. In Proceedings of COLT-98. find
  6. S. Brody, R. Navigli and M. Lapata. 2006. Ensemble methods for unsupervised WSD. In Proceedings of COLING-ACL-2006. find
  7. S. Clark, J. R. Curran, and M. Osborne. 2003. Bootstrapping POS taggers using unlabelled data. In Proceedings of CoNLL-2003. find
  8. W. Dai, G.-R. Xue, Q. Yang, Y. Yu. 2007a. Transferring Naïve Bayes Classifiers for text classification. In Proceedings of AAAI-07. find
  9. W. Dai, G.-R. Xue, Q. Yang, Y. Yu. 2007b. Co-clustering based classification for out-of-domain documents. In Proceedings of KDD-07. find
  10. H. DauméIII and D. Marcu. 2006. Domain adaptation for statistical classifiers. Journal of Artificial Intelligence Research, 26:101–126. find
  11. A. Devitt and K. Ahmad. 2007. Sentiment polarity identification in financial news: a cohesion-based approach. In Proceedings of ACL2007. find
  12. T. G. Dietterich. 1997. Machine learning research: four current directions. AI Magazine, 18(4), 1997. find
  13. A. Gliozzo and C. Strapparava. 2005. Cross language text categorization by acquiring multilingual domain models from comparable corpora. In Proceedings of the ACL Workshop on Building and Using Parallel Texts. find
  14. K. Hiroshi, N. Tetsuya and W. Hideo. 2004. Deeper sentiment analysis using machine translation technology. In Proceedings of COLING-04. find
  15. J. Jiang and C. Zhai. 2007. A two-stage approach to domain adaptation for statistical classifiers. In Proceedings of CIKM-07. find
  16. T. Joachims. 1999. Transductive inference for text classification using support vector machines. In Proceedings of ICML-99. find
  17. T. Joachims. 2002. Learning to classify text using support vector machines. find
  18. Dissertation, Kluwer, 2002. find
  19. A. Kennedy and D. Inkpen. 2006. Sentiment classification of movie reviews using contextual valence shifters. Computational Intelligence, 22(2):110-125. find
  20. S.-M. Kim and E. Hovy. 2004. Determining the sentiment of opinions. In Proceedings of COLING-04. find our card
  21. S. Kiritchenko and S. Matwin. 2001. Email classification with co-training. In Proceedings of the 2001 Conference of the Centre for Advanced Studies on Collaborative Research. find
  22. L.-W. Ku, Y.-T. Liang and H.-H. Chen. 2006. Opinion extraction, summarization and tracking in news and blog corpora. In Proceedings of AAAI-2006. find
  23. J. Li and M. Sun. 2007. Experimental study on sentiment classification of Chinese review using machine learning techniques. In Proceeding of IEEE-NLPKE-07. find
  24. X. Ling, W. Dai, Y. Jiang, G.-R. Xue, Q. Yang, and Y. Yu. 2008. Can Chinese Web pages be classified with English data source? In Proceedings of WWW-08. find
  25. B. Liu, M. Hu and J. Cheng. 2005. Opinion observer: Analyzing and comparing opinions on the web. In Proceedings of WWW-2005. find
  26. R. McDonald, K. Hannan, T. Neylon, M. Wells and J. Reynar. 2007. Structured models for fine-to-coarse sentiment analysis. In Proceedings of ACL-07. find
  27. R. Mihalcea. 2004. Co-training and self-training for word sense disambiguation. In Proceedings of CONLL-04. find
  28. R. Mihalcea, C. Banea and J. Wiebe. 2007. Learning multilingual subjective language via cross-lingual projections. In Proceedings of ACL-2007. find our card
  29. T. Mullen and N. Collier. 2004. Sentiment analysis using support vector machines with diverse information sources. In Proceedings of EMNLP-04. find
  30. V. Ng and C. Cardie. 2003. Weakly supervised natural language learning without redundant views. In Proceedings of HLT-NAACL-03. find
  31. K. Nigam, A. K. McCallum, S. Thrun, and T. Mitchell. 2000. Text Classification from Labeled and Unlabeled Documents using EM. Machine Learning, 39(2-3):103–134. find
  32. B. Pang, L. Lee and S. Vaithyanathan. 2002. Thumbs up? sentiment classification using machine learning techniques. In Proceedings of EMNLP-02. find our card
  33. B. Pang and L. Lee. 2004. A sentimental education: sentiment analysis using subjectivity summarization based on minimum cuts. In Proceedings of ACL-04. find our card
  34. J. Read. 2005. Using emoticons to reduce dependency in machine learning techniques for sentiment classification. In Proceedings of ACL-05. find
  35. L. Rigutini, M. Maggini and B. Liu. 2005. An EM based training algorithm for cross-language text categorization. In Proceedings of WI-05. find
  36. A. Sarkar. 2001. Applying cotraining methods to statistical parsing. In Proceedings of NAACL-2001. find
  37. I. Titov and R. McDonald. 2008. A joint model of text and aspect ratings for sentiment summarization. In Proceedings of ACL-08:HLT. find our card
  38. B. K. Y. Tsou, R. W. M. Yuen, O. Y. Kwong, T. B. Y. La and W. L. Wong. 2005. Polarity classification of celebrity coverage in the Chinese press. In Proceedings of International Conference on Intelligence Analysis. find
  39. P. Turney. 2002. Thumbs up or thumbs down? semantic orientation applied to unsupervised classification of reviews. In Proceedings of ACL-2002. find our card
  40. X. Wan. 2008. Using bilingual knowledge and ensemble techniques for unsupervised Chinese sentiment analysis. In Proceedings of EMNLP-2008. find
  41. T. Wilson, J. Wiebe and P. Hoffmann. 2005. Recognizing Contextual Polarity in Phrase-Level Sentiment Analysis. In Proceedings of HLT/EMNLP-05. find our card
  42. G.-R. Xue, W. Dai, Q. Yang, Y. Yu. 2008. Topic-bridged PLSA for cross-domain text classification. In Proceedings of SIGIR-08. find
  43. Q. Ye, W. Shi and Y. Li. 2006. Sentiment classification for movie reviews in Chinese by improved semantic oriented approach. In Proceedings of 39th Hawaii International Conference on System Sciences, 2006. find

↑ back to top

This page in Markdown → https://md.research.elamur.ai/free/en/papers/wan-2009-co-training-cross-lingual-sentiment/