Elamur Research⌕
☰
home / papers / 2006 / Partially Supervised Coreference Resolution for Opinion Summarization through Structured Rule Learning

Partially Supervised Coreference Resolution for Opinion Summarization through Structured Rule Learning

Summary

Opinion summaries need every opinion grouped under the real-world entity that holds it. The authors frame this as source coreference resolution: deciding which opinion-source mentions in a document point to the same entity. Standard NP coreference systems expect coreference labels for every noun phrase. Opinion corpora such as MPQA, however, mark chains only for source NPs. The authors call this setting partially supervised clustering, where a clustering function is learned from examples that are only partly labeled. They distinguish it from semi-supervised clustering, which has no separate training phase. Their algorithm, StRip, modifies the RIPPER rule learner so that the pairwise coreference classifier is trained jointly with the clustering step, and it can also make use of the unlabeled NPs. StRip uses the Ng and Cardie (2002) features and single-link clustering. On MPQA it beats extensively tuned SVM and RIPPER baselines, trained either on MUC6/MUC7 or on MPQA. This shows that partial in-domain supervision, combined with unlabeled data, can outperform full supervision from another domain.

Key points

  • Best StRip run (full MPQA, unlabeled NPs included) reaches B³ 83.2 versus 81.8 for the strongest baseline, and positive-link F1 of 67.1 versus 60.9 (precision 77.1, recall 59.4).
  • Baselines are the best runs found across 336 SVM light settings and 144 RIPPER settings, trained on MUC6, MUC7, or MPQA restricted to source NPs.
  • When StRip is trained only on source NPs (MPQA src), it scores B³ 82.3 and F1 64.6, but this gain is not statistically significant (p > 0.20). The full-corpus gains are significant under both a paired t-test and the Wilcoxon test.
  • Baselines trained on MPQA match or exceed those trained on MUC. This suggests that domain similarity matters more here than complete supervision.
  • Stated limitations and future work: target coreference resolution (MPQA lacked target annotations at the time), fully automatic extraction of sources and opinions, and learning that optimizes B³ directly.

Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.

Fields: Linguistics & languages

Topics: Coreference resolution

Cited by

10 papers of our library cite it (of the 140673 whose bibliographies we read); 2 of them have a page here, 8 not yet.

Cite

The paper itself, not this page. BibTeX from the library's record of the work.

@inproceedings{stoyanov2006partially,
  title = {{Partially Supervised Coreference Resolution for Opinion Summarization through Structured Rule Learning}},
  author = {Stoyanov, Veselin and Cardie, Claire},
  booktitle = {Proceedings of the 2006 Conference on Empirical Methods in Natural Language Processing},
  year = {2006},
  doi = {10.3115/1610075.1610123},
  url = {https://doi.org/10.3115/1610075.1610123}
}

References

31 references, 1 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

  1. A. Bagga and B. Baldwin. 1998. Entity-based cross-document coreferencing using the vector space model. In In Proceedings of COLING/ACL. find
  2. S. Basu. 2005. Semi-supervised Clustering: Probabilistic Models, Algorithms and Experiments. Ph.D. thesis, Department of Computer Sciences, UT at Austin. find
  3. S. Bethard, H. Yu, A. Thornton, V. Hativassiloglou, and D. Jurafsky. 2004. Automatic extraction of opinion propositions and their holders. In 2004 AAAI Spring Symposium on Exploring Attitude and Affect in Text. find
  4. Y. Choi, C. Cardie, E. Riloff, and S. Patwardhan. 2005. Identifying sources of opinions with conditional random fields and extraction patterns. In Proceedings of EMNLP. find
  5. C. Coglianese. 2004. E-rulemaking: Information technology and regulatory policy: New directions in digital government research. Technical report, Harvard University, J. F. Kennedy School of Government. find
  6. W. Cohen. 1995. Fast effective rule induction. In Proceedings of ICML. find
  7. S. Das and M. Chen. 2001. Yahoo for amazon: Extracting market sentiment from stock message boards. In Proceedings of APFAAC. find
  8. K. Dave, S. Lawrence, and D. Pennock. 2003. Mining the peanut gallery: Opinion extraction and semantic classification of product reviews. In Proceedings of IWWWC. find
  9. I. Davidson and S. Ravi. 2005. Clustering with constraints: Feasibility issues and the k-means algorithm. In Proceedings of SDM. find
  10. A. Demiriz, K. P. Bennett, and M. J. Embrechts. 1999. Semi-supervised clustering using genetic algorithms. In Proceeding of ANNIE. find
  11. T. Finley and T. Joachims. 2005. Supervised clustering with support vector machines. In Proceedings of ICML. find
  12. R. Iida, K. Inui, H. Takamura, and Y. Matsumoto. 2003. Incorporating contextual cues in trainable models for coreference resolution. In Proceedings of the EACL Workshop on The Computational Treatment of Anaphora. find
  13. T. Joachims. 1998. Making large-scale support vector machine learning practical. In A. Smola B. Schölkopf, C. Burges, editor, Advances in Kernel Methods: Support Vector Machines. MIT Press, Cambridge, MA. find
  14. S. Kim and E. Hovy. 2005. Identifying opinion holders for question answering in opinion texts. In Proceedings of AAAI Workshop on Question Answering in Restricted Domains. find
  15. X. Li and D. Roth. 2005. Discriminative training of clustering functions: Theory and experiments with entity identification. In Proceedings of CoNLL. find
  16. A. McCallum and B. Wellner. 2003. Toward conditional models of identity uncertainty with application to proper noun coreference. In Proceedings of the IJCAI Workshop on Information Integration on the Web. find
  17. T. Morton. 2000. Coreference for NLP applications. In Proceedings of ACL. find
  18. V. Ng and C. Cardie. 2002. Improving machine learning approaches to coreference resolution. In In Proceedings of ACL. find our card
  19. B. Pang and L. Lee. 2004. A sentimental education: Sentiment analysis using subjectivity summarization based on minimum cuts. In Proceedings of ACL. find
  20. B. Pang, L. Lee, and S. Vaithyanathan. 2002. Thumbs up? Sentiment classification using machine learning techniques. In Proceedings of EMNLP. find
  21. E. Riloff and J. Wiebe. 2003. Learning extraction patterns for subjective expressions. In Proceesings of EMNLP. find
  22. E. Riloff, J. Wiebe, and W. Phillips. 2005. Exploiting subjectivity classification to improve information extraction. In Proceedings of AAAI. find
  23. V. Stoyanov and C. Cardie. 2006. Toward opinion summarization: Linking the sources. In Proceedings of the ACL Workshop on Sentiment and Subjectivity in Text. find
  24. V. Stoyanov, C. Cardie, and J. Wiebe. 2005. Multi-Perspective question answering using the OpQA corpus. In Proceedings of EMNLP. find
  25. P. Turney. 2002. Thumbs up or thumbs down? Semantic orientation applied to unsupervised classification of reviews. In Proceedings of ACL. find
  26. K. Wagstaff and C. Cardie. 2000. Clustering with instance-level constraints. In Proceedings of the 17-th National Conference on Artificial Intelligence and 12-th Conference on Innovative Applications of Artificial Intelligence. find
  27. J. Wiebe and E. Riloff. 2005. Creating subjective and objective sentence classifiers from unannotated texts. In Proceedings of CICLing. find
  28. J. Wiebe, T. Wilson, and C. Cardie. 2005. Annotating expressions of opinions and emotions in language. Language Resources and Evaluation, 1(2). find
  29. T. Wilson, J. Wiebe, and R. Hwa. 2004. Just how mad are you? Finding strong and weak opinion clauses. In Proceedings of AAAI. find
  30. I.H. Witten and E. Frank. 2000. Data Mining: Practical machine learning tools with Java implementations. Morgan Kaufmann, San Francisco. find
  31. H. Yu and V. Hatzivassiloglou. 2003. Towards answering opinion questions: Separating facts from opinions and identifying the polarity of opinion sentences. In Proceedings of EMNLP. find

↑ back to top

This page in Markdown → https://md.research.elamur.ai/free/en/papers/stoyanov-2006-partially-supervised-coreference-resolution-opinion/