Elamur Research⌕
☰
home / papers / 2006 / Extracting Opinions, Opinion Holders, and Topics Expressed in Online News Media Text

Extracting Opinions, Opinion Holders, and Topics Expressed in Online News Media Text

Summary

Kim and Hovy address a gap in opinion mining. Earlier systems could detect subjective words and their polarity, but they rarely identified who holds an opinion and what it is about. This matters especially for news, where many people, organizations and countries voice different views. The authors build a pipeline anchored on opinion-bearing verbs and adjectives, and it has three steps:

  1. Find an opinion word.
  2. Run semantic role labeling around that word.
  3. Map the labeled roles to holder and topic.

Training data comes from FrameNet II. They select frames whose lexical units match a hand-annotated list of opinion words, then treat frame elements as holder and topic candidates. A Maximum Entropy role classifier is trained on several thousand FrameNet sentences. Some opinion words have no FrameNet entry, so CBC clustering assigns each of them the frame of its most similar covered word. Tested on news sentences from online outlets, the system clearly outperforms a baseline. The weak baseline suggests that simple subject/object heuristics cannot capture holder and topic relations.

Key points

  • Opinion word lexicon: 2011 verbs and 1860 adjectives were sampled at random from larger pools (8011 verbs, 19748 adjectives). Each was labeled positive, negative or neutral. The final lexicon has 69 positive and 151 negative verbs, plus 199 positive and 304 negative adjectives.
  • Frame selection: FrameNet II contains 450 frames and more than 3000 frame elements, along with roughly 7500 lexical units and in excess of 100,000 sentences carrying annotations. From it, 49 opinion-related frames were selected for verbs and 43 for adjectives, such as Desiring, Emotion_directed, Mental_property and Subject_stimulus. These frames yielded 8256 verb sentences and 11877 adjective sentences for training.
  • Role classifier: a Maximum Entropy model, implemented with the MEGA model optimization package. Its features are:
    • target word
    • phrase type
    • head word
    • parse tree path
    • position relative to the target
    • voice
    • frame name
  • Coverage expansion: words missing from FrameNet, such as vilify, borrow the frame of a distributionally similar covered word found by CBC (Clustering By Committee). For vilify, that word is criticize, which belongs to the frame Judgment_communication.
  • Evaluation data: test sentences came from New York Times, IRIN News and BBC websites and were annotated by two humans (Human1, Human2). The authors note two scoring problems:
    • Several holder or topic answers could be valid for the same opinion.
    • Coreferent correct answers were counted as errors because annotators marked only one mention.

Interesting ideas

  • I1. Reuse a general resource for a specific task: the authors relabel FrameNet's existing annotations instead of annotating holders and topics from scratch. Opinion words become frame targets, and frame elements become holder or topic. Mapping a broad semantic resource onto a narrower task like this could bootstrap training data for low-resource tasks or languages.
  • I2. Neutral as a third lexicon class: weak cues such as central or able are marked neutral rather than forced into positive or negative. This keeps noise out of the lexicon and is a cheap way to improve precision in any sentiment word list.
  • I3. Frame inheritance for unseen words: a word missing from the lexical resource takes the frame of its nearest distributional neighbour. This simple fallback could extend sparse resources, for example a smaller FrameNet-style database for another language, without new manual annotation.
  • I4. Anchor extraction on a trigger word: the system first finds the opinion-bearing predicate and only then looks for its arguments. This narrows the search space and makes the extracted triples interpretable, which suits event, relation or stance extraction from news.
  • I5. Be explicit about scoring artifacts: the authors acknowledge that multiple valid answers and coreferent mentions can make a correct output look wrong. Evaluation protocols for span extraction should allow alternative gold answers or coreference-aware matching.

Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.

Fields: Natural language processing

Topics: Semantic role labeling

Cited by

99 papers of our library cite it (of the 140884 whose bibliographies we read); 3 of them have a page here, 96 not yet.

Cite

The paper itself, not this page. BibTeX from the library's record of the work.

@inproceedings{kim2006extracting,
  title = {{Extracting Opinions, Opinion Holders, and Topics Expressed in Online News Media Text}},
  author = {Kim, Soo-Min and Hovy, Eduard},
  booktitle = {Proceedings of the Workshop on Sentiment and Subjectivity in Text},
  year = {2006},
  doi = {10.3115/1654641.1654642},
  url = {https://doi.org/10.3115/1654641.1654642}
}

References

20 references, 8 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

  1. Baker, Collin F. and Hiroaki Sato. 2003. The Frame-Net Data and Software. Poster and Demonstration at Association for Computational Linguistics. Sapporo, Japan. find
  2. Berger, Adam, Stephen Della Pietra, and Vincent Della Pietra. 1996. A maximum entropy approach to natural language processing, Computational Linguistics, (22-1). find our card
  3. Bethard, Steven, Hong Yu, Ashley Thornton, Vasileios Hatzivassiloglou, and Dan Jurafsky. 2004. Automatic Extraction of Opinion Propositions and their Holders, AAAI Spring Symposium on Exploring Attitude and Affect in Text: Theories and Applications. find
  4. Choi, Y., Cardie, C., Riloff, E., and Patwardhan, S. 2005. Identifying Sources of Opinions with Conditional Random Fields and Extraction Patterns. Proceedings of HLT/EMNLP-05. find
  5. Esuli, Andrea and Fabrizio Sebastiani. 2005. Determining the semantic orientation of terms through gloss classification. Proceedings of CIKM-05, 14th ACM International Conference on Information and Knowledge Management, Bremen, DE, pp. 617-624. find
  6. Fillmore, C. Frame semantics and the nature of language. 1976. In Annals of the New York Academy of Sciences: Conferences on the Origin and Development of Language and Speech, Volume 280: 20-32. find
  7. Fleischman, Michael, Namhee Kwon, and Eduard Hovy. 2003. Maximum Entropy Models for FrameNet Classification. Proceedings of EMNLP, Sapporo, Japan. find our card
  8. Gildea, D. and Jurafsky, D. Automatic Labeling of semantic roles. 2002. In Computational Linguistics. 28(3), 245-288. find our card
  9. Harris, Zellig, 1954. Distributional structure. Word, 10(23) :146--162. find
  10. Hatzivassiloglou, Vasileios and Kathleen McKeown. 1997. Predicting the Semantic Orientation of Adjectives. Proceedings of 35th Annual Meeting of the Assoc. for Computational Linguistics (ACL-97): 174-181 find
  11. Hatzivassiloglou, Vasileios and Wiebe, Janyce. 2000. Effects of Adjective Orientation and Gradability on Sentence Subjectivity. Proceedings of International Conference on Computational Linguistics (COLING-2000). Saarbrücken, Germany. find
  12. Hu, Minqing and Bing Liu. 2004. Mining and summarizing customer reviews". Proceedings of the ACM SIGKDD International Conference on Knowledge Discovery & Data Mining (KDD-2004), Seattle, Washington, USA. find
  13. Kim, Soo-Min and Eduard Hovy. 2004. Determining the Sentiment of Opinions. Proceedings of COL-ING-04. pp. 1367-1373. Geneva, Switzerland. find our card
  14. Kim, Soo-Min and Eduard Hovy. 2005. Identifying Opinion Holders for Question Answering in Opinion Texts. Proceedings of AAAI-05 Workshop on Question Answering in Restricted Domains Pang, Bo, Lillian Lee, and Shivakumar Vaithyanathan. 2002. Thumbs up? Sentiment Classification using Machine Learning T… find
  15. Pantel, Patrick and Dekang Lin. 2002. Discovering Word Senses from Text. Proceedings of ACM Conference on Knowledge Discovery and Data Mining. (KDD-02). pp. 613-619. Edmonton, Canada. find
  16. Popescu, Ana-Maria and Oren Etzioni. 2005. Extracting Product Features and Opinions from Reviews , Proceedings of HLT-EMNLP 2005. find our card
  17. Riloff, Ellen, Janyce Wiebe, and Theresa Wilson. 2003. Learning Subjective Nouns Using Extraction Pattern Bootstrapping. Proceedings of Seventh Conference on Natural Language Learning (CoNLL-03). ACL SIGNLL. Pages 25-32. find our card
  18. Turney, Peter D. 2002. Thumbs up or thumbs down? Semantic orientation applied to unsupervised classification of reviews, Proceedings of ACL-02, Philadelphia, Pennsylvania, 417-424 find our card
  19. Wiebe, Janyce, Bruce M., Rebecca F., and Thomas P. O'Hara. 1999. Development and use of a gold standard data set for subjectivity classifications. Proceedings of ACL-99. University of Maryland, June, pp. 246-253. find
  20. Wilson, Theresa, Janyce Wiebe, and Paul Hoffmann. 2005. Recognizing Contextual Polarity in Phrase-Level Sentiment Analysis. Proceedings of HLT/EMNLP 2005, Vancouver, Canada find our card

↑ back to top

This page in Markdown → https://md.research.elamur.ai/free/en/papers/kim-2006-extracting-opinions-opinion-holders-topics/