Elamur ResearchβŒ•
☰
home / papers / 2009 / Finding Hedges by Chasing Weasels: Hedge Detection Using Wikipedia Tags and Shallow Linguistic Features

Finding Hedges by Chasing Weasels: Hedge Detection Using Wikipedia Tags and Shallow Linguistic Features

Summary

Ganter and Strube study how to automatically spot sentences containing linguistic hedges, which signal weak commitment to a claim. Earlier work focused almost entirely on biomedical texts. The authors treat Wikipedia as a corpus that editors have already labeled: style guidelines ask them to flag unsupported, non-neutral phrasing with a weasel tag ({{weasel-inline}} or {{weasel-word}}). The authors argue that such weasel words closely match the notion of hedging.

From several Wikipedia dumps, they built two classifiers:

  • wpw scores the words appearing just before weasel tags, combining relative frequency with distance to the tag.
  • asp adds shallow syntactic patterns detected over POS tags.

On automatically labeled test data, the two systems perform similarly, so the patterns appear to add little there. On a manually annotated gold standard, however, asp clearly wins, because it also finds hedges that editors never tagged.

The work matters because it shows that a freely available, multi-domain resource can supply hedge-detection training data. It also exposes a key limitation: weasel tags are sparse and short-lived, which leaves many hedges unlabeled.

Key points

  • Training data: Wikipedia dumps from 2006–2008, restricted to articles containing the string {{weasel. These yielded 168,923 unique sentences with only 437 weasel tags.
  • Evaluation sets:
    • Development: the July 14, 2008 dump, as a balanced 500-sentence set.
    • Test: the March 6, 2009 dump, with 70,437 sentences and 328 tags, also reduced to a balanced 500-sentence set.
    • Manual gold standard: four annotators reached Cohen's ΞΊ = 0.65; the final set has 246 sentences.
  • wpw method: each word in the 5-gram preceding a tag gets a score, Score(w) = RelF(w) + AvgDist(w). A sentence's score is the tanh-normalized sum of its word scores, compared against a threshold Οƒ.
  • asp method: text is POS-tagged with the TnT tagger, and finite-state automata detect three pattern types: numerically underspecified subjects, passive constructions, and adverbs. Only the head of each detected pattern is scored.
  • Results:
Datawpwasp
Development (BEP)0.690.70
Test (F-score)0.70 (P 0.55, R 0.98)0.68
Manual gold standard (F-score)0.590.69 (P 0.61, R 0.78)

Our summary of the paper, in our own words β€” not its text. The work belongs to its authors; read it at the links above.

Fields: Linguistics & languages

Topics: Part-of-speech tagging

Cited by

35 papers of our library cite it (of the 140673 whose bibliographies we read); 3 of them have a page here, 32 not yet.

Cite

The paper itself, not this page. BibTeX from the library's record of the work.

@inproceedings{ganter2009finding,
  title = {{Finding Hedges by Chasing Weasels: Hedge Detection Using Wikipedia Tags and Shallow Linguistic Features}},
  author = {Ganter, Viola and Strube, Michael},
  booktitle = {Proceedings of the ACL-IJCNLP 2009 Conference Short Papers},
  year = {2009},
  url = {https://aclanthology.org/P09-2044/}
}

References

10 references, 7 of them with a card of ours. β†— leads to the work, Β«findΒ» searches for one we have no id for yet.

  1. Bachenko, Joan, Eileen Fitzpatrick & Michael Schonwetter (2008). Verification and implementation of language-based deception indicators in civil and criminal narratives. In Proceedings of the 22nd International Conference on Computational Linguistics, Manchester, U.K., 18–22 August 2008, pp. 41–48. find
  2. Brants, Thorsten (2000). TnT – A statistical Part-of-Speech tagger. In Proceedings of the 6th Conference on Applied Natural Language Processing, Seattle, Wash., 29 April – 4 May 2000, pp. 224–231. β†— our card
  3. Carletta, Jean (1996). Assessing agreement on classification tasks: The kappa statistic. Computational Linguistics, 22(2):249–254. β†— our card
  4. Hyland, Ken (1998). Hedging in scientific research articles. Amsterdam, The Netherlands: John Benjamins. β†— our card
  5. Lakoff, George (1973). Hedges: A study in meaning criteria and the logic of fuzzy concepts. Journal of Philosophical Logic, 2:458–508. β†— our card
  6. Light, Marc, Xin Ying Qiu & Padmini Srinivasan (2004). The language of Bioscience: Facts, speculations, and statements in between. In Proceedings of the HLT-NAACL 2004 Workshop: Biolink 2004, Linking Biological Literature, Ontologies and Databases, Boston, Mass., 6 May 2004, pp. 17–24. find
  7. Medlock, Ben & Ted Briscoe (2007). Weakly supervised learning for hedge classification in scientific literature. In Proceedings of the 45th Annual Meeting of the Association for Computational Linguistics, Prague, Czech Republic, 23–30 June 2007, pp. 992–999. find our card
  8. Pang, Bo & Lillian Lee (2008). Opinion mining and sentiment analysis. Foundations and Trends in Information Retrieval, 2(1-2):1–135. β†—
  9. Riloff, Ellen, Janyce Wiebe & Theresa Wilson (2003). Learning subjective nouns using extraction pattern bootstrapping. In Proceedings of the 7th Conference on Computational Natural Language Learning, Edmonton, Alberta, Canada, 31 May – 1 June 2003, pp. 25–32. find our card
  10. Szarvas, György (2008). Hedge classification in biomedical texts with a weakly supervised selection of keywords. In Proceedings of the 46th Annual Meeting of the Association for Computational Linguistics: Human Language Technologies, Columbus, Ohio, 15–20 June 2008, pp. 281– 289. find our card