Elamur Research
☰
home / papers / 2007 / Weakly Supervised Learning for Hedge Classification in Scientific Literature

Weakly Supervised Learning for Hedge Classification in Scientific Literature

Written for the Research library (claude-opus-5-5) from the paper text, in our own words: the license "CC BY-NC-SA 3.0 (ACL Anthology, before 2016)" does not allow redistributing the text itself. Original — aclanthology.org/P07-1125.pdf

Summary

Medlock and Briscoe study how to automatically detect hedging, meaning speculative rather than firmly asserted statements, in biomedical writing. They frame it as a binary sentence-level task: is a sentence spec or nspec? Such a filter matters for tasks like extracting gene interactions, where a tentative claim is much weaker evidence than a confident one. The authors extend the annotation guidelines of Light et al. (2004) with explicit rules for borderline cases and show that independent annotators agree well under them. Their data come from Drosophila functional genomics papers processed with the RASP toolkit. Because only a small amount of hand-labelled seed data is available, they derive a probabilistic weakly supervised model that works with bag-of-words features. It iteratively grows its training data from a large unlabelled pool. This model clearly beats a keyword-matching baseline, and an SVM committee self-training approach serves as a further comparison. The work shows that hedge detection is achievable with minimal supervision and releases a public dataset. The authors intend to use it in tools that help curators build gene databases.

Key points

  • Corpus: 5579 full-text Drosophila melanogaster papers. Six annotated papers form the test set, with 380 spec and 1157 nspec sentences. A further 300,000 unlabelled sentences serve as training material; each has at least 10 words and a main verb.
  • Annotator agreement (an author and an independent genetics expert):
    • Initial labels: F1rel 0.8293 and κ 0.9336.
    • After fixing negligent misses: F1rel 0.9652 and κ 0.9848.
  • Setup: an nspec seed set of 7541 sentences was manually cleaned down from 8830. Smoothing was set to α = 5 and up to m = 5 hedge cue features were assumed per sentence. The SVM experiments used SVM light with a linear kernel.
  • Results after 150 iterations:
    • The substring baseline of Light et al. reaches 0.60 BEP.
    • Classifiers built on the probabilistic learner reach about 0.76 BEP.
    • All weakly supervised variants significantly outperform the baseline (sign test, p < 0.01).
  • Error analysis: the models miss assertive statements of knowledge paucity. They also confuse observed non-universal behaviour with genuine speculation. The authors suggest richer representations, such as handling negation, as a possible fix.

Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.

Fields: Linguistics & languages

Cited by

70 papers of our library cite it (of the 140673 whose bibliographies we read); 4 of them have a page here, 66 not yet.

Cite

The paper itself, not this page. BibTeX from the library's record of the work.

@inproceedings{medlock2007weakly,
  title = {{Weakly Supervised Learning for Hedge Classification in Scientific Literature}},
  author = {Medlock, Ben and Briscoe, Ted},
  booktitle = {Proceedings of the 45th Annual Meeting of the Association of Computational Linguistics},
  year = {2007},
  url = {https://aclanthology.org/P07-1125/}
}

References

19 references, 2 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

  1. Ron Artstein and Massimo Poesio. 2005. Kappa 3 = alpha (or beta). Technical report, University of Essex Department of Computer Science. find
  2. Michele Banko and Eric Brill. 2001. Scaling to very very large corpora for natural language disambiguation. In Meeting of the Association for Computational Linguistics, pages 26–33. find
  3. Avrim Blum and Tom Mitchell. 1998. Combining labelled and unlabelled data with co-training. In Proceedings of COLT’ 98, pages 92–100, New York, NY, USA. ACM Press. find
  4. Jinxiu Chen, Donghong Ji, Chew L. Tan, and Zhengyu Niu. 2006. Relation extraction using label propagation based semi-supervised learning. In Proceedings of ACL’06, pages 129–136. find
  5. M. Collins and Y. Singer. 1999. Unsupervised models for named entity classification. In Proceedings of the Joint SIGDAT Conference on Empirical Methods in NLP and Very Large Corpora. find
  6. George Hripcsak and Adam Rothschild. 2004. Agreement, the f-measure, and reliability in information retrieval. J Am Med Inform Assoc., 12(3):296–298. find
  7. K. Hyland. 1994. Hedging in academic writing and eap textbooks. English for Specific Purposes, 13:239–256. find
  8. Thorsten Joachims. 1999. Making large-scale support vector machine learning practical. In A. Smola B. Schölkopf, C. Burges, editor, Advances in Kernel Methods: Support Vector Machines. MIT Press, Cambridge, MA. find
  9. M. Light, X.Y. Qiu, and P. Srinivasan. 2004. The language of bioscience: Facts, speculations, and statements in between. In Proceedings of BioLink 2004 Workshop on Linking Biological Literature, Ontologies and Databases: Tools for Users, Boston, May 2004. find
  10. David McClosky, Eugene Charniak, and Mark Johnson. 2006. Effective self-training for parsing. In HLT-NAACL. find our card
  11. Vincent Ng and Claire Cardie. 2003. Weakly supervised natural language learning without redundant views. In Proceedings of NAACL ’03, pages 94–101, Morristown, NJ, USA. find
  12. K. Nigam and R. Ghani. 2000. Understanding the behavior of co-training. In Proceedings of KDD-2000 Workshop on Text Mining. find
  13. Kamal Nigam, Andrew K. McCallum, Sebastian Thrun, and Tom M. Mitchell. 2000. Text classification from labeled and unlabeled documents using EM. Machine Learning, 39(2/3):103–134. find
  14. Ellen Riloff, Janyce Wiebe, and Theresa Wilson. 2003. Learning subjective nouns using extraction pattern bootstrapping. In Seventh Conference on Natural Language Learning (CoNLL-03). ACL SIGNLL., pages 25–32. find
  15. Ben Wellner. 2005. Weakly supervised learning methods for improving the quality of gene name normalization data. In Proceedings of the ACL-ISMB Workshop on Linking Biological Literature, Ontologies and Databases, pages 1–8, Detroit, June. Association for Computational Linguistics. find
  16. Janyce Wiebe, Theresa Wilson, Rebecca Bruce, Matthew Bell, and Melanie Martin. 2004. Learning subjective language. Comput. Linguist., 30(3):277–308. find
  17. Roman Yangarber. 2003. Counter-training in discovery of semantic patterns. In Proceedings of ACL’03, pages 343–350, Morristown, NJ, USA. find
  18. David Yarowsky. 1995. Unsupervised word sense disambiguation rivaling supervised methods. In Proceedings of ACL’95, pages 189–196, Morristown, NJ, USA. ACL. find our card
  19. Zhu Zhang. 2004. Weakly-supervised relation classification for information extraction. In CIKM ’04: Proceedings of the thirteenth ACM international conference on Information and knowledge management, pages 581–588, New York, NY, USA. ACM Press. find