Elamur Research⌕
☰
home / papers / 2010 / A High-Precision Approach to Detecting Hedges and their Scopes

A High-Precision Approach to Detecting Hedges and their Scopes

Paper · 2010 · Halil Kilicoglu and Sabine Bergler

Summary

Kilicoglu and Bergler take their earlier biomedical speculation-detection system and enter it in the hedge detection shared task held at CoNLL-2010. The task has two parts: flagging uncertain sentences (Task 1B for biology, Task 1W for Wikipedia) and marking the textual scope of each hedge cue (Task 2). Their main aim was to test portability, so they used the training data only to tune an existing rule-based system and did not train a new model on it.

The method has three parts:

  • Cue dictionary. Hedging cues come from Hyland's inventory, expanded with WordNet and UMLS SPECIALIST nominalizations. Each cue is categorized and weighted.
  • Syntactic adjustment. Weights are raised or lowered depending on constructions found by the Stanford Lexicalized Parser.
  • Wikipedia adaptation. Adding vagueness quantifiers let the same approach catch weasel words.

For scope, they kept their older dependency heuristics and added two things: the phrase from the constituent tree that dominates the cue, and left/right expansion rules.

The system reached the highest precision on the biological tasks and placed fourth of fifteen on scope resolution. This suggests that a precision-first, linguistically grounded design can carry over to new genres.

Key points

  • Cue weights. Weights run on a 1–5 scale, and a sentence counts as uncertain when its summed cue weights reach a threshold of 4. For example, may = 5 and apparently = 2.
  • Syntactic modifiers. An epistemic verb with a that-complement gains +2. The same verb with no complement loses 1. Whether- and if-complements act as cues by themselves.
  • Task 1B tuning. Cue categories never annotated in the training data were removed: approximative adverbs and adjectives, and verbs of effort. This left 147 cues. predict was lowered from 4 to 3 because its annotation was inconsistent.
  • Task 1W changes.
    • Added vagueness quantifiers (some, several, many, various).
    • Kept only might among the modals.
    • Restored approximative adverbs with higher weights.
    • Dropped tendency verbs and nouns.
  • Scope ablation (Table 3). The old event-based baseline scored only F 2.91, showing that its definition of scope does not match BioScope. Adding parse-tree phrases and left/right expansion raised the full system to P 62.47 / R 49.47 / F 55.21.

Interesting ideas

  • I1. Cues carry graded weights, and syntax can push a cue over the decision threshold or keep it below. An item becomes decisive only in the right construction, as when apparent hedges only with a clausal complement.
  • I2. Certainty words such as know are listed as "un-hedgers", which turn into hedges when negated, so polarity flips a cue's meaning.
  • I3. One cue lexicon is adapted to a new genre by switching cue categories on or off and reweighting them, with no retraining, which keeps every change readable.
  • I4. The scope ablation starts from the old baseline, so it shows plainly that a mismatched definition of scope, not the method, caused near-zero scores.
  • I5. A crude default sets the scope to the phrase dominating the cue. Targeted left/right expansion rules then fix the cases where that default fails.

For Mova

How these ideas could serve Mova, our small open language model — its files, stages and tests:

  • I1 (frames): speech-act and factuality marking in prag could score epistemic verbs together with their complement type from the UD tree, instead of firing on the lemma alone.
  • I2 (world): when a state is asserted, a negated certainty verb could mark the fact as doubtful in the world of the text instead of dropping it.
  • I3 (frames): genre profiles (fables vs. school texts) could be small, explainable weight overrides on a shared hedge/modality lexicon.
  • I4 (learn): when adopting a new treebank or annotation dialect, first run the old pipeline on it as a control to measure how much of the error is definitional.
  • I5 (tree): span-like outputs over UD trees, such as negation or modal scope, could start from the cue head's subtree and add a few explicit expansion rules, each of which can be logged.

Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.

 Institutions

🎓Concordia UniversityMontreal · Canada5 works · 303 citations

📚 Shelves 4 Computer Science · 9,086 works Artificial Intelligence · 8,889 works Arts and Humanities · 9,296 works Language and Linguistics · 9,206 works

Cite

Cite as Halil Kilicoglu and Sabine Bergler. 2010. A High-Precision Approach to Detecting Hedges and their Scopes. Proceedings of the Fourteenth Conference on Computational Natural Language Learning – Shared Task.

The paper itself, not this page. BibTeX from the library's record of the work.

@inproceedings{kilicoglu2010high,
  title = {{A High-Precision Approach to Detecting Hedges and their Scopes}},
  author = {Halil Kilicoglu and Sabine Bergler},
  booktitle = {Proceedings of the Fourteenth Conference on Computational Natural Language Learning – Shared Task},
  year = {2010},
  url = {https://aclanthology.org/W10-3010/}
}

References

26 references, 9 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

  1. Marie-Catherine deMarneffe, Bill MacCartney, and Christopher D. Manning. 2006. Generating typed dependency parses from phrase structure parses. In Proceedings of the 5th International Conference on Language Resources and Evaluation, pages 449– 454. our card
  2. Richárd Farkas, Veronika Vincze, György Móra, János Csirik, and György Szarvas. 2010. The CoNLL-2010 Shared Task: Learning to Detect Hedges and their Scope in Natural Language Text. In Proceedings of the Fourteenth Conference on Computational Natural Language Learning (CoNLL-2010): Shared Task, pages 1–12, Uppsala, Sweden, July. Association for Computational Linguistics. our card
  3. Christiane Fellbaum. 1998. WordNet: an electronic lexical database. MIT Press, Cambridge, MA. find
  4. Viola Ganter and Michael Strube. 2009. Finding Hedges by Chasing Weasels: Hedge Detection Using Wikipedia Tags and Shallow Linguistic Features. In Proceedings of the ACL-IJCNLP 2009 Conference Short Papers, pages 173–176. our card
  5. Ken Hyland. 1998. Hedging in scientific research articles. John Benjamins B.V., Amsterdam, Netherlands. ↗ our card
  6. Halil Kilicoglu and Sabine Bergler. 2008. Recognizing speculative language in biomedical research articles: a linguistically motivated perspective. BMC Bioinformatics, 9 Suppl 11:s10. find
  7. Halil Kilicoglu and Sabine Bergler. 2009. Syntactic dependency based heuristics for biological event extraction. In Proceedings of Natural Language Processing in Biomedicine (BioNLP) NAACL 2009 Workshop, pages 119–127. find
  8. Jin-Dong Kim, Tomoko Ohta, and Jun’ichi Tsujii. 2008. Corpus annotation for mining biomedical events from literature. BMC Bioinformatics, 9:10. ↗
  9. Jin-Dong Kim, Tomoko Ohta, Sampo Pyysalo, Yoshinobu Kano, and Jun’ichi Tsujii. 2009. Overview of BioNLP’09 Shared Task on Event Extraction. In Proceedings of Natural Language Processing in Biomedicine (BioNLP) NAACL 2009 Workshop, pages 1–9. our card
  10. Dan Klein and Christopher D Manning. 2003. Accurate unlexicalized parsing. In Proceedings of the 41th Meeting of the Association for Computational Linguistics, pages 423–430. our card
  11. Shalom Lappin. 2000. An intensional parametric semantics for vague quantifiers. Linguistics and Philosophy, 23(6):599–620. ↗
  12. Marc Light, Xin Y. Qiu, and Padmini Srinivasan. 2004. The language of bioscience: facts, speculations, and statements in between. In BioLINK 2004: Linking Biological Literature, Ontologies and Databases, pages 17–24. find
  13. Alexa T. McCray, Suresh Srinivasan, and Allen C. Browne. 1994. Lexical methods for managing variation in biomedical terminologies. In Proceedings of the 18th Annual Symposium on Computer Applications in Medical Care, pages 235–239. find
  14. Ben Medlock and Ted Briscoe. 2007. Weakly supervised learning for hedge classification in scientific literature. In Proceedings of the 45th Meeting of the Association for Computational Linguistics, pages 992–999. our card
  15. Roser Morante and Walter Daelemans. 2009. Learning the scope of hedge cues in biomedical texts. In Proceedings of the BioNLP 2009 Workshop, pages 28–36. our card
  16. Arzucan Özgür and Dragomir R. Radev. 2009. Detecting speculations and their scopes in scientific text. In Proceedings of the 2009 Conference on Empirical Methods in Natural Language Processing, pages 1398–1407. find
  17. Bo Pang and Lillian Lee. 2008. Sentiment Analysis and Opinion Mining. Now Publishers Inc, Boston, MA. find
  18. James Pustejovsky, Robert Knippen, Jessica Littman, and Roser Saurı́. 2005. Temporal and event information in natural language text. Language Resources and Evaluation, 39(2):123–164. ↗
  19. Victoria L. Rubin, Elizabeth D. Liddy, and Noriko Kando. 2005. Certainty identification in texts: Categorization model and manual tagging results. In James G. Shanahan, Yan Qu, and Janyce Wiebe, editors, Computing Attitude and Affect in Text: Theories and Applications, volume 20, pages 61–76. Springer Netherlands, Dordrecht. ↗
  20. Roser Saurı́ and James Pustejovsky. 2009. FactBank: a corpus annotated with event factuality. Language Resources and Evaluation, 43(3):227–268. find
  21. Roser Saurı́, Marc Verhagen, and James Pustejovsky. 2006. Annotating and recognizing event modality in text. In Proceedings of 19th International FLAIRS Conference. find
  22. Roser Saurı́. 2008. A Factuality Profiler for Eventualities in Text. Ph.D. thesis, Brandeis University. find
  23. György Szarvas. 2008. Hedge classification in biomedical texts with a weakly supervised selection of keywords. In Proceedings of the 46th Meeting of the Association for Computational Linguistics, pages 281–289. our card
  24. Paul Thompson, Giulia Venturi, John McNaught, Simonetta Montemagni, and Sophia Ananiadou. 2008. Categorising modality in biomedical texts. In Proceedings of LREC 2008 Workshop on Building and Evaluating Resources for Biomedical Text Mining. find
  25. Veronika Vincze, György Szarvas, Richárd Farkas, György Móra, and János Csirik. 2008. The Bio-Scope corpus: biomedical texts annotated for uncertainty, negation and their scopes. BMC Bioinformatics, 9 Suppl 11:S9. find
  26. Janyce Wiebe, Theresa Wilson, and Claire Cardie. 2005. Annotating expressions of opinions and emotions in language. Language Resources and Evaluation, 39(2):165–210. ↗

Updated

↑ back to top

This page in Markdown → https://md.research.elamur.ai/free/en/papers/kilicoglu-2010-high-precision-approach-detecting-hedges/