Elamur Research⌕
☰
home / papers / 2010 / Hedge Detection and Scope Finding by Sequence Labeling with Procedural Feature Selection

Hedge Detection and Scope Finding by Sequence Labeling with Procedural Feature Selection

Paper · 2010 · Shaodian Zhang, Hai Zhao, Guodong Zhou et al.

Summary

The paper describes a system built for the CoNLL-2010 shared task. The task has two parts: detecting speculative language (hedges) and resolving the scope of each hedge cue. Data came from the BioScope corpus (biomedical abstracts and full texts) and from Wikipedia "weasel" annotations. Both tasks are cast as token-level sequence labeling with Maximum Entropy Markov Models.

  • Detection: words are tagged as cue or non-cue. A sentence counts as uncertain if any word carries a cue tag.
  • Scope finding: the system first labels cues, then labels scope boundaries.
  • Sentences with several cues: each one is duplicated once per cue. An "Indicator" feature marks words already covered by earlier scopes, which discourages crossing scopes.
  • Feature selection: about 200 templates per task, many built from dependency-tree relations, were reduced by greedy selection. This was run separately for each sub-task and for each corpus, because the expert annotation and the crowd annotation behave differently.

In the closed track, detection F-scores were 0.8363 on BioScope and 0.5618 on Wikipedia, the latter ranking near the top. Scope F was about 0.44. Further feature selection after the deadline gave better results.

Key points

  • Hedge detection reached F 0.8363 on BioScope and F 0.5618 on Wikipedia. Scope resolution scored F 0.4425 (closed) and 0.4441 (open).
  • About a quarter of hedged sentences contain more than one cue. For example, the BioScope abstracts have 498 multi-hedge sentences out of 11,871.
  • Scope labels are Begin, Middle, End and out-of-scope, but only Begin and End count in evaluation.
  • Many features come from dependency parses produced by the Zhao et al. (2009a) parser:
    • paths between words and their shared and non-shared parts;
    • children sets;
    • support verbs, nouns and prepositions.
  • Strings collected along paths or children are joined in one of three ways: in order (seq), as a sorted set (bag), or with adjacent repeats removed (noDup).
  • The authors blame the modest scope results on two things: feature selection that was not run to completion, and mixing abstracts with full texts in training when the test set held only full texts.

Interesting ideas

  • I1. Optimizing a sentence-level decision and optimizing a token-level intermediate step can call for different feature sets, so the authors tune each step separately rather than reusing one model.
  • I2. Copying a multi-cue sentence once per cue turns one hard joint labeling problem into several single-target problems, whose results are then merged.
  • I3. A soft feature marking tokens already inside earlier scopes pushes later predictions to respect tree-like nesting without imposing a hard constraint.
  • I4. Features are built from dependency paths and grouped as ordered, set-like or de-duplicated sequences, which captures words that are far apart in the sentence but close in the tree.
  • I5. Expert-annotated and crowd-annotated corpora get separate feature selection, and training on a mixture of genres hurt performance on full texts.

For Mova

How these ideas could serve Mova, our small open language model — its files, stages and tests:

  • I1 (learn): when a component serves both as a final output and as an input to later stages, evaluate and tune it against each use separately.
  • I2 (frames): role labeling for sentences with several predicates could run one pass per predicate, which keeps frames from bleeding into each other.
  • I3 (frames): when assigning spans for negation or modality, earlier decisions could be passed on as features, so that partially overlapping scopes become unlikely and any remaining overlap can be flagged as an explainable error.
  • I4 (frames): speculation and modality detectors could read their evidence directly from the UD tree, which keeps the decision traceable to specific arcs.
  • I5 (learn): track which source and genre each piece of silver or treebank data comes from, and evaluate per domain before pooling.

Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.

 Institutions

🎓Shanghai Jiao Tong UniversityShanghai · China149 works · 16,161 citations🎓Soochow UniversitySuzhou · China63 works · 4,507 citations

 Fields, topics and concepts

📚 Shelves 4The field of science this work belongs to (OpenAlex fields).Computer Science · 16,667 worksArtificial Intelligence · 16,194 worksArts and Humanities · 16,039 worksLanguage and Linguistics · 15,891 works

Cite

Cite as Shaodian Zhang, Hai Zhao, Guodong Zhou and Bao-Liang Lu. 2010. Hedge Detection and Scope Finding by Sequence Labeling with Procedural Feature Selection. Proceedings of the Fourteenth Conference on Computational Natural Language Learning – Shared Task.

The paper itself, not this page. BibTeX from the library's record of the work.

@inproceedings{zhang2010hedge,
  title = {{Hedge Detection and Scope Finding by Sequence Labeling with Procedural Feature Selection}},
  author = {Shaodian Zhang, Hai Zhao, Guodong Zhou and Bao-Liang Lu},
  booktitle = {Proceedings of the Fourteenth Conference on Computational Natural Language Learning – Shared Task},
  year = {2010},
  url = {https://aclanthology.org/W10-3013/}
}

References

23 references, 12 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

  1. Michael Collins. 1999. Head-Driven Statistical Models for Natural Language Parsing. Ph.D. thesis, University of Pennsylvania. find
  2. Richárd Farkas, Veronika Vincze, György Móra, János Csirik, and György Szarvas. 2010. The CoNLL-2010 Shared Task: Learning to Detect Hedges and their Scope in Natural Language Text. In Proceedings of the Fourteenth Conference on Computational Natural Language Learning (CoNLL-2010): Shared Task, pages 1–12, Uppsala, Sweden, July. Association for Computational Linguistics.
  3. Viola Ganter and Michael Strube. 2009. Finding hedges by chasing weasels: Hedge detection using Wikipedia tags and shallow linguistic features. In Proceedings of the ACL-IJCNLP 2009 Conference Short Papers, pages 173–176, Suntec, Singapore, 4, August.
  4. Ken Hyland. 1996. Writing without conviction: Hedging in science research articles. Applied Linguistics, 17:433–54. ↗
  5. Zheng Ping Jiang and Hwee Tou Ng. 2006. Semantic role labeling of NomBank: A maximum entropy approach. In Proceedings of the EMNLP-2006, pages 138–145, Sydney, Australia. find
  6. Halil Kilicoglu and Sabine Bergler. 2008. Recognizing speculative language in biomedical research articles: a linguistically motivated perspective. BMC Bioinformatics, 9.
  7. Marc Light, Xin Ying Qiu, and Padimini Srinivasan. 2004. The language of bioscience: Facts, speculations, and statements in between. In Proc. of the BioLINK 2004, pages 17–24. find
  8. Chrysanne Di Marco and Robert E. Mercer. 2004. Hedging in scientific articles as a means of classifying citations. In Working Notes of the AAAI Spring Symposium on Exploring Attitude and Affect in Text: Theories and Applications, pages 50–54. find
  9. Andrew McCallum. 2000. Maximum entropy markov models for information extraction and segmentation. In Proceedings of ICML 2000, pages 591–598, Stanford, California. find
  10. Ryan McDonald, Fernando Pereira, Kiril Ribarov, and Jan Hajic. 2005. Non-projective dependency parsing using spanning tree algorithms. In Proceedings of HLT/EMNLP 05, pages 523–530, Vancouver, Canada, October.
  11. Ben Medlock and Ted Briscoe. 2008. Weakly supervised learning for hedge classification in scientific literature. In Proceedings of 45th Annual Meeting of the ACL, pages 992–999, Prague, Czech Republic, June.
  12. Ben Medlock. 2008. Exploring hedge identification in biomedical literature. Journal of Biomedical Informatics, 41:636–654. Dataset BioScope TP 669 441 788 991 Task1-closed Task2-closed Cue-matching Task1-closed Wikipedia Table 8: Official results of our submission for in-domain tasks Roser Morante and Walter Daelemans. 2009. Learning the scope of hedge cues in biomedical texts. In Proceedings of the Workshop on BioNLP, pages 28–36, Boulder, Colorado, June. ↗
  13. Joakim Nivre and Mario Scholz. 2004. Deterministic dependency parsing of English text. In Proceedings of COLING-2004, pages 64–70, Geneva, Switzerland, August 23rd-27th.
  14. Joakim Nivre. 2009. Non-projective dependency parsing in expected linear time. In Proceedings of ACL-IJCNLP 2009, pages 351–359, Suntec, Singapore, 2-7 August.
  15. György Szarvas, Veronika Vincze, Richárd Farkas, and János Csirik. 2008. The BioScope corpus: annotation for negation, uncertainty and their scope in biomedical texts. In Proceedings of BioNLP 2008, pages 38–45, Columbus, Ohio, USA, June.
  16. György Szarvas. 2008. Hedge classification in biomedical texts with a weakly supervised selection of keywords. In Proceedings of ACL-08, pages 281– 289, Columbus, Ohio, USA, June.
  17. Paul Thompson, Giulia Venturi, John McNaught, Simonetta Montemagni, and Sophia Ananiadou. 2008. Categorising modality in biomedical texts. In Proc. of the LREC 2008 Workshop on Building and Evaluating Resources for Biomedical Text Mining, pages 27–34, Marrakech. find
  18. Kristina Toutanova, Aria Haghighi, and Christopher D. Manning. 2005. Joint learning improves semantic role labeling. In Proceedings of ACL-2005, pages 589–596, Ann Arbor, USA. ↗
  19. Nianwen Xue. 2006. Semantic role labeling of nominalized predicates in Chinese. In Proceedings of the Human Language Technology Conference of the NAACL (NAACL-2006), pages 431–438, New York City, USA, June. find
  20. Tong Zhang, Fred Damerau, and David Johnson. 2001. Text chunking using regularized winnow. In Proceedings of the 39th Annual Meeting on Association for Computational Linguistics, pages 539–546, Toulouse, France. find
  21. Hai Zhao, Wenliang Chen, Jun’ichi Kazama, Kiyotaka Uchimoto, and Kentaro Torisawa. 2009a. Multilingual dependency learning: Exploiting rich features for tagging syntactic and semantic dependencies. In Proceedings of CoNLL-2009, June 4-5, Boulder, Colorado, USA. FP 141 519 172 303 FN 121 592 259 1243 precision 0.8259 0.4594 0.8208 0.7658 recall 0.8468 0.4269 0.7526 0.4436 F-score 0.8363 0.4425 0.7853 0.5618 find
  22. Hai Zhao, Wenliang Chen, and Chunyu Kit. 2009b. Semantic dependency parsing of NomBank and PropBank: An efficient integrated approach via a large-scale feature selection. In Proceedings of EMNLP-2009, pages 30–39, Singapore.
  23. Hai Zhao, Wenliang Chen, Chunyu Kit, and Guodong Zhou. 2009c. Multilingual dependency learning: A huge feature engineering method to semantic dependency parsing. In Proceedings of CoNLL-2009, June 4-5, Boulder, Colorado, USA. find

Updated

↑ back to top

This page in Markdown → https://md.research.elamur.ai/free/en/papers/zhang-2010-hedge-detection-scope-finding-sequence/