Elamur Research⌕
☰
home / papers / 2012 / Improving Speculative Language Detection using Linguistic Knowledge

Improving Speculative Language Detection using Linguistic Knowledge

Paper · 2012 · Guillermo Moncecchi, Jean-Luc Minel and Dina Wonsever

Summary

Scientific writers often soften claims with words like "may" or "suggest". Information extraction systems need to notice this so that tentative statements are not stored as established facts. The authors split the task into two steps. First they find hedge cues, then they decide which part of the sentence each cue affects (its scope). Both steps are treated as token-level sequence labelling, trained and tested on the Abstract part of the Bioscope corpus. Their main contribution is a working method rather than a new model. They train a classifier, inspect its mistakes on a held-out split, and add linguistic knowledge to fix those mistakes. That knowledge takes the form of extra attributes, rules that suggest a class to the learner, and post-processing driven by the constituent tree. The fixes raised scope F1 by about eight points on unseen data. On the CoNLL-2010 Shared Task data the system was competitive with the top entries. The work shows that expert rules and supervised learning can be combined step by step, with a separate test split guarding against overfitting.

Key points

  • Data split: only the Bioscope Abstract sub-corpus was used. 20% was held back for final evaluation, and 20% of the remainder served as a held-out set for error analysis.
  • Labelling schemes: hedge cues use B/I/O labels. Scope boundaries use first/last-token classes (F, L). Extra classes X and Y mark scopes that coincide with the parent or grandparent constituent of the cue.
  • Hedge cue results (evaluation split): F1 rose from 86.8 to 91.3, as recall climbed from 78.0 to 88.6 while precision dipped slightly from 97.9 to 94.1.
  • Scope results (evaluation split):
    • With gold cues, accuracy improved from 74.0 to 84.7.
    • With predicted cues, F1 improved from 70.0 to 77.9.
    • On the held-out split the gains were larger, reaching 79.1 F1.
  • CoNLL-2010 Task 2:
    • Cue detection reached 79.9 F1, above the third-ranked system; the best was 81.3.
    • Scope detection reached 54.7 F1, above the fifth-ranked system.
    • This is about five points below the 59.4 later reported by Velldal et al. (2012).

Interesting ideas

  • I1. Improve the model in rounds: study errors only on a held-out split, keep a final split untouched, and re-score earlier configurations on it to check that the error-driven tweaks did not overfit.
  • I2. Feed expert rules to the learner as a suggested class, so the model can learn when to trust each rule, instead of letting the rules override its predictions.
  • I3. Describe a candidate scope through the tree ancestors of the cue (parent, grandparent, and higher), and let output classes say which ancestor the span ends with. This turns span finding into choosing a constituent.
  • I4. When the statistical output is incomplete (a missing or duplicated boundary), fall back to the nearest enclosing clause or phrase, chosen by the cue's part of speech. Passive verb cues are pushed up to the full clause.
  • I5. Report scope accuracy separately with gold cues and with predicted cues, so errors from the first stage are not blamed on the second.

For Mova

How these ideas could serve Mova, our small open language model — its files, stages and tests:

  • I1 (learn): Self-induced repair rules could be developed on one slice and accepted only if they also help on a sealed slice, which would act as a natural negative control.
  • I2 (tokens): Hand-written tagging or lemma heuristics in crate en could become "rule says X" attributes for the perceptron, keeping their contribution visible and explainable.
  • I3 (frames): Scope of modals, negation, or hedging verbs over a UD subtree could be chosen among the cue's head ancestors, so each frame is marked as asserted or only speculated.
  • I4 (frames): A deterministic tree-based fallback gives an explainable default whenever the learned scope fails, rather than dropping the hedge.
  • I5 (answer): Evaluating each reader stage on oracle input versus pipeline input would show which step of text → tree → frames causes wrong answers.

Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.

 Institutions

🎓Universidad de la República de UruguayMontevideo · Uruguay4 works · 429 citations🎓Université Paris NanterreNanterre · France5 works · 197 citations

📚 Shelves 4The field of science this work belongs to (OpenAlex fields).Computer Science · 9,086 worksArtificial Intelligence · 8,889 worksArts and Humanities · 9,296 worksLanguage and Linguistics · 9,206 works

Cite

Cite as Guillermo Moncecchi, Jean-Luc Minel and Dina Wonsever. 2012. Improving Speculative Language Detection using Linguistic Knowledge. Proceedings of the Workshop on Extra-Propositional Aspects of Meaning in Computational Linguistics.

The paper itself, not this page. BibTeX from the library's record of the work.

@inproceedings{moncecchi2012improving,
  title = {{Improving Speculative Language Detection using Linguistic Knowledge}},
  author = {Guillermo Moncecchi, Jean-Luc Minel and Dina Wonsever},
  booktitle = {Proceedings of the Workshop on Extra-Propositional Aspects of Meaning in Computational Linguistics},
  year = {2012},
  url = {https://aclanthology.org/W12-3805/}
}

References

26 references, 9 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

  1. S. Ananiadou, D. Kell, and J. Tsuj. 2006. Text mining and its potential applications in systems biology. Trends in Biotechnology, 24(12):571–579, December. ↗
  2. Richárd Farkas, Veronika Vincze, György Móra, János Csirik, and György Szarvas. 2010a. The CoNLL-2010 shared task: Learning to detect hedges and their scope in natural language text. In Proceedings of the Fourteenth Conference on Computational Natural Language Learning, pages 1–12, Uppsala, Sweden, July. Association for Computational Linguistics.
  3. Richárd Farkas, Veronika Vincze, György Szarvas, György Móra, and János Csirik, editors. 2010b. Proceedings of the Fourteenth Conference on Computational Natural Language Learning. Association for Computational Linguistics, Uppsala, Sweden, July. find
  4. Ken Hyland. 1995. The author in the text: Hedging scientific writing. Hongkong Papers in Linguistics and Language Teaching, 18:33–42. find
  5. Ken Hyland. 1996a. Talking to the academy: Forms of hedging in science research articles. Written Communication, 13(2):251–281. find
  6. Ken Hyland. 1996b. Writing without conviction? Hedging in science research articles. Applied Linguistics, 17(4):433–454, December. find
  7. Ken Hyland. 2005. Metadiscourse: Exploring Interaction in Writing. Continuum Discourse. Continuum. find
  8. Halil Kilicoglu and Sabine Bergler. 2010. A high-precision approach to detecting hedges and their scopes. In Proceedings of the Fourteenth Conference on Computational Natural Language Learning, pages 70–77, Uppsala, Sweden, July. Association for Computational Linguistics.
  9. Dan Klein and Christopher D. Manning. 2003. Accurate unlexicalized parsing. In ACL ’03: Proceedings of the 41st Annual Meeting on Association for Computational Linguistics, pages 423–430, Morristown, NJ, USA. Association for Computational Linguistics.
  10. John Lafferty, Andrew McCallum, and Fernando Pereira. 2001. Conditional random fields: Probabilistic models for segmenting and labeling sequence data. In Proceedings of ICML-01, pages 282–289. find
  11. George Lakoff. 1973. Hedges: A study in meaning criteria and the logic of fuzzy concepts. Journal of Philosophical Logic, 2(4):458–508, October. ↗
  12. Marc Light, Xin Y. Qiu, and Padmini Srinivasan. 2004. The language of bioscience: Facts, speculations, and statements in between. In Lynette Hirschman and James Pustejovsky, editors, HLT-NAACL 2004 Workshop: BioLINK 2004, Linking Biological Literature, Ontologies and Databases, pages 17–24, Boston, Massachusetts, USA, May. Association for Computational Linguistics. find
  13. Ben Medlock and Ted Briscoe. 2007. Weakly supervised learning for hedge classification in scientific literature. In Proceedings of the 45th Annual Meeting of the Association of Computational Linguistics.
  14. Guillermo Moncecchi, Jean-Luc Minel, and Dina Wonsever. 2010. Enriching the bioscope corpus with lexical and syntactic information. In Workshop in Natural Language Processing and Web-based Tecnhologies 2010, pages 137–146, November. find
  15. Roser Morante and Walter Daelemans. 2009. Learning the scope of hedge cues in biomedical texts. In Proceedings of the BioNLP 2009 Workshop, pages 28– 36, Boulder, Colorado, June. Association for Computational Linguistics.
  16. Roser Morante and Caroline Sporleder. 2012. Modality and negation: An introduction to the special issue. Computational Linguistics, pages 1–72, February. ↗
  17. Roser Morante, Anthony Liekens, and Walter Daelemans. 2008. Learning the scope of negation in biomedical texts. In EMNLP ’08: Proceedings of the Conference on Empirical Methods in Natural Language Processing, pages 715–724, Morristown, NJ, USA. Association for Computational Linguistics. find
  18. Roser Morante, Vincent Van Asch, and Walter Daelemans. 2010. Memory-based resolution of in-sentence scopes of hedge cues. In Proceedings of the Fourteenth Conference on Computational Natural Language Learning, pages 40–47, Uppsala, Sweden, July. Association for Computational Linguistics.
  19. R. F. Palmer. 2001. Mood and Modality. Cambridge Textbooks in Linguistics. Cambridge University Press, New York. ↗
  20. Aiala Rosá. 2011. Identificación de opiniones de diferentes fuentes en textos en español. Ph.D. thesis, Universidad de la República (Uruguay), Université Paris Ouest (France), September. find
  21. Buzhou Tang, Xiaolong Wang, Xuan Wang, Bo Yuan, and Shixi Fan. 2010. A cascade method for detecting hedges and their scope in natural language text. In Proceedings of the Fourteenth Conference on Computational Natural Language Learning, pages 13–17, Uppsala, Sweden, July. Association for Computational Linguistics.
  22. Yoshimasa Tsuruoka, Yuka Tateishi, Jin-Dong Kim, Tomoko Ohta, John McNaught, Sophia Ananiadou, and Jun’ichi Tsujii. 2005. Developing a robust Part-of-Speech tagger for biomedical text. In Panayiotis Bozanis and Elias N. Houstis, editors, Advances in Informatics, volume 3746, chapter 36, pages 382–392. Springer Berlin Heidelberg, Berlin, Heidelberg. find
  23. Erik Velldal, Lilja Øvrelid, and Stephan Oepen. 2010. Resolving speculation: Maxent cue classification and dependency-based scope rules. In Proceedings of the Fourteenth Conference on Computational Natural Language Learning, pages 48–55, Uppsala, Sweden, July. Association for Computational Linguistics. find
  24. Erik Velldal, Lilja Øvrelid, Jonathon Read, and Stephan Oepen. 2012. Speculation and negation: Rules, rankers, and the role of syntax. Computational Linguistics, pages 1–64, February. ↗
  25. Veronika Vincze, Gyorgy Szarvas, Richard Farkas, Gyorgy Mora, and Janos Csirik. 2008. The bioscope corpus: biomedical texts annotated for uncertainty, negation and their scopes. BMC Bioinformatics, 9(Suppl 11):S9+. ↗
  26. Kail von Fintel, 2006. Modality and Language. MacMillan Reference USA. find

Updated

↑ back to top

This page in Markdown → https://md.research.elamur.ai/free/en/papers/moncecchi-2012-improving-speculative-language-detection-linguistic/