Elamur Research⌕
☰
home / papers / 2015 / A Compositional Interpretation of Biomedical Event Factuality

A Compositional Interpretation of Biomedical Event Factuality

Paper · 2015 · Halil Kilicoglu, Graciela Rosemblat, Michael Cairelli et al.

Summary

The paper asks whether a biomedical event reported in a MEDLINE abstract is presented as established, possible, or negated. The authors do not train a classifier. They apply a general framework for extra-propositional meaning in which every relation is a predication. Predications can nest inside each other, and each one carries a source and a scalar modality value between 0 and 1. A dictionary sorts embedding predicates into a fine-grained taxonomy (MODAL, RELATIONAL, VALENCE SHIFTER, PROPOSITIONAL) and gives each a prior value. An event's factuality is the product of the values of all operators whose scope covers it. Predications are built from dependency parses plus hand-written transformation rules. The output is then mapped to the certainty and polarity labels of the meta-knowledge corpus, which is built on GENIA events. With almost no use of training data, certainty-level results come close to a supervised state-of-the-art system. Negative polarity is clearly weaker. This suggests that treating negation as simply the bottom of a modal scale is not enough. The work shows that a transparent, rule-based composition can stand up well against machine learning on a semantic task.

Key points

  • Evaluation uses the meta-knowledge corpus (Thompson et al., 2011), which annotates GENIA event corpus events with Certainty Level (L1–L3), Polarity, Knowledge Type, Manner and Source. Only certainty and polarity are targeted here.
  • On Analysis events, certainty accuracy is about 82%, against a 55% majority-class baseline. F1 for L3/L2/L1 is 86.2/73.3/79.6.
  • On all events, F1 is 97.5/66.6/68.1 for L3/L2/L1. The supervised system of Miwa et al. (2012) reaches 97.6/66.5/74.9.
  • Polarity results are mixed. Negative polarity reaches 49.4% F1 against Miwa et al.'s 63.4%. Positive polarity is close, at 97.5% vs 97.7%.
  • In a graded evaluation, gold labels are mapped to numbers (L3-Positive=1 down to L3-Negative=0). The system's average distance is 0.12, compared with 0.21 for the majority baseline.

Interesting ideas

  • I1. Factuality is computed by multiplying the scalar values of all embedding operators that scope over an event, starting from 1 for a plain declarative written by the author; this makes each step traceable and stacks naturally for nested hedges.
  • I2. Embedding predicates are sorted into a small taxonomy of classes, each tied to its own scale (epistemic, deontic, success, and so on), with secondary epistemic readings recorded; this lets one dictionary cover modals, negators, and verbs like "fail" or "manage".
  • I3. Ordinal labels are scored by average numeric distance rather than plain accuracy, so a near miss (somewhat vs. highly speculative) costs less than a gross error.
  • I4. The error analysis shows that parser mistakes, especially PP-attachment errors, quietly break semantic dependencies, and that patch rules on dependencies catch only some of these cases.
  • I5. The weak negation result suggests that negative polarity should come from explicit scope under a NEGATOR predicate, not from a low value on a modal scale.

For Mova

How these ideas could serve Mova, our small open language model — its files, stages and tests:

  • I1 (frames): Each frame could carry a source and an epistemic value in [0,1], updated by every modal or valence-shifting predicate that dominates it in the UD tree, so the confidence of every claim can be explained.
  • I2 (world): A compiled lexicon of such operator classes with prior scalar values could sit next to the VerbNet classes, letting the world model tell whether a state actually holds, might hold, or was attempted but failed.
  • I3 (learn): Graded factuality and certainty outputs could be scored by distance to gold or silver values, with a majority-class baseline used as a negative control.
  • I4 (tree): Collecting attachments that break operator scope (for example "ability … to X" attached to the main verb) would give a focused set of repair rules and test cases for the parser.
  • I5 (frames): Polarity could be stored as its own flag driven by negator scope, kept separate from the epistemic value, so that "may not" and "did not" stay distinct in the frame.

Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.

 Institutions

🏛️National Institutes of HealthBethesda · United States9 works · 3,833 citations🔬United States National Library of MedicineBethesda · United States5 works · 5,321 citations

📚 Shelves 6 Computer Science · 9,086 works Artificial Intelligence · 8,889 works Arts and Humanities · 9,296 works Language and Linguistics · 9,206 works Biochemistry, Genetics and Molecular Biology · 116 works Molecular Biology · 114 works

Cite

Cite as Halil Kilicoglu, Graciela Rosemblat, Michael Cairelli and Thomas Rindflesch. 2015. A Compositional Interpretation of Biomedical Event Factuality. Proceedings of the Second Workshop on Extra-Propositional Aspects of Meaning in Computational Semantics (ExProM 2015).

The paper itself, not this page. BibTeX from the library's record of the work. DOI 10.3115/v1/w15-1303

@inproceedings{kilicoglu2015compositional,
  title = {{A Compositional Interpretation of Biomedical Event Factuality}},
  author = {Halil Kilicoglu, Graciela Rosemblat, Michael Cairelli and Thomas Rindflesch},
  booktitle = {Proceedings of the Second Workshop on Extra-Propositional Aspects of Meaning in Computational Semantics (ExProM 2015)},
  year = {2015},
  doi = {10.3115/v1/w15-1303},
  url = {https://doi.org/10.3115/v1/w15-1303}
}

References

28 references, 11 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

  1. Jari Björne, Filip Ginter, and Tapio Salakoski. 2012. University of Turku in the BioNLP’11 Shared Task. BMC Bioinformatics, 13 Suppl 11:S4. ↗
  2. Marie-Catherine de Marneffe, Bill MacCartney, and Christopher D. Manning. 2006. Generating typed dependency parses from phrase structure parses. In Proceedings of the 5th International Conference on Language Resources and Evaluation, pages 449–454. our card
  3. Marie-Catherine de Marneffe, Christopher D. Manning, and Christopher Potts. 2012. Did it happen? The pragmatic complexity of veridicality assessment. Computational Linguistics, 38(2):301–333. our card
  4. Richárd Farkas, Veronika Vincze, György Móra, János Csirik, and György Szarvas. 2010. The CoNLL-2010 Shared Task: Learning to Detect Hedges and Their Scope in Natural Language Text. In Proceedings of the Fourteenth Conference on Computational Natural Language Learning — Shared Task, pages 1–12. our card
  5. Ken Hyland. 1998. Hedging in scientific research articles. John Benjamins B.V., Amsterdam, Netherlands. ↗ our card
  6. Halil Kilicoglu and Sabine Bergler. 2008. Recognizing speculative language in biomedical research articles: a linguistically motivated perspective. BMC Bioinformatics, 9 Suppl 11:s10. find
  7. Halil Kilicoglu and Sabine Bergler. 2010. A High-Precision Approach to Detecting Hedges and Their Scopes. In Proceedings of the Fourteenth Conference on Computational Natural Language Learning, pages 70–77. our card
  8. Halil Kilicoglu and Sabine Bergler. 2011. Effective Bio-Event Extraction using Trigger Words and Syntactic Dependencies. Computational Intelligence, 27(4):583–609. find
  9. Halil Kilicoglu. 2012. Embedding Predications. Ph.D. thesis, Concordia University. find
  10. Jin-Dong Kim, Tomoko Ohta, and Jun’ichi Tsujii. 2008. Corpus annotation for mining biomedical events from literature. BMC Bioinformatics, 9:10. ↗
  11. Jin-Dong Kim, Tomoko Ohta, Sampo Pyysalo, Yoshinobu Kano, and Jun’ichi Tsujii. 2009. Overview of BioNLP’09 Shared Task on Event Extraction. In Proceedings of Natural Language Processing in Biomedicine (BioNLP) NAACL 2009 Workshop, pages 1–9. our card
  12. Jin-Dong Kim, Ngan Nguyen, Yue Wang, Jun’ichi Tsujii, Toshihisa Takagi, and Akinori Yonezawa. 2012. The Genia Event and Protein Coreference tasks of the BioNLP Shared Task 2011. BMC Bioinformatics, 13 Suppl 11:S1. ↗
  13. Marc Light, Xin Y. Qiu, and Padmini Srinivasan. 2004. The language of bioscience: facts, speculations, and statements in between. In BioLINK 2004: Linking Biological Literature, Ontologies and Databases, pages 17–24. find
  14. Christopher D. Manning, Mihai Surdeanu, John Bauer, Jenny Finkel, Steven J. Bethard, and David McClosky. 2014. The Stanford CoreNLP natural language processing toolkit. In Proceedings of 52nd Annual Meeting of the Association for Computational Linguistics: System Demonstrations, pages 55–60. our card
  15. Ben Medlock and Ted Briscoe. 2007. Weakly supervised learning for hedge classification in scientific literature. In Proceedings of the 45th Meeting of the Association for Computational Linguistics, pages 992–999. our card
  16. Makoto Miwa, Paul Thompson, John McNaught, Douglas B. Kell, and Sophia Ananiadou. 2012. Extracting semantically enriched events from biomedical literature. BMC Bioinformatics, 13:108. ↗
  17. Roser Morante and Caroline Sporleder. 2012. Modality and negation: An introduction to the special issue. Computational Linguistics, 38(2):223–260. ↗ our card
  18. Roser Morante, Vincent van Asch, and Walter Daelemans. 2010. Memory-based resolution of in-sentence scopes of hedge cues. In Proceedings of the Fourteenth Conference on Computational Natural Language Learning, pages 40–47. our card
  19. Sergei Nirenburg and Victor Raskin. 2004. Ontological Semantics. The MIT Press, Cambridge, MA. find
  20. Roser Saurı́ and James Pustejovsky. 2009. FactBank: a corpus annotated with event factuality. Language Resources and Evaluation, 43(3):227–268. find
  21. Roser Saurı́ and James Pustejovsky. 2012. Are You Sure That This Happened? Assessing the Factuality Degree of Events in Text. Computational Linguistics, 38(2):261–299. our card
  22. Roser Saurı́. 2008. A Factuality Profiler for Eventualities in Text. Ph.D. thesis, Brandeis University. find
  23. Pontus Stenetorp, Sampo Pyysalo, Tomoko Ohta, Sophia Ananiadou, and Jun’ichi Tsujii. 2012. Bridging the gap between scope-based and event-based negation/speculation annotations: A bridge not too far. In Proceedings of the Workshop on Extra-Propositional Aspects of Meaning in Computational Linguistics, pages 47–56. find
  24. György Szarvas, Veronika Vincze, Richárd Farkas, György Móra, and Iryna Gurevych. 2012. Cross-genre and cross-domain detection of semantic uncertainty. Computational Linguistics, 38(2):335–367. ↗
  25. Paul Thompson, Raheel Nawaz, John McNaught, and Sophia Ananiadou. 2011. Enriching a biomedical event corpus with meta-knowledge annotation. BMC Bioinformatics, 12:393. ↗
  26. Özlem Uzuner, Imre Solti, and Eithon Cadag. 2010. Extracting medication information from clinical text. JAMIA, 17(5):514–518. ↗
  27. Veronika Vincze, György Szarvas, Richárd Farkas, György Móra, and János Csirik. 2008. The Bio-Scope corpus: biomedical texts annotated for uncertainty, negation and their scopes. BMC Bioinformatics, 9 Suppl 11:S9. find
  28. W. John Wilbur, Andrey Rzhetsky, and Hagit Shatkay. 2006. New directions in biomedical text annotations: definitions, guidelines and corpus construction. BMC Bioinformatics, 7:356. find

Updated

↑ back to top

This page in Markdown → https://md.research.elamur.ai/free/en/papers/kilicoglu-2015-compositional-interpretation-biomedical-event-factuality/