Elamur Research⌕
☰
home / papers / 2011 / Event Extraction as Dependency Parsing

Event Extraction as Dependency Parsing

Paper · 2011 · David McClosky, Mihai Surdeanu and Christopher Manning

Summary

The paper addresses biomedical event extraction, where events can take other events as arguments and so form nested structures. Many earlier systems predict each event and argument separately. The authors instead treat the whole event-argument structure as a dependency tree. Event anchors and protein mentions become nodes, argument slots become arc labels, and a virtual ROOT node sits on top. Several MSTParser instances with different decoders are trained on these semantic trees, not on syntax. Their n-best outputs are converted back to events and rescored by a reranker that uses global features and is tuned to the task metric. A separate classifier predicts the anchors. The authors evaluate on the BioNLP'09 shared task, where the system reaches 53.5% F1 on development and 48.6% on test. These scores are competitive with top systems that rely on far more domain-specific engineering. The work shows that joint, structure-aware modeling helps, and that standard parsing machinery can serve as an event extractor that is largely domain-independent.

Key points

  • Results: The system scores 48.6% F1 on the BioNLP'09 test set and 53.5% on development. For comparison, the best shared-task system scored 52.0% and the best post-evaluation system 56.3%.
  • Effect of reranking: Reranking across multiple decoders beats parsing without reranking in five of six configurations.
    • With predicted anchors, development F1 rises from 52.7 to 53.5.
    • With gold anchors, it rises from 72.7 to 73.1.
  • Ceiling from the tree conversion: The conversion itself caps performance at 87.1 F1. The main losses come from:
    • removing cross-sentence events, about 5% of the corpus;
    • keeping only the first event when an argument is shared, which removes 4.7% of training events.
  • Oracle reranking: Concatenating the n-best lists of three decoders gives an oracle of 86.7 with gold anchors, only 0.4 below the conversion limit. With predicted anchors the oracle is 73.4.
  • Simple vs. complex events: Simple events are much easier than complex ones on test. PHOSPHORYLATION reaches 81.2 F1, while REGULATION reaches only 28.3.

Interesting ideas

  • I1. Use a parser as a structured predictor. Any nested, anchor-based annotation can be recast as a labeled dependency tree, so a ready-made dependency parser can predict it. This could transfer to other nested semantic tasks without building a custom joint model.
  • I2. Make the conversion reversible and measure its loss. Lossy heuristics turn graphs into trees, and paired heuristics restore duplicated events afterward. Scoring the round trip on gold data gives an explicit ceiling, which is good practice whenever a task is squeezed into a simpler representation.
  • I3. Diversify candidates, then rerank. Pooling n-best lists from different decoders and choosing with a reranker trained on the task metric lets global, whole-structure features shape the final decision. Oracle scores show how much headroom the candidate pool leaves.
  • I4. Diagnose components with gold substitution. Swapping gold data into each pipeline stage in turn (anchors, parse, reranking, conversion) shows where errors originate. The method is cheap and applies to any multi-stage pipeline.
  • I5. Extend the tree to whole documents. A single document-level ROOT could keep cross-sentence events and allow discourse-level features. The same trick could help other tasks with sentence-crossing relations.

Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.

 Institutions

🎓Stanford UniversityStanford · United States310 works · 284,480 citations

📚 Shelves 6The field of science this work belongs to (OpenAlex fields).Computer Science · 9,086 worksArtificial Intelligence · 8,889 worksArts and Humanities · 9,296 worksLanguage and Linguistics · 9,206 worksBiochemistry, Genetics and Molecular Biology · 116 worksMolecular Biology · 114 works

🧭 Topics 2Research subjects and languages the work is about.Information extraction · 109 worksDependency parsing · 120 works

Cite

Cite as David McClosky, Mihai Surdeanu and Christopher Manning. 2011. Event Extraction as Dependency Parsing. Proceedings of the 49th Annual Meeting of the Association for Computational Linguistics: Human Language Technologies.

The paper itself, not this page. BibTeX from the library's record of the work.

@inproceedings{mcclosky2011event,
  title = {{Event Extraction as Dependency Parsing}},
  author = {David McClosky, Mihai Surdeanu and Christopher Manning},
  booktitle = {Proceedings of the 49th Annual Meeting of the Association for Computational Linguistics: Human Language Technologies},
  year = {2011},
  url = {https://aclanthology.org/P11-1163/}
}

References

29 references, 12 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

  1. Jari Björne, Juho Heimonen, Filip Ginter, Antti Airola, Tapio Pahikkala, and Tapio Salakoski. 2009. Extracting Complex Biological Events with Rich Graph-Based Feature Sets. Proceedings of the Workshop on BioNLP: Shared Task. find
  2. Nate Chambers and Dan Jurafsky. 2009. Unsupervised Learning of Narrative Schemas and their Participants. Proceedings of ACL. find
  3. Eugene Charniak and Mark Johnson. 2005. Coarse-to-fine n-best parsing and MaxEnt discriminative reranking. In Proceedings of the 2005 Meeting of the Association for Computational Linguistics (ACL), pages 173–180
  4. Nancy Chinchor. 1997. Overview of MUC-7. Proceedings of the Message Understanding Conference (MUC-7). find
  5. Michael Collins. 2000. Discriminative reranking for natural language parsing. In Machine Learning: Proceedings of the Seventeenth International Conference (ICML 2000), pages 175–182.
  6. Jenny R. Finkel and Christopher D. Manning. 2009. Joint Parsing and Named Entity Recognition. Proceedings of NAACL.
  7. Jenny R. Finkel and Christopher D. Manning. 2009b. Nested Named Entity Recognition. Proceedings of EMNLP.
  8. Jan Hajič, Massimiliano Ciaramita, Richard Johansson, Daisuke Kawahara, Maria A. Marti, Lluis Marquez, Adam Meyers, Joakim Nivre, Sebastian Pado, Jan Stepanek, Pavel Stranak, Mihai Surdeanu, Nianwen Xue, and Yi Zhang. 2009. The CoNLL-2009 Shared Task: Syntactic and Semantic Dependencies in Multiple…
  9. Eduard Hovy, Mitchell P. Marcus, Martha Palmer, Lance Ramshaw, and Ralph Weischedel. 2006. OntoNotes: The 90% Solution. Proceedings of the NAACL-HLT. find
  10. Liang Huang. 2008. Forest reranking: Discriminative parsing with non-local features. In Proceedings of ACL-08: HLT, pages 586–594, Association for Computational Linguistics. find
  11. Mark Johnson and Ahmet Engin Ural. 2010. Reranking the Berkeley and Brown Parsers. Proceedings of NAACL. find
  12. Jin-Dong Kim, Tomoko Ohta, Sampo Pyysalo, Yoshinobu Kano, and Jun’ichi Tsujii. 2009. Overview of the BioNLP’09 Shared Task on Event Extraction. Proceedings of the NAACL-HLT 2009 Workshop on Natural Language Processing in Biomedicine (BioNLP’09). find
  13. Mitchell P. Marcus, Beatrice Santorini, and Marry Ann Marcinkiewicz. 1993. Building a Large Annotated Corpus of English: The Penn Treebank. Computational Linguistics, 19(2):313–330.
  14. Marie-Catherine de Marneffe and Christopher D. Manning. 2008. Stanford Typed Hierarchies Representation. Proceedings of the COLING Workshop on Cross-Framework and Cross-Domain Parser Evaluation. find
  15. David McClosky. 2010. Any Domain Parsing: Automatic Domain Adaptation for Natural Language Parsing. PhD thesis, Department of Computer Science, Brown University. find
  16. David McClosky, Mihai Surdeanu, and Christopher D. Manning. 2011. Event extraction as dependency parsing in BioNLP 2011. In BioNLP 2011 Shared Task (submitted). find
  17. Ryan McDonald, Koby Crammer, and Fernando Pereira. 2005. Online Large-Margin Training of Dependency Parsers. Proceedings of ACL.
  18. Ryan McDonald, Fernando Pereira, Kiril Ribarov, and Jan Hajič. 2005b. Non-projective Dependency Parsing using Spanning Tree Algorithms. Proceedings of HLT/EMNLP.
  19. Ryan McDonald and Fernando Pereira. 2006. Online Learning of Approximate Dependency Parsing Algorithms. Proceedings of EACL.
  20. Scott Miller, Michael Crystal, Heidi Fox, Lance Ramshaw, Richard Schwartz, Rebecca Stone, and Ralph Weischedel. 1997. BBN: Description of the SIFT System as Used for MUC-7. Proceedings of the Message Understanding Conference (MUC-7). find
  21. Makoto Miwa, Sampo Pyysalo, Tadayoshi Hara, and Jun’ichi Tsujii. 2010a. A Comparative Study of Syntactic Parsers for Event Extraction. Proceedings of the 2010 Workshop on Biomedical Natural Language Processing. find
  22. Makoto Miwa, Rune Saetre, Jin-Dong Kim, and Jun’ichi Tsujii. 2010b. Event Extraction with Complex Event Classification Using Rich Features. Journal of Bioinformatics and Computational Biology, 8 (1). find
  23. Hoifung Poon and Lucy Vanderwende. 2010. Joint Inference for Knowledge Extraction from Biomedical Literature. Proceedings of NAACL. find
  24. Sebastian Riedel, Hong-Woo Chun, Toshihisa Takagi, and Jun’ichi Tsujii. 2009. A Markov Logic Approach to Bio-Molecular Event Extraction. Proceedings of the Workshop on BioNLP: Shared Task. find
  25. Alexander M. Rush, David Sontag, Michael Collins, and Tommi Jaakkola. 2010. On dual decomposition and linear programming relaxations for natural language processing. Proceedings of EMNLP.
  26. Kenji Sagae and Jun’ichi Tsujii. 2008. Shift-Reduce Dependency DAG Parsing. Proceedings of the COLING. find
  27. Mihai Surdeanu, Richard Johansson, Adam Meyers, Lluis Marquez, and Joakim Nivre. 2008. The CoNLL-2008 Shared Task on Joint Parsing of Syntactic and Semantic Dependencies. Proceedings of CoNLL.
  28. Kristina Toutanova, Aria Haghighi, and Christopher D. Manning. 2008. A Global Joint Model for Semantic Role Labeling. Computational Linguistics 34(2).
  29. Zhang, H. and Zhang, M. and Tan, C.L. and Li, H. 2009. K-best combination of syntactic parsers. Proceedings of Empirical Methods in Natural Language Processing. find

Updated

↑ back to top

This page in Markdown → https://md.research.elamur.ai/free/en/papers/mcclosky-2011-event-extraction-as-dependency-parsing/