Elamur Research⌕
☰
home / papers / 2016 / Event Detection and Co-reference with Minimal Supervision

Event Detection and Co-reference with Minimal Supervision

Summary

Supervised event systems are usually trained on small annotated sets with fixed type inventories, such as 33 types in ACE 2005 or 38 in TAC KBP, so they tend to overfit and transfer poorly. Peng, Song and Roth propose MSEP (Minimally Supervised Event Pipeline), which needs only a few plain-text event examples per type, taken from annotation guidelines. Identifying events and linking co-referent ones are both framed as judging how semantically related pieces of text are. Each mention is parsed with Illinois SRL, and its parts are mapped to five abstract slots: action, subject agent, object agent, location and time. Each slot, plus the surrounding clause, is embedded with task-independent representations learned from large text collections, mainly ESA. These vectors are concatenated into a structured event vector. A mention is assigned a type by cosine similarity to averaged example vectors, and the same similarity decides whether two mentions co-refer. Thresholds are tuned only on the examples, so MSEP requires no training. Detection comes close to top supervised systems, co-reference beats them on B³ and CEAF, and MSEP holds up better across domains.

Key points

  • Supervision consists of 172 example sentences covering the 33 ACE event types, about five per type. Thresholds are tuned on these examples and reused for every test set.
  • SRL covers event structure well. Predicate recall over triggers is 91.9 on ACE and 88.1 on TAC KBP. Argument-match F1 is 86.4 on ACE and 85.0 on TAC KBP.
  • The event representations compared are ESA, Brown clusters, Word2Vec and dependency embeddings. Further variants add action–argument fragment combinations (AUG) and knowledge-based conflict detection (KNOW).
  • In end-to-end co-reference on TAC-KBP test data, MSEP scores 37.1 average F1, compared with 39.1 for the top TAC system. Pairing supervised SSED detection with MSEP co-reference gives 38.5.
  • In newswire/discussion-forum transfer experiments on TAC-KBP, supervised baselines win in-domain. Out of domain, MSEP matches or slightly surpasses them on both tasks.

Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.

Fields: Linguistics & languages

Topics: Coreference resolution

Cited by

55 papers of our library cite it (of the 140673 whose bibliographies we read); 11 of them have a page here, 44 not yet.

Cite

The paper itself, not this page. BibTeX from the library's record of the work. DOI 10.18653/v1/d16-1038

@inproceedings{peng2016event,
  title = {{Event Detection and Co-reference with Minimal Supervision}},
  author = {Peng, Haoruo and Song, Yangqiu and Roth, Dan},
  booktitle = {Proceedings of the 2016 Conference on Empirical Methods in Natural Language Processing},
  year = {2016},
  doi = {10.18653/v1/d16-1038},
  url = {https://doi.org/10.18653/v1/d16-1038}
}

References

44 references, 13 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

  1. A. Bagga and B. Baldwin. 1998. Algorithms for scoring coreference chains. In MUC-7. find our card
  2. A. Bagga and B. Baldwin. 1999. Cross-document event coreference: Annotations, experiments, and observations. In Proceedings of the Workshop on Coreference and its Applications. find our card
  3. C. A. Bejan and S. Harabagiu. 2010. Unsupervised event coreference resolution with rich linguistic features. In ACL. find our card
  4. E. Bengtson and D. Roth. 2008. Understanding the value of features for coreference resolution. In EMNLP. find our card
  5. O. Bronstein, I. Dagan, Q. Li, H. Ji, and A. Frank. 2015. Seed-based event trigger labeling: How far can event descriptions get us? In ACL. P. Brown, V. Della Pietra, P. deSouza, J. Lai, and R. Mercer. 1992. Class-based n-gram models of natural language. find
  6. Computational Linguistics. M. Chang, L. Ratinov, D. Roth, and V. Srikumar. 2008. Importance of semantic represenation: Dataless classification. In AAAI. find
  7. Z. Chen and H. Ji. 2009. Graph-based event coreference resolution. In Proceedings of the Workshop on Graph-based Methods for Natural Language Processing. find our card
  8. Z. Chen, H. Ji, and R. Haralick. 2009. A pairwise event coreference model, feature impact and evaluation for event coreference resolution. In Proceedings of the Workshop on Events in Emerging Text Types. find our card
  9. Y. Chen, L. Xu, K. Liu, D. Zeng, and J. Zhao. 2015. Event extraction via dynamic multi-pooling convolutional neural networks. In ACL. find
  10. X. Cheng and D. Roth. 2013. Relational inference for wikification. In EMNLP. find
  11. A. Cybulska and P. Vossen. 2012. Using semantic relations to solve event coreference in text. In Proceedings of the Workshop on Semantic relations. find
  12. L. Danlos and B. Gaiffe. 2003. Event coreference and discourse relations. find
  13. Philosophical Studies Series. Q. Do, Y. S. Chan, and D. Roth. 2011. Minimally supervised event causality extraction. In EMNLP. find
  14. Q. Do, W. Lu, and D. Roth. 2012. Joint inference for event timeline construction. In EMNLP. find
  15. A. Elkhlifi and R. Faiz. 2009. Automatic annotation approach of events in news articles. International Journal of Computing & Information Sciences. find
  16. Evgeniy Gabrilovich and Shaul Markovitch. 2009. Wikipedia-based semantic interpretation for natural language processing. J. Artif. Int. Res., 34(1):443– 498, March. find
  17. K. Goyal, S. K. Jauhar, H. Li, M. Sachan, S. Srivastava, and E. Hovy. 2013. A structured distributional semantic model for event co-reference. In ACL. find our card
  18. Y. Hong, J. Zhang, B. Ma, J. Yao, G. Zhou, and Q. Zhu. 2011. Using cross-entity inference to improve event extraction. In ACL. find
  19. E. Hovy, T.o Mitamura, F. Verdejo, J. Araki, and A. Philpot. 2013. Events are not simple: Identity, non-identity, and quasi-identity. In NAACL-HLT. find
  20. A. Hsi, J. Carbonell, and Y. Yang. 2015. Modeling event extraction via multilingual data sources. In TAC. find
  21. R. Huang and E. Riloff. 2012a. Bootstrapped training of event extraction classifiers. In EACL. find
  22. R. Huang and E. Riloff. 2012b. Modeling textual cohesion for event extraction. In AAAI. find
  23. L. Huang, T. Cassidy, X. Feng, H. Ji, C. R. Voss, J. Han, and A. Sil. 2016. Liberal event extraction and event schema induction. In ACL. find
  24. K. Humphreys, R. Gaizauskas, and S. Azzam. 1997. Event coreference for information extraction. In Proceedings of Workshop on Operational Factors in Practical, Robust Anaphora Resolution for Unrestricted Texts. find our card
  25. H. Ji and R. Grishman. 2008. Refining event extraction through cross-document inference. In ACL. find
  26. H. Lee, M. Recasens, A. Chang, M. Surdeanu, and D. Jurafsky. 2012. Joint entity and event coreference resolution across documents. In EMNLP. find our card
  27. O. Levy and Y. Goldberg. 2014. Dependencybased word embeddings. In ACL. find
  28. Q. Li, H. Ji, and L. Huang. 2013. Joint event extraction via structured prediction with global features. In ACL. find
  29. S. Liao and R. Grishman. 2010. Using document level cross-event inference to improve event extraction. In ACL. find
  30. Z. Liu, T. Mitamura, and E. Hovy. 2015. Evaluation algorithms for event nugget detection: A pilot study. In Proceedings of the Workshop on Events at the NAACL-HLT. find
  31. X. Luo. 2005. On coreference resolution performance metrics. In EMNLP. find our card
  32. T. Mikolov, W. Yih, and G. Zweig. 2013. Linguistic regularities in continuous space word representations. In NAACL. find
  33. T. Mitamura, Y. Yamakawa, S. Holm, Z. Song, A. Bies, S. Kulick, and S. Strassel. 2015. Event nugget annotation: Processes and issues. In Proceedings of the Workshop on Events at NAACL-HLT. find
  34. M. Naughton. 2009. Sentence Level Event Detection and Coreference Resolution. Ph.D. thesis, National University of Ireland, Dublin. find
  35. V. Ng and C. Cardie. 2002. Improving machine learning approaches to coreference resolution. In ACL. find our card
  36. NIST. 2005. The ACE evaluation plan. find
  37. S. Pradhan, L. Ramshaw, R. Weischedel, J. MacBride, and L. Micciulla. 2007. Unrestricted coreference: Identifying entities and events in ontonotes. In ICSC. find
  38. V. Punyakanok, D. Roth, W. Yih, and D. Zimak. 2004. Semantic role labeling via integer linear programming inference. In COLING. find
  39. M. Recasens and E. Hovy. 2011. Blanc: Implementing the rand index for coreference evaluation. Natural Language Engineering, 17(04):485–510. find
  40. M. Sammons, H. Peng, Y. Song, S. Upadhyay, C.-T. Tsai, P. Reddy, S. Roy, and D. Roth. 2015. Illinois ccg tac 2015 event nugget, entity discovery and linking, and slot filler validation systems. In TAC. find
  41. Y. Song and D. Roth. 2014. On dataless hierarchical text classification. In AAAI. find
  42. V. Stoyanov, C. Cardie, N. Gilbert, E. Riloff, D. Buttler, and D. Hysom. 2010. Coreference resolution with reconcile. In ACL. find our card
  43. M. Vilain, J. Burger, J. Aberdeen, D. Connolly, and L. Hirschman. 1995. A model-theoretic coreference scoring scheme. In Proceedings of the 6th conference on Message understanding. find our card
  44. R. Zhao, Q. Do, and D. Roth. 2012. A robust shallow temporal reasoning system. In North American Chapter of the Association for Computational Linguistics: Human Language Technologies (NAACL-HLT Demo). find

↑ back to top

This page in Markdown →