Event Detection and Co-reference with Minimal Supervision

Summary

Supervised event systems are usually trained on small annotated sets with fixed type inventories, such as 33 types in ACE 2005 or 38 in TAC KBP, so they tend to overfit and transfer poorly. Peng, Song and Roth propose MSEP (Minimally Supervised Event Pipeline), which needs only a few plain-text event examples per type, taken from annotation guidelines. Identifying events and linking co-referent ones are both framed as judging how semantically related pieces of text are. Each mention is parsed with Illinois SRL, and its parts are mapped to five abstract slots: action, subject agent, object agent, location and time. Each slot, plus the surrounding clause, is embedded with task-independent representations learned from large text collections, mainly ESA. These vectors are concatenated into a structured event vector. A mention is assigned a type by cosine similarity to averaged example vectors, and the same similarity decides whether two mentions co-refer. Thresholds are tuned only on the examples, so MSEP requires no training. Detection comes close to top supervised systems, co-reference beats them on B³ and CEAF, and MSEP holds up better across domains.

Key points

Supervision consists of 172 example sentences covering the 33 ACE event types, about five per type. Thresholds are tuned on these examples and reused for every test set.
SRL covers event structure well. Predicate recall over triggers is 91.9 on ACE and 88.1 on TAC KBP. Argument-match F1 is 86.4 on ACE and 85.0 on TAC KBP.
The event representations compared are ESA, Brown clusters, Word2Vec and dependency embeddings. Further variants add action–argument fragment combinations (AUG) and knowledge-based conflict detection (KNOW).
In end-to-end co-reference on TAC-KBP test data, MSEP scores 37.1 average F1, compared with 39.1 for the top TAC system. Pairing supervised SSED detection with MSEP co-reference gives 38.5.
In newswire/discussion-forum transfer experiments on TAC-KBP, supervised baselines win in-domain. Out of domain, MSEP matches or slightly surpasses them on both tasks.

Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.

Fields: Linguistics & languages

Topics: Coreference resolution

Cited by

55 papers of our library cite it (of the 140673 whose bibliographies we read); 11 of them have a page here, 44 not yet.

Cite

The paper itself, not this page. BibTeX from the library's record of the work. DOI 10.18653/v1/d16-1038

@inproceedings{peng2016event,
  title = {{Event Detection and Co-reference with Minimal Supervision}},
  author = {Peng, Haoruo and Song, Yangqiu and Roth, Dan},
  booktitle = {Proceedings of the 2016 Conference on Empirical Methods in Natural Language Processing},
  year = {2016},
  doi = {10.18653/v1/d16-1038},
  url = {https://doi.org/10.18653/v1/d16-1038}
}

References

44 references, 13 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

A. Bagga and B. Baldwin. 1998. Algorithms for scoring coreference chains. In MUC-7. find our card
A. Bagga and B. Baldwin. 1999. Cross-document event coreference: Annotations, experiments, and observations. In Proceedings of the Workshop on Coreference and its Applications. find our card
C. A. Bejan and S. Harabagiu. 2010. Unsupervised event coreference resolution with rich linguistic features. In ACL. find our card
E. Bengtson and D. Roth. 2008. Understanding the value of features for coreference resolution. In EMNLP. find our card
O. Bronstein, I. Dagan, Q. Li, H. Ji, and A. Frank. 2015. Seed-based event trigger labeling: How far can event descriptions get us? In ACL. P. Brown, V. Della Pietra, P. deSouza, J. Lai, and R. Mercer. 1992. Class-based n-gram models of natural language. find
Computational Linguistics. M. Chang, L. Ratinov, D. Roth, and V. Srikumar. 2008. Importance of semantic represenation: Dataless classification. In AAAI. find
Z. Chen and H. Ji. 2009. Graph-based event coreference resolution. In Proceedings of the Workshop on Graph-based Methods for Natural Language Processing. find our card
Z. Chen, H. Ji, and R. Haralick. 2009. A pairwise event coreference model, feature impact and evaluation for event coreference resolution. In Proceedings of the Workshop on Events in Emerging Text Types. find our card
Y. Chen, L. Xu, K. Liu, D. Zeng, and J. Zhao. 2015. Event extraction via dynamic multi-pooling convolutional neural networks. In ACL. find
X. Cheng and D. Roth. 2013. Relational inference for wikification. In EMNLP. find
A. Cybulska and P. Vossen. 2012. Using semantic relations to solve event coreference in text. In Proceedings of the Workshop on Semantic relations. find
L. Danlos and B. Gaiffe. 2003. Event coreference and discourse relations. find
Philosophical Studies Series. Q. Do, Y. S. Chan, and D. Roth. 2011. Minimally supervised event causality extraction. In EMNLP. find
Q. Do, W. Lu, and D. Roth. 2012. Joint inference for event timeline construction. In EMNLP. find
A. Elkhlifi and R. Faiz. 2009. Automatic annotation approach of events in news articles. International Journal of Computing & Information Sciences. find
Evgeniy Gabrilovich and Shaul Markovitch. 2009. Wikipedia-based semantic interpretation for natural language processing. J. Artif. Int. Res., 34(1):443– 498, March. find
K. Goyal, S. K. Jauhar, H. Li, M. Sachan, S. Srivastava, and E. Hovy. 2013. A structured distributional semantic model for event co-reference. In ACL. find our card
Y. Hong, J. Zhang, B. Ma, J. Yao, G. Zhou, and Q. Zhu. 2011. Using cross-entity inference to improve event extraction. In ACL. find
E. Hovy, T.o Mitamura, F. Verdejo, J. Araki, and A. Philpot. 2013. Events are not simple: Identity, non-identity, and quasi-identity. In NAACL-HLT. find
A. Hsi, J. Carbonell, and Y. Yang. 2015. Modeling event extraction via multilingual data sources. In TAC. find
R. Huang and E. Riloff. 2012a. Bootstrapped training of event extraction classifiers. In EACL. find
R. Huang and E. Riloff. 2012b. Modeling textual cohesion for event extraction. In AAAI. find
L. Huang, T. Cassidy, X. Feng, H. Ji, C. R. Voss, J. Han, and A. Sil. 2016. Liberal event extraction and event schema induction. In ACL. find
K. Humphreys, R. Gaizauskas, and S. Azzam. 1997. Event coreference for information extraction. In Proceedings of Workshop on Operational Factors in Practical, Robust Anaphora Resolution for Unrestricted Texts. find our card
H. Ji and R. Grishman. 2008. Refining event extraction through cross-document inference. In ACL. find
H. Lee, M. Recasens, A. Chang, M. Surdeanu, and D. Jurafsky. 2012. Joint entity and event coreference resolution across documents. In EMNLP. find our card
O. Levy and Y. Goldberg. 2014. Dependencybased word embeddings. In ACL. find
Q. Li, H. Ji, and L. Huang. 2013. Joint event extraction via structured prediction with global features. In ACL. find
S. Liao and R. Grishman. 2010. Using document level cross-event inference to improve event extraction. In ACL. find
Z. Liu, T. Mitamura, and E. Hovy. 2015. Evaluation algorithms for event nugget detection: A pilot study. In Proceedings of the Workshop on Events at the NAACL-HLT. find
X. Luo. 2005. On coreference resolution performance metrics. In EMNLP. find our card
T. Mikolov, W. Yih, and G. Zweig. 2013. Linguistic regularities in continuous space word representations. In NAACL. find
T. Mitamura, Y. Yamakawa, S. Holm, Z. Song, A. Bies, S. Kulick, and S. Strassel. 2015. Event nugget annotation: Processes and issues. In Proceedings of the Workshop on Events at NAACL-HLT. find
M. Naughton. 2009. Sentence Level Event Detection and Coreference Resolution. Ph.D. thesis, National University of Ireland, Dublin. find
V. Ng and C. Cardie. 2002. Improving machine learning approaches to coreference resolution. In ACL. find our card
NIST. 2005. The ACE evaluation plan. find
S. Pradhan, L. Ramshaw, R. Weischedel, J. MacBride, and L. Micciulla. 2007. Unrestricted coreference: Identifying entities and events in ontonotes. In ICSC. find
V. Punyakanok, D. Roth, W. Yih, and D. Zimak. 2004. Semantic role labeling via integer linear programming inference. In COLING. find
M. Recasens and E. Hovy. 2011. Blanc: Implementing the rand index for coreference evaluation. Natural Language Engineering, 17(04):485–510. find
M. Sammons, H. Peng, Y. Song, S. Upadhyay, C.-T. Tsai, P. Reddy, S. Roy, and D. Roth. 2015. Illinois ccg tac 2015 event nugget, entity discovery and linking, and slot filler validation systems. In TAC. find
Y. Song and D. Roth. 2014. On dataless hierarchical text classification. In AAAI. find
V. Stoyanov, C. Cardie, N. Gilbert, E. Riloff, D. Buttler, and D. Hysom. 2010. Coreference resolution with reconcile. In ACL. find our card
M. Vilain, J. Burger, J. Aberdeen, D. Connolly, and L. Hirschman. 1995. A model-theoretic coreference scoring scheme. In Proceedings of the 6th conference on Message understanding. find our card
R. Zhao, Q. Do, and D. Roth. 2012. A robust shallow temporal reasoning system. In North American Chapter of the Association for Computational Linguistics: Human Language Technologies (NAACL-HLT Demo). find

↑ back to top

This page in Markdown → https://md.research.elamur.ai/free/en/papers/peng-2016-event-detection-co-reference-minimal/

📄 Event Detection and Co-reference with Minimal Supervision