Elamur Research⌕
☰
home / papers / 2021 / A Context-Dependent Gated Module for Incorporating Symbolic Semantics into Event Coreference Resolution

A Context-Dependent Gated Module for Incorporating Symbolic Semantics into Event Coreference Resolution

Summary

The paper addresses within-document event coreference resolution: grouping event mentions in a text that describe the same real-world event. The authors argue that strong pretrained encoders such as SpanBERT still benefit from symbolic features like event type, modality, polarity, genericity, tense and realis. However, upstream systems predict these features automatically, so they are often wrong, and their usefulness depends on context. The authors propose a Context-Dependent Gated Module (CDGM). It splits each feature-based pair representation into two parts relative to the trigger-based representation: a part parallel to it (redundant) and a part orthogonal to it (new). A sigmoid gate then decides how to mix the two. To teach the gates to handle unreliable inputs, they add a noisy training scheme that randomly swaps feature values during training. On ACE 2005 and KBP 2016, with predicted mentions and features, the full model clearly beats a SpanBERT baseline and naive feature concatenation. It surpasses prior systems on KBP 2016. The work shows a general way to use imperfect symbolic signals alongside contextual embeddings.

Key points

  • Features and tools: ACE 2005 features are type, polarity, modality, genericity and tense; KBP 2016 features are type and realis. Event mentions and subtypes come from OneIE, and the remaining attributes come from a SpanBERT-based joint classifier. Feature accuracy on the test sets ranges from about 0.76 (tense) to 0.99 (polarity).
  • ACE 2005, end-to-end: CDGM + Noise reaches 62.07 CoNLL and 59.76 AVG. The baseline scores 58.93 / 55.78, and simple concatenation scores 57.55 / 54.79, slightly below the baseline. A multi-task alternative reaches only 56.5 AVG. These ACE scores are not directly comparable with earlier work, which used 10-fold cross-validation.
  • KBP 2016, end-to-end: the full model scores 43.55 CoNLL and 40.61 AVG, ahead of E3C (41.97 / 38.66) and the baseline (40.57 / 37.59).
  • Ablations: the gains over concatenation are largest for noisier features, such as modality (+3.73 AVG) and genericity (+3.12 AVG). With ground-truth features, CDGM alone works best (88.30 AVG), and noisy training no longer helps.
  • Code: publicly available at github.com/laituan245/eventcoref.

Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.

Fields: Linguistics & languages

Topics: Coreference resolution

Cited by

21 papers of our library cite it (of the 140673 whose bibliographies we read); 3 of them have a page here, 18 not yet.

Cite

The paper itself, not this page. BibTeX from the library's record of the work. DOI 10.18653/v1/2021.naacl-main.274 · arXiv 2104.01697

@inproceedings{lai2021context,
  title = {{A Context-Dependent Gated Module for Incorporating Symbolic Semantics into Event Coreference Resolution}},
  author = {Lai, Tuan and Ji, Heng and Bui, Trung and Tran, Quan Hung and Dernoncourt, Franck and Chang, Walter},
  booktitle = {Proceedings of the 2021 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies},
  year = {2021},
  doi = {10.18653/v1/2021.naacl-main.274},
  eprint = {2104.01697},
  archivePrefix = {arXiv},
  url = {https://doi.org/10.18653/v1/2021.naacl-main.274}
}

References

42 references, 13 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

  1. Chase Duncan, Liang-Wei Chan, Haoruo Peng, Hao Wu, Shyam Upadhyay, Nitish Gupta, Chen-Tse Tsai, Mark Sammons, and Dan Roth. 2017. UI CCG TAC-KBP2017 submissions: Entity discovery and linking, and event nugget detection and co-reference. In Proceedings of the 2017 Text Analysis Conference, TAC 2017,… find
  2. C. Chen and Vincent Ng. 2014. Sinocoreferencer: An end-to-end chinese event coreference resolver. In LREC. find our card
  3. Chen Chen and Vincent Ng. 2016. Joint inference over a lightly supervised information extraction pipeline: Towards event coreference resolution for resource-scarce languages. In Proceedings of the Thirtieth AAAI Conference on Artificial Intelligence, AAAI’16, page 2913–2920. AAAI Press. find
  4. Yubo Chen, Liheng Xu, Kang Liu, Daojian Zeng, and Jun Zhao. 2015. Event extraction via dynamic multi-pooling convolutional neural networks. In Proceedings of the 53rd Annual Meeting of the Association for Computational Linguistics and the 7th International Joint Conference on Natural Language Proces… find
  5. Zheng Chen and Heng Ji. 2009. Graph-based event coreference resolution. In Proceedings of the 2009 Workshop on Graph-based Methods for Natural Language Processing (TextGraphs-4), pages 54–57, Suntec, Singapore. Association for Computational Linguistics. find our card
  6. Zheng Chen, Heng Ji, and Robert Haralick. 2009. A pairwise event coreference model, feature impact and evaluation for event coreference resolution. In Proceedings of the Workshop on Events in Emerging Text Types, pages 17–22, Borovets, Bulgaria. Association for Computational Linguistics. find our card
  7. Prafulla Kumar Choubey and Ruihong Huang. 2017. Event coreference resolution by iteratively unfolding inter-dependencies among events. In Proceedings of the 2017 Conference on Empirical Methods in Natural Language Processing, pages 2124–2133, Copenhagen, Denmark. Association for Computational Lingui… find our card
  8. Prafulla Kumar Choubey, Kaushik Raju, and Ruihong Huang. 2018. Identifying the most dominant event in a news article by mining event coreference relations. In Proceedings of the 2018 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologie… find our card
  9. C. Harris, K. J. Millman, S. Walt, Ralf Gommers, P. Virtanen, D. Cournapeau, E. Wieser, J. Taylor, S. Berg, Nathaniel J. Smith, R. Kern, Matti Picus, S. Hoyer, M. Kerkwijk, Matthew Brett, Allan Haldane, Jaime Fern’andez del R’io, Mark Wiebe, P. Peterson, Pierre G’erard-Marchant, K. Sheppard, T. Redd… find our card
  10. Eduard Hovy, Teruko Mitamura, Felisa Verdejo, Jun Araki, and Andrew Philpot. 2013. Events are not simple: Identity, non-identity, and quasi-identity. In Workshop on Events: Definition, Detection, Coreference, and Representation, pages 21–28, Atlanta, Georgia. Association for Computational Linguistic… find
  11. Yin Jou Huang, Jing Lu, Sadao Kurohashi, and Vincent Ng. 2019. Improving event coreference resolution by learning argument compatibility from unlabeled data. In Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technolog… find
  12. Heng Ji and Ralph Grishman. 2011. Knowledge base population: Successful approaches and challenges. In Proceedings of the 49th Annual Meeting of the Association for Computational Linguistics: Human Language Technologies, pages 1148–1158, Portland, Oregon, USA. Association for Computational Linguistic… find
  13. Heng Ji, Joel Nothman, H Trang Dang, and Sydney Informatics Hub. 2016. Overview of tac-kbp2016 trilingual edl and its impact on end-to-end cold-start kbp. Proceedings of TAC. find
  14. Mandar Joshi, Danqi Chen, Yinhan Liu, Daniel Weld, Luke Zettlemoyer, and Omer Levy. 2020. Spanbert: Improving pre-training by representing and predicting spans. Transactions of the Association for Computational Linguistics, 8(0):64–77. find our card
  15. Anders Krogh and John Hertz. 1992. A simple weight decay can improve generalization. In Advances in Neural Information Processing Systems, volume 4. Morgan-Kaufmann. find
  16. Tuan Lai, Quan Hung Tran, Trung Bui, and Daisuke Kihara. 2019. A gated self-attention memory network for answer selection. In Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing (EMNLP-IJCNLP… find
  17. LDC. 2005. English annotation guidelines for events version 5.4. 3. Linguistic Data Consortium, 1. find
  18. Kenton Lee, Luheng He, Mike Lewis, and Luke Zettlemoyer. 2017. End-to-end neural coreference resolution. In Proceedings of the 2017 Conference on Empirical Methods in Natural Language Processing, pages 188–197, Copenhagen, Denmark. Association for Computational Linguistics. find our card
  19. Ying Lin, Heng Ji, Fei Huang, and Lingfei Wu. 2020. A joint neural model for information extraction with global features. In Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics, pages 7999–8009, Online. Association for Computational Linguistics. find our card
  20. Ying Lin, Liyuan Liu, Heng Ji, Dong Yu, and Jiawei Han. 2019. Reliability-aware dynamic feature composition for name tagging. In Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics, pages 165–174, Florence, Italy. Association for Computational Linguistics. find
  21. J. Lu and Vincent Ng. 2017a. Learning antecedent structures for event coreference resolution. 2017 16th IEEE International Conference on Machine Learning and Applications (ICMLA), pages 113– 118. find
  22. Jing Lu and Vincent Ng. 2015. Utd’s event nugget detection and coreference system at KBP 2015. In Proceedings of the 2015 Text Analysis Conference, TAC 2015, Gaithersburg, Maryland, USA, November 16-17, 2015, 2015. NIST. find
  23. Jing Lu and Vincent Ng. 2016. Event coreference resolution with multi-pass sieves. In Proceedings of the Tenth International Conference on Language Resources and Evaluation (LREC’16), pages 3996– 4003, Portorož, Slovenia. European Language Resources Association (ELRA). find
  24. Jing Lu and Vincent Ng. 2017b. Joint learning for event coreference resolution. In Proceedings of the 55th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 90–101, Vancouver, Canada. Association for Computational Linguistics. find
  25. Jing Lu and Vincent Ng. 2018. Event coreference resolution: A survey of two decades of research. In Proceedings of the Twenty-Seventh International Joint Conference on Artificial Intelligence, IJCAI-18, pages 5479–5486. International Joint Conferences on Artificial Intelligence Organization. find
  26. Yaojie Lu, Hongyu Lin, Jialong Tang, Xianpei Han, and Le Sun. 2020. End-to-end neural event coreference resolution. arXiv preprint arXiv:2009.08153. — [arXiv:2009.08153](https://arxiv.org/abs/2009.08153) find
  27. Tomas Mikolov, Ilya Sutskever, Kai Chen, Greg Corrado, and Jeffrey Dean. 2013. Distributed representations of words and phrases and their compositionality. In Proceedings of the 26th International Conference on Neural Information Processing Systems - Volume 2, NIPS’13, page 3111–3119, Red Hook, NY,… find
  28. Teruko Mitamura, Zhengzhong Liu, and Eduard H. Hovy. 2016. Overview of TAC-KBP 2016 event nugget track. In Proceedings of the 2016 Text Analysis Conference, TAC 2016, Gaithersburg, Maryland, USA, November 14-15, 2016. NIST. find
  29. Thien Huu Nguyen, Adam Meyers, and Ralph Grishman. 2016. New york university 2016 system for KBP event nugget: A deep learning approach. In Proceedings of the 2016 Text Analysis Conference, TAC 2016, Gaithersburg, Maryland, USA, November 14-15, 2016. NIST. find
  30. Adam Paszke, S. Gross, Francisco Massa, A. Lerer, J. Bradbury, G. Chanan, T. Killeen, Z. Lin, N. Gimelshein, L. Antiga, Alban Desmaison, Andreas Köpf, E. Yang, Zach DeVito, Martin Raison, Alykhan Tejani, Sasank Chilamkurthy, B. Steiner, Lu Fang, Junjie Bai, and Soumith Chintala. 2019. Pytorch: An im… find our card
  31. Haoruo Peng, Yangqiu Song, and Dan Roth. 2016. Event detection and co-reference with minimal supervision. In Proceedings of the 2016 Conference on Empirical Methods in Natural Language Processing, pages 392–402, Austin, Texas. Association for Computational Linguistics. find our card
  32. Jeffrey Pennington, Richard Socher, and Christopher Manning. 2014. GloVe: Global vectors for word representation. In Proceedings of the 2014 Conference on Empirical Methods in Natural Language Processing (EMNLP), pages 1532–1543, Doha, Qatar. Association for Computational Linguistics. find
  33. M. Recasens, E. Hovy, and M. Martí. 2011. Identity, non-identity, and near-identity: Addressing the complexity of coreference. Lingua, 121:1138–1152. find
  34. Mark Sammons, Haoruo Peng, Yangqiu Song, Shyam Upadhyay, Chen-Tse Tsai, Pavankumar Reddy, Subhro Roy, and Dan Roth. 2015. find
  35. Illinois CCG TAC 2015 event nugget, entity discovery and linking, and slot filler validation systems. In Proceedings of the 2015 Text Analysis Conference, TAC 2015, Gaithersburg, Maryland, USA, November 16-17, 2015, 2015. NIST. find
  36. Nitish Srivastava, Geoffrey Hinton, Alex Krizhevsky, Ilya Sutskever, and Ruslan Salakhutdinov. 2014. Dropout: A simple way to prevent neural networks from overfitting. Journal of Machine Learning Research, 15(56):1929–1958. find
  37. Lucy Vanderwende, Michele Banko, and Arul Menezes. 2004. Event-centric summary generation. In Working notes of the Document Understanding Conference 2004. ACL. find
  38. Christopher Walker, Stephanie Strassel, Julie Medero, and Kazuaki Maeda. 2006. find
  39. Ace 2005 multilingual training corpus. Linguistic Data Consortium, Philadelphia, 57:45. find
  40. Thomas Wolf, Lysandre Debut, Victor Sanh, Julien Chaumond, Clement Delangue, Anthony Moi, Pier-ric Cistac, Tim Rault, Remi Louf, Morgan Funtow-icz, Joe Davison, Sam Shleifer, Patrick von Platen, Clara Ma, Yacine Jernite, Julien Plu, Canwen Xu, Teven Le Scao, Sylvain Gugger, Mariama Drame, Quentin Lh… find our card
  41. Thomas Wolf, Lysandre Debut, Victor Sanh, Julien Chaumond, Clement Delangue, Anthony Moi, Pier-ric Cistac, Tim Rault, Remi Louf, Morgan Funtow-icz, Joe Davison, Sam Shleifer, Patrick von Platen, Clara Ma, Yacine Jernite, Julien Plu, Canwen Xu, Teven Le Scao, Sylvain Gugger, Mariama Drame, Quentin Lh… find our card
  42. Train (\# Docs) 529 509 Dev (\# Docs) 30 139 Test (\# Docs) 40 169 *Table 9: Basic statistics of the datasets.* A.1 Symbolic Feature Predictors find

↑ back to top

This page in Markdown →