Elamur Research⌕
☰
home / papers / 2022 / Pairwise Representation Learning for Event Coreference

Pairwise Representation Learning for Event Coreference

Summary

Event coreference resolution decides whether two event mentions describe the same real-world event. Earlier systems usually encoded each mention on its own and then classified the pair. As a result, a mention's vector never reflected its counterpart's context, which is a serious gap when the mentions come from different documents. These systems also merged all event arguments into one undifferentiated vector. Yu, Yin and Roth propose PAIRWISE RL. The sentences containing both mentions are joined and fed through RoBERTa together, so tokens from each sentence interact with the other's from the start. The model then adds structured argument evidence. For arg0, arg1, location and time, a dedicated MLP converts the paired argument vectors into a single scalar, so one clear mismatch (such as two unrelated cities) can carry decisive weight. A final classifier scores the pair, and agglomerative clustering builds coreference chains. Despite its simplicity, the method outperforms more complex state-of-the-art systems on cross-document ECB+ and within-document KBP17. This shows that jointly encoding mention pairs is a strong and inexpensive approach.

Key points

  • Architecture: RoBERTa encodes the concatenated sentence pair, and the token vectors of each span are summed. For every role (trigger, arg0, arg1, loc, time), the pair is represented as [v_i, v_j, v_i ∘ v_j], and a missing argument becomes a zero vector. Two four-layer MLPs are used, one for the argument scalars and one for the binary coreferent/not decision. For entity coreference, only the span-pair part is used.
  • ECB+ setup: The data follows the Cybulska and Vossen (2015) topic split. The training set has 25 topics, 574 documents and 3,808 event mentions; the test set (topics 36–45) has 10 topics, 206 documents and 1,780 event mentions. Gold mentions are used, and entity coreference is evaluated as well.
  • Pipeline details: Topic clustering is reused from Barhom et al. (2019), with gold topics for training and predicted topics at inference. Clusters are merged by their average pair score, with the threshold tuned on dev data.
  • Error analysis: On 50 sampled errors, joint encoding mainly helps with long sentences. The remaining errors call for cross-sentence context or world knowledge, such as matching time expressions or geographic relations. Some errors appear to be annotation mistakes. Because arguments come from SRL, they are confined to the mention's own sentence.
  • Availability: The code is released via CogComp (cogcomp.org). Zeng et al. (2020) is noted as concurrent work with a similar idea.

Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.

Fields: Linguistics & languages

Topics: Coreference resolution

Cited by

19 papers of our library cite it (of the 140673 whose bibliographies we read); 6 of them have a page here, 13 not yet.

Cite

The paper itself, not this page. BibTeX from the library's record of the work. DOI 10.18653/v1/2022.starsem-1.6

@inproceedings{yu2022pairwise,
  title = {{Pairwise Representation Learning for Event Coreference}},
  author = {Yu, Xiaodong and Yin, Wenpeng and Roth, Dan},
  booktitle = {Proceedings of the 11th Joint Conference on Lexical and Computational Semantics},
  year = {2022},
  doi = {10.18653/v1/2022.starsem-1.6},
  url = {https://doi.org/10.18653/v1/2022.starsem-1.6}
}

References

18 references, 10 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

  1. Dzmitry Bahdanau, Kyunghyun Cho, and Yoshua Bengio. 2015. Neural machine translation by jointly learning to align and translate. In ICLR. find our card
  2. Karl Moritz Hermann, Tomás Kociský, Edward Grefenstette, Lasse Espeholt, Will Kay, Mustafa Suleyman, and Phil Blunsom. 2015. Teaching machines to read and comprehend. In NeurIPS, pages 1693–1701. find our card
  3. Shany Barhom, Vered Shwartz, Alon Eirew, Michael Bugert, Nils Reimers, and Ido Dagan. 2019. Revisiting joint modeling of cross-document entity and event coreference resolution. In Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics, pages 4179–4189, Florence, Ital… find our card
  4. Yin Jou Huang, Jing Lu, Sadao Kurohashi, and Vincent Ng. 2019. Improving event coreference resolution by learning argument compatibility from unlabeled data. In Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technolog… find
  5. Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N. Gomez, Lukasz Kaiser, and Illia Polosukhin. 2017. Attention is all you need. In NeurIPS, pages 5998–6008. find
  6. Kian Kenyon-Dean, Jackie Chi Kit Cheung, and Doina Precup. 2018. Resolving event coreference with supervised representation learning and clustering-oriented regularization. In Proceedings of the Seventh Joint Conference on Lexical and Computational Semantics, pages 1–10, New Orleans, Louisiana. Asso… find our card
  7. Wenpeng Yin and Hinrich Schütze. 2018. Attentive convolution: Equipping cnns with rnn-style attention mechanisms. TACL, 6:687–702. find
  8. Yutao Zeng, Xiaolong Jin, Saiping Guan, Jiafeng Guo, and Xueqi Cheng. 2020. Event coreference resolution with their paraphrases and argument-aware embeddings. In Proceedings of COLING, pages 3084– 3094. find our card
  9. Heeyoung Lee, Marta Recasens, Angel Chang, Mihai Surdeanu, and Dan Jurafsky. 2012. Joint entity and event coreference resolution across documents. In Proceedings of the 2012 Joint Conference on Empirical Methods in Natural Language Processing and Computational Natural Language Learning, pages 489–50… find our card
  10. Kenton Lee, Luheng He, Mike Lewis, and Luke Zettlemoyer. 2017. End-to-end neural coreference resolution. In Proceedings of the 2017 Conference on Empirical Methods in Natural Language Processing, pages 188–197, Copenhagen, Denmark. Association for Computational Linguistics. find our card
  11. Jiwei Li, Minh-Thang Luong, and Dan Jurafsky. 2015. A hierarchical neural autoencoder for paragraphs and documents. In ACL, pages 1106–1115. find
  12. Yinhan Liu, Myle Ott, Naman Goyal, Jingfei Du, Mandar Joshi, Danqi Chen, Omer Levy, Mike Lewis, Luke Zettlemoyer, and Veselin Stoyanov. 2019. Roberta: A robustly optimized bert pretraining approach. arXiv preprint arXiv:1907.11692. find
  13. Yaojie Lu, Hongyu Lin, Jialong Tang, Xianpei Han, and Le Sun. 2020. End-to-end neural event coreference resolution. arXiv preprint arXiv:2009.08153. — [arXiv:2009.08153](https://arxiv.org/abs/2009.08153) find
  14. Ramesh Nallapati, Bowen Zhou, Cícero Nogueira dos Santos, Çaglar Gülçehre, and Bing Xiang. 2016. Abstractive text summarization using sequence-to-sequence rnns and beyond. In CoNLL, pages 280– 290. ACL. find
  15. Haoruo Peng, Yangqiu Song, and Dan Roth. 2016. Event detection and co-reference with minimal supervision. In Proceedings of the 2016 Conference on Empirical Methods in Natural Language Processing, pages 392–402, Austin, Texas. Association for Computational Linguistics. find our card
  16. Tim Rocktäschel, Edward Grefenstette, Karl Moritz Hermann, Tomás Kociský, and Phil Blunsom. 2016. Reasoning about entailment with neural attention. In ICLR. find our card
  17. Peter J Rousseeuw. 1987. Silhouettes: a graphical aid to the interpretation and validation of cluster analysis. Journal of computational and applied mathematics, 20:53–65. find
  18. Peng Shi and Jimmy Lin. 2019. Simple bert models for relation extraction and semantic role labeling. arXiv preprint arXiv:1904.05255. find our card

↑ back to top