Elamur Research⌕
☰
home / papers / 2023 / What happens before and after: Multi-Event Commonsense in Event Coreference Resolution

What happens before and after: Multi-Event Commonsense in Event Coreference Resolution

Summary

Cross-document event coreference systems usually judge whether two mentions denote the same real-world event by comparing their wording and surrounding context. They therefore struggle when coreferring mentions look unrelated, such as "spent" and "hospitalized" in a hospital story. The authors supply temporal commonsense, meaning what typically happens before and after each event. They fine-tune GPT-3 (Davinci) into a multi-event inference engine. Given a sentence with several events and one marked target, it generates before/after inferences specific to that target. Existing single-event commonsense models tend to blend inferences for different events instead. Up to five inferences per relation are encoded with RoBERTa and pooled by attention, using one of two variants:

  • Intra-span: the mention attends to its own inferences.
  • Inter-span: the mention attends to the other mention's inferences.

The pooled vectors are added to the pairwise scorer of Cattan et al. (2021a). On ECB+ with gold mentions, both variants outperform the baseline, and intra-span works best. The study shows that generated commonsense knowledge can help link lexically divergent event mentions.

Key points

  • Results: Evaluated on ECB+ at topic level, with gold mentions and singletons removed. CoNLL F1 was 61.30 for the baseline, 62.05 for inter-span (+0.75) and 62.67 for intra-span (+1.37), averaged over 15 random seeds.
  • Architecture: Each mention gets a commonsense vector built from attention-weighted before and after vectors. It is concatenated with the mention span representations before pairwise scoring, followed by agglomerative clustering as in Cattan et al. (2021a).
  • Inference quality: Human evaluation found that 19.3% of GPT-3 inferences were not specific enough to the target event, which hurt performance.
  • Dataset limitation: Only about 11% of coreferring validation pairs in ECB+ were contextually dissimilar (cosine similarity below 0.9). This limits how much commonsense can help on this dataset.
  • Evaluation limitation: Using gold rather than predicted mentions likely overestimates real-world performance. The authors plan to add more commonsense relations and apply the approach to other discourse tasks.

Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.

Fields: Linguistics & languages

Topics: Coreference resolution

Cited by

15 papers of our library cite it (of the 140673 whose bibliographies we read); 5 of them have a page here, 10 not yet.

Cite

The paper itself, not this page. BibTeX from the library's record of the work. DOI 10.18653/v1/2023.eacl-main.125 · arXiv 2302.09715

@inproceedings{ravi2023what,
  title = {{What happens before and after: Multi-Event Commonsense in Event Coreference Resolution}},
  author = {Ravi, Sahithya and Tanner, Chris and Ng, Raymond and Shwartz, Vered},
  booktitle = {Proceedings of the 17th Conference of the European Chapter of the Association for Computational Linguistics},
  year = {2023},
  doi = {10.18653/v1/2023.eacl-main.125},
  eprint = {2302.09715},
  archivePrefix = {arXiv},
  url = {https://doi.org/10.18653/v1/2023.eacl-main.125}
}

References

52 references, 23 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

  1. Emily Allaway, Shuai Wang, and Miguel Ballesteros. 2021. Sequential cross-document coreference resolution. In Proceedings of the 2021 Conference on Empirical Methods in Natural Language Processing, pages 4659–4671, Online and Punta Cana, Dominican Republic. Association for Computational Linguistics. find our card
  2. Ateret Anaby-Tavor, Boaz Carmeli, Esther Goldbraich, Amir Kantor, George Kour, Segev Shlomov, N. Tepper, and Naama Zwerdling. 2020. Do not have enough data? deep learning to the rescue! In AAAI. find
  3. Amit Bagga and Breck Baldwin. 1998. Algorithms for scoring coreference chains. In LREC. find our card
  4. Shany Barhom, Vered Shwartz, Alon Eirew, Michael Bugert, Nils Reimers, and Ido Dagan. 2019. Revisiting joint modeling of cross-document entity and event coreference resolution. In Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics, pages 4179–4189, Florence, Ital… find our card
  5. Iz Beltagy, Matthew E. Peters, and Arman Cohan. 2020. Longformer: The long-document transformer. arXiv:2004.05150. find
  6. Antoine Bosselut, Hannah Rashkin, Maarten Sap, Chaitanya Malaviya, Asli Celikyilmaz, and Yejin Choi. 2019. COMET: Commonsense transformers for automatic knowledge graph construction. In Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics, pages 4762–4779, Florence… find our card
  7. Tom Brown, Benjamin Mann, Nick Ryder, Melanie Subbiah, Jared D Kaplan, Prafulla Dhariwal, Arvind Neelakantan, Pranav Shyam, Girish Sastry, Amanda Askell, Sandhini Agarwal, Ariel Herbert-Voss, Gretchen Krueger, Tom Henighan, Rewon Child, Aditya Ramesh, Daniel Ziegler, Jeffrey Wu, Clemens Winter, Chri… find
  8. Avi Caciularu, Arman Cohan, Iz Beltagy, Matthew Peters, Arie Cattan, and Ido Dagan. 2021. CDLM: Cross-document language modeling. In Findings of the Association for Computational Linguistics: EMNLP 2021, pages 2648–2662, Punta Cana, Dominican Republic. Association for Computational Linguistics. find
  9. Arie Cattan, Alon Eirew, Gabriel Stanovsky, Mandar Joshi, and Ido Dagan. 2021a. Cross-document coreference resolution over predicted mentions. In Findings of the Association for Computational Linguistics: ACL-IJCNLP 2021, pages 5100–5107, Online. Association for Computational Linguistics. find our card
  10. Arie Cattan, Alon Eirew, Gabriel Stanovsky, Mandar Joshi, and Ido Dagan. 2021b. Realistic evaluation principles for cross-document coreference resolution. ArXiv, abs/2106.04192. find our card
  11. Tuhin Chakrabarty, Yejin Choi, and Vered Shwartz. 2022. It’s not rocket science: Interpreting figurative language in narratives. Transactions of the Association for Computational Linguistics, 10:589–606. find our card
  12. Tuhin Chakrabarty, Debanjan Ghosh, Smaranda Muresan, and Nanyun Peng. 2020. Rˆ3: Reverse, retrieve, and rank for sarcasm generation with commonsense knowledge. In Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics, pages 7976–7986, Online. Association for Computa… find our card
  13. Nathanael Chambers and Dan Jurafsky. 2008. Unsupervised learning of narrative event chains. In Proceedings of ACL-08: HLT, pages 789–797, Columbus, Ohio. Association for Computational Linguistics. find
  14. Prafulla Kumar Choubey and Ruihong Huang. 2017. Event coreference resolution by iteratively unfolding inter-dependencies among events. In Proceedings of the 2017 Conference on Empirical Methods in Natural Language Processing, pages 2124–2133, Copenhagen, Denmark. Association for Computational Lingui… find our card
  15. Agata Cybulska and Piek Vossen. 2014. Using a sledgehammer to crack a nut? lexical diversity and event coreference resolution. In Proceedings of the Ninth International Conference on Language Resources and Evaluation (LREC’14), pages 4545–4552, Reykjavik, Iceland. European Language Resources Associa… find our card
  16. Saadia Gabriel, Chandra Bhagavatula, Vered Shwartz, Ronan Le Bras, Maxwell Forbes, and Yejin Choi. 2021a. Paragraph-level commonsense transformers with recurrent memory. In AAAI. find
  17. Saadia Gabriel, Antoine Bosselut, Jeff Da, Ari Holtzman, Jan Buys, Kyle Lo, Asli Celikyilmaz, and Yejin Choi. 2021b. Discourse understanding and factual consistency in abstractive summarization. In Proceedings of the 16th Conference of the European Chapter of the Association for Computational Lingui… find
  18. Deepanway Ghosal, Navonil Majumder, Rada Mihalcea, and Soujanya Poria. 2021. STaCK: Sentence ordering with temporal commonsense knowledge. In Proceedings of the 2021 Conference on Empirical Methods in Natural Language Processing, pages 8676–8686, Online and Punta Cana, Dominican Republic. Associatio… find
  19. William Held, Dan Iter, and Dan Jurafsky. 2021. Focus on what matters: Applying discourse coherence theory to cross document coreference. In Proceedings of the 2021 Conference on Empirical Methods in Natural Language Processing, pages 1406–1417, Online and Punta Cana, Dominican Republic. Association… find our card
  20. Ari Holtzman, Jan Buys, Li Du, Maxwell Forbes, and Yejin Choi. 2020. The curious case of neural text degeneration. In International Conference on Learning Representations. find
  21. Jena D. Hwang, Chandra Bhagavatula, Ronan Le Bras, Jeff Da, Keisuke Sakaguchi, Antoine Bosselut, and Yejin Choi. 2021. find
  22. Comet-atomic 2020: On symbolic and neural commonsense knowledge graphs. In AAAI. find our card
  23. William R Kearns, Neha Kaura, Myra Divina, Cuong Vo, Dong Si, Teresa Ward, and Weichao Yuwen. 2020. A wizard-of-oz interface and persona-based methodology for collecting health counseling dialog. In Extended Abstracts of the 2020 CHI Conference on Human Factors in Computing Systems, pages 1–9. find
  24. Varun Kumar, Ashutosh Choudhary, and Eunah Cho. 2020. Data augmentation using pre-trained transformer models. In Proceedings of the 2nd Workshop on Life-long Learning for Spoken Language Systems, pages 18–26, Suzhou, China. Association for Computational Linguistics. find our card
  25. Tuan Lai, Heng Ji, Trung Bui, Quan Hung Tran, Franck Dernoncourt, and Walter Chang. 2021. A context-dependent gated module for incorporating symbolic semantics into event coreference resolution. In Proceedings of the 2021 Conference of the North American Chapter of the Association for Computational… find our card
  26. Yinhan Liu, Myle Ott, Naman Goyal, Jingfei Du, Mandar Joshi, Danqi Chen, Omer Levy, Mike Lewis, Luke Zettlemoyer, and Veselin Stoyanov. 2019. Roberta: A robustly optimized bert pretraining approach. ArXiv, abs/1907.11692. find
  27. Xiaoqiang Luo. 2005. On coreference resolution performance metrics. In Proceedings of Human Language Technology Conference and Conference on Empirical Methods in Natural Language Processing, pages 25–32, Vancouver, British Columbia, Canada. Association for Computational Linguistics. find our card
  28. Bodhisattwa Prasad Majumder, Harsh Jhamtani, Taylor Berg-Kirkpatrick, and Julian McAuley. 2020. Like hiking? you probably enjoy nature: Persona-grounded dialog with commonsense expansions. In Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing (EMNLP), pages 9194–9… find
  29. Gary Marcus and Ernest Davis. 2020. Experiments testing gpt-3’s ability at commonsense reasoning: results. find
  30. Yehudit Meged, Avi Caciularu, Vered Shwartz, and Ido Dagan. 2020. Paraphrasing vs coreferring: Two sides of the same coin. In Findings of the Association for Computational Linguistics: EMNLP 2020, pages 4897–4907, Online. Association for Computational Linguistics. find our card
  31. Nafise Sadat Moosavi and Michael Strube. 2016. Which coreference evaluation metric do you trust? a proposal for a link-based entity aware metric. In Proceedings of the 54th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 632–642, Berlin, Germany. Associ… find our card
  32. Nasrin Mostafazadeh, Nathanael Chambers, Xiaodong He, Devi Parikh, Dhruv Batra, Lucy Vanderwende, Pushmeet Kohli, and James Allen. 2016. A corpus and cloze evaluation for deeper understanding of commonsense stories. In Proceedings of the 2016 Conference of the North American Chapter of the Associati… find
  33. Yannis Papanikolaou and Andrea Pierleoni. 2020. Dare: Data augmented relation extraction with gpt-2. ArXiv, abs/2004.13845. — [arXiv:2004.13845](https://arxiv.org/abs/2004.13845) find
  34. Jae Sung Park, Chandra Bhagavatula, Roozbeh Mot-taghi, Ali Farhadi, and Yejin Choi. 2020. Visual-comet: Reasoning about the dynamic context of a still image. In ECCV. find
  35. Karl Pichotta and Raymond Mooney. 2014. Statistical script learning with multi-argument events. In Proceedings of the 14th Conference of the European Chapter of the Association for Computational Linguistics, pages 220–229, Gothenburg, Sweden. Association for Computational Linguistics. find
  36. Rachel Rudinger, Pushpendre Rastogi, Francis Ferraro, and Benjamin Van Durme. 2015. Script induction as language modeling. In Proceedings of the 2015 Conference on Empirical Methods in Natural Language Processing, pages 1681–1686, Lisbon, Portugal. Association for Computational Linguistics. find
  37. Maarten Sap, Ronan LeBras, Emily Allaway, Chandra Bhagavatula, Nicholas Lourie, Hannah Rashkin, Brendan Roof, Noah A Smith, and Yejin Choi. 2019. Atomic: An atlas of machine commonsense for if-then reasoning. In AAAI. find
  38. Roger C Schank and Robert P Abelson. 1975. Scripts, plans, and knowledge. In IJCAI, volume 75, pages 151–157. find
  39. Timo Schick and Hinrich Schütze. 2021. Generating datasets with pretrained language models. ArXiv, abs/2104.07540. find our card
  40. Shyam Upadhyay, Nitish Gupta, Christos Christodoulopoulos, and Dan Roth. 2016. Revisiting the evaluation for cross document event coreference. In Proceedings of COLING 2016, the 26th International Conference on Computational Linguistics: Technical Papers, pages 1949–1958, Osaka, Japan. find our card
  41. The COLING 2016 Organizing Committee. find
  42. Marc Vilain, John Burger, John Aberdeen, Dennis Connolly, and Lynette Hirschman. 1995. A model-theoretic coreference scoring scheme. In Sixth Message Understanding Conference (MUC-6): Proceedings of a Conference Held in Columbia, Maryland, November 6-8, 1995. find our card
  43. Shuohang Wang, Yang Liu, Yichong Xu, Chenguang Zhu, and Michael Zeng. 2021. Want to reduce labeling cost? GPT-3 can help. In Findings of the Association for Computational Linguistics: EMNLP 2021, pages 4195–4205, Punta Cana, Dominican Republic. Association for Computational Linguistics. find our card
  44. Peter West, Chandrasekhar Bhagavatula, Jack Hessel, Jena D. Hwang, Liwei Jiang, Ronan Le Bras, Ximing Lu, Sean Welleck, and Yejin Choi. 2022. Symbolic knowledge distillation: from general language models to commonsense models. In NAACL. find our card
  45. Sarah Wiegreffe, Jack Hessel, Swabha Swayamdipta, Mark Riedl, and Yejin Choi. 2022. Reframing human-AI collaboration for generating free-text explanations. In Proceedings of the 2022 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologie… find
  46. Travis Wolfe, Mark Dredze, and Benjamin Van Durme. 2015. Predicate argument alignment using a global coherence model. In Proceedings of the 2015 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, pages 11–20, Denver, Colorado. Asso… find
  47. Nishant Yadav, Nicholas Monath, Rico Angell, and Andrew McCallum. 2021a. Event and entity coreference using trees to encode uncertainty in joint decisions. In Proceedings of the Fourth Workshop on Computational Models of Reference, Anaphora and Coreference, pages 100–110, Punta Cana, Dominican Repub… find
  48. Rohan Kumar Yadav, Lei Jiao, Ole-Christoffer Granmo, and Morten Goodwin. 2021b. Enhancing interpretable clauses semantically using pretrained word representation. In Proceedings of the Fourth BlackboxNLP Workshop on Analyzing and Interpreting Neural Networks for NLP, pages 265–274, Punta Cana, Domin… find
  49. Bishan Yang, Claire Cardie, and Peter Frazier. 2015. A hierarchical distance-dependent Bayesian model for event coreference resolution. Transactions of the Association for Computational Linguistics, 3:517– 528. find
  50. Xiaohan Yang, Eduardo Peynetti, Vasco Meerman, and Chris Tanner. 2022. What GPT knows about who is who. In Proceedings of the Third Workshop on Insights from Negative Results in NLP, pages 75– 81, Dublin, Ireland. Association for Computational Linguistics. find
  51. Xiaodong Yu, Wenpeng Yin, and Dan Roth. 2020. Paired representation learning for event and entity coreference. ArXiv, abs/2010.12808. — [arXiv:2010.12808](https://arxiv.org/abs/2010.12808) find
  52. Yutao Zeng, Xiaolong Jin, Saiping Guan, Jiafeng Guo, and Xueqi Cheng. 2020. Event coreference resolution with their paraphrases and argument-aware embeddings. In Proceedings of the 28th International Conference on Computational Linguistics, pages 3084–3094, Barcelona, Spain (Online). International C… find our card

↑ back to top