Elamur Research⌕
☰
home / papers / 2016 / Richer Event Description: Integrating event coreference with temporal, causal and bridging annotation

Richer Event Description: Integrating event coreference with temporal, causal and bridging annotation

Summary

Event phenomena such as coreference, temporal ordering, causality and subevent structure are usually annotated in separate corpora. Each covers only one or two layers, which makes it hard to train systems that model these phenomena jointly. The authors introduce the Richer Event Description (RED) corpus, where all these layers are annotated together within each document. It covers entities, events, times, identity and bridging coreference, and temporal, causal and subevent links. Annotation runs in two passes. In the first pass, annotators mark events, entities and temporal expressions, plus attributes such as modality, polarity and DocTimeRel (each event's relation to document time). Entity coreference and bridging are also done in this pass. The second pass starts only after the first is adjudicated, so that event coreference and event-event relations are built on vetted markables. Because two events can be linked in only one way, the guidelines need sharp boundaries between relation types. Markable and feature agreement is high, while agreement on event-event relations is clearly lower. An error analysis groups the remaining disagreements by cause. The resource supports joint learning and storyline-oriented work that goes beyond spotting isolated event mentions.

Key points

  • Corpus size: RED has 95 news and discussion-forum documents (54,287 tokens) with 8,731 events, 1,127 temporal expressions and 10,320 entity markables. It also has 2,390 identity chains, 1,863 bridging links and 4,969 event-event relations.
  • Annotation scheme: The scheme follows Thyme-TimeML (itself derived from ISO-TimeML). Only head words are marked, and singleton mentions are kept. IDENTITY links require the same modality and polarity, and other cases go to SET/MEMBER, PART/WHOLE or general BRIDGING.
  • Agreement: Agreement was measured on a 55-document subset. Event detection reaches F1 86.1 between annotators and 93.0 against gold. Entity coreference reaches CoNLL F1 70.4 between annotators and 81.8 against gold.
  • Event-event relations: Agreement on these relations is much weaker. CONTAINS is the most reliable at about 64 F1 between annotators, and most other relation types fall around 20–30. About 18% of event chains take part in two or more relations, and CONTAINS-SUBEVENT links often span long distances.
  • Error analysis: The authors examined 60 relations that were discarded during adjudication. Presupposition cases were the most common cause (12), followed by modality (8), idiomatic expressions (8), containment attachment (6), relations inferrable from document time (5) and resultatives (4).

Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.

Fields: Linguistics & languages

Topics: Coreference resolution

Cited by

94 papers of our library cite it (of the 140673 whose bibliographies we read); 10 of them have a page here, 84 not yet.

Cite

The paper itself, not this page. BibTeX from the library's record of the work. DOI 10.18653/v1/w16-5706

@inproceedings{ogorman2016richer,
  title = {{Richer Event Description: Integrating event coreference with temporal, causal and bridging annotation}},
  author = {O'Gorman, Tim and Wright-Bettner, Kristin and Palmer, Martha},
  booktitle = {Proceedings of the 2nd Workshop on Computing News Storylines (CNS 2016)},
  year = {2016},
  doi = {10.18653/v1/w16-5706},
  url = {https://doi.org/10.18653/v1/w16-5706}
}

References

34 references, 10 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

  1. Laura Banarescu, Claire Bonial, Shu Cai, Madalina Georgescu, Kira Griffitt, Ulf Hermjakob, Kevin Knight, Philipp Koehn, Martha Palmer, and Nathan Schneider. 2013. Abstract meaning representation for sembanking. In Proc. of the Linguistic Annotation Workshop and Interoperability with Discourse. find our card
  2. Steven Bethard, William J Corvey, Sara Klingenstein, and James H Martin. 2008. Building a Corpus of Temporal-Causal Structure. In LREC. find
  3. Steven John Bethard. 2007. Finding event, temporal and causal structure in text: A machine learning approach. find
  4. Lynn Carlson, Daniel Marcu, and Mary Ellen Okurowski. 2003. Building a discourse-tagged corpus in the framework of rhetorical structure theory. In Current and new directions in discourse and dialogue, pages 85–112. find our card
  5. Springer. Herbert H Clark. 1975. Bridging. In Proceedings of the 1975 workshop on Theoretical issues in natural language processing, pages 169–174. Association for Computational Linguistics. find
  6. Agata Cybulska and Piek Vossen. 2014. Using a sledgehammer to crack a nut? Lexical diversity and event coreference resolution. In LREC, pages 4545–4552. find our card
  7. Jennifer D’Souza and Vincent Ng. 2013. Classifying temporal relations with rich linguistic knowledge. In HLT-NAACL, pages 918–927. find
  8. Jesse Dunietz, Lori Levin, and Jaime Carbonell. 2015. Annotating Causal Language Using Corpus Lexicography of Constructions. In The 9th Linguistic Annotation Workshop held in conjuncion with NAACL 2015, page 188. find
  9. Charles J. Fillmore and Collin F. Baker. 2000. FrameNet: Frame Semantics Meets the Corpus. In Poster presentation, 74th Annual Meeting of the Linguistic Society of America, January. Goran Glava and Jan najder. 2014. Constructing Coherent Event Hierarchies from News Stories. TextGraphs-9, page 34. find
  10. Goran Glava, Jan Snajder, Parisa Kordjamshidi, and Marie-Francine Moens. 2014. HiEve: A Corpus for Extracting Event Hierarchies from News Stories. In Proceedings of 9th language resources and evaluation conference. find
  11. Jerry R Hobbs. 2005. Toward a useful concept of causality for lexical semantics. Journal of Semantics, 22(2):181–209. find
  12. Yu Hong, Tongtao Zhang, Sharone Horowit-Hendler, Heng Ji, Tim O’Gorman, and Martha Palmer. 2016. Building a Cross-document Event-Event Relation Corpus. In Proceedings of LREC 2016. find
  13. Rei Ikuta, William F. Styler IV, Mariah Hamang, Tim OGorman, and Martha Palmer. 2014. Challenges of Adding Causation to Richer Event Descriptions. Proceedings of the 2nd Workshop on Events, page 12. find
  14. Jonathan K Kummerfeld and Dan Klein. 2013. Error-driven analysis of challenges in coreference resolution. In Proceedings of the 2013 Conference on Empirical Methods in Natural Language Processing, Seattle, Wash, pages 18–21. find our card
  15. Heeyoung Lee, Marta Recasens, Angel Chang, Mihai Surdeanu, and Dan Jurafsky. 2012. Joint entity and event coreference resolution across documents. In Proceedings of the 2012 Joint Conference on Empirical Methods in Natural Language Processing and Computational Natural Language Learning, pages 489–50… find our card
  16. Shasha Liao and Ralph Grishman. 2010. Using document level cross-event inference to improve event extraction. In Proceedings of the 48th Annual Meeting of the Association for Computational Linguistics, pages 789–797. Association for Computational Linguistics. find
  17. Kenneth C Litkowski and Orin Hargraves. 2005. The preposition project. In Proceedings of the Second ACL-SIGSEM Workshop on the Linguistic Dimensions of Prepositions and their Use in Computational Linguistics Formalisms and Applications, pages 171–179. find
  18. Anne-Lyse Minard, Manuela Speranza, Ruben Urizar, Begona Altuna, Marieke van Erp, Anneleen Schoen, and Chantal van Son. 2016. MEANTIME, the News-Reader multilingual event and time corpus. Proceedings of LREC2016. find
  19. Paramita Mirza, Rachele Sprugnoli, Sara Tonelli, and Manuela Speranza. 2014. Annotating causality in the TempEval-3 corpus. In Proceedings of the EACL 2014 Workshop on Computational Approaches to Causality in Language (CAtoCL), pages 10–19. find
  20. Nasrin Mostafazadeh, Alyson Grealish, Nathanael Chambers, James Allen, and Lucy Vanderwende. 2016. CaTeRS: Causal and Temporal Relation Scheme for Semantic Annotation of Event Structures. In Proceedings of the 4th Workshop on Events. find
  21. M. Palmer, P. Kingsbury, and D. Gildea. 2005. The proposition bank: An annotated corpus of semantic roles. Computational Linguistics, 31(1):71–106. find our card
  22. Haoruo Peng, Kai-Wei Chang, and Dan Roth. 2015. A joint framework for coreference resolution and mention head detection. find our card
  23. CoNLL 2015, 51:12. Massimo Poesio, Renata Vieira, and Simone Teufel. 1997. Resolving bridging references in unrestricted text. In Proceedings of a Workshop on Operational Factors in Practical, Robust Anaphora Resolution for Unrestricted Texts, pages 1–6. Association for Computational Linguistics. find
  24. Sameer Pradhan, Xiaoqiang Luo, Marta Recasens, Eduard Hovy, Vincent Ng, and Michael Strube. 2014. find
  25. Scoring coreference partitions of predicted mentions: A reference implementation. In Proceedings of the 52nd Annual Meeting of the Association for Computational Linguistics (Short Papers), pages 30–35. Rashmi Prasad, Nikhil Dinesh, Alan Lee, Eleni Milt-sakaki, Livio Robaldo, Aravind K Joshi, and Bon… find our card
  26. Citeseer. J. Pustejovsky and A. Stubbs. 2011. Increasing informativeness in temporal annotation. In ACL HLT 2011, page 152. find our card
  27. James Pustejovsky, Jos M Castano, Robert Ingria, Roser Sauri, Robert J Gaizauskas, Andrea Setzer, Graham Katz, and Dragomir R Radev. 2003. TimeML: Robust specification of event and temporal expressions in text. New directions in question answering, 3:28–34. find
  28. Mehwish Riaz and Roxana Girju. 2013. Toward a better understanding of causality between verbal events: Extraction and analysis of the causal power of verb-verb associations. In Proceedings of the annual SIGdial Meeting on Discourse and Dialogue (SIGDIAL). find
  29. Nathan Schneider, Vivek Srikumar, Jena D Hwang, and Martha Palmer. 2015. A hierarchy with, of, and for preposition supersenses. In The 9th Linguistic Annotation Workshop held in conjuncion with NAACL 2015, page 112. find our card
  30. Zhiyi Song, Ann Bies, Stephanie Strassel, Tom Riese, Justin Mott, Joe Ellis, Jonathan Wright, Seth Kulick, Neville Ryant, and Xiaoyi Ma. 2015. From light to rich ere: annotation of entities, relations, and events. In Proceedings of the 3rd Workshop on EVENTS at the NAACL-HLT, pages 89–98. find
  31. Vivek Srikumar and Dan Roth. 2013. Modeling semantic relations expressed by prepositions. Transactions of the Association for Computational Linguistics, 1:231– 242. William F. Styler IV, Steven Bethard, Sean Finan, Martha Palmer, Sameer Pradhan, Piet C. de Groen, Brad Erickson, Timothy Miller, Chen… find
  32. Naushad UzZaman and James F Allen. 2011. Temporal evaluation. In Proceedings of the 49th Annual Meeting of the Association for Computational Linguistics: Human Language Technologies: short papers-Volume 2, pages 351–356. Association for Computational Linguistics. find
  33. Piek Vossen, Tommaso Caselli, and Yiota Kontzopoulou. 2015. Storylines for structuring massive streams of news. In Proceedings of the First Workshop on Computing News Storylines, pages 40–49. find
  34. Ralph Weischedel, Eduard Hovy, Mitchell Marcus, Martha Palmer, Robert Belvin, Sameer Pradhan, Lance Ramshaw, and Nianwen Xue. 2011. OntoNotes: A large training corpus for enhanced processing. Handbook of Natural Language Processing and Machine Translation. Springer. find

↑ back to top

This page in Markdown → https://md.research.elamur.ai/free/en/papers/gorman-2016-richer-event-description-integrating-event/