Elamur Research⌕
☰
home / papers / 2018 / Cross-Document, Cross-Language Event Coreference Annotation Using Event Hoppers

Cross-Document, Cross-Language Event Coreference Annotation Using Event Hoppers

Summary

Event coreference is usually annotated within single documents. Knowledge-base construction tasks such as Cold Start in TAC KBP, however, need events linked across many documents and several languages. LDC researchers extend the Rich ERE event hopper construct to cross-document and cross-lingual annotation. Event hoppers are a looser notion of coreference than ACE-style schemes. Mentions can be grouped when they intuitively describe the same happening, even if arguments differ in granularity or conflict, realis differs, or volition is described differently.

Starting from existing within-document Rich ERE annotation, annotators compared pairs of hoppers from different documents and judged whether they were coreferential. They used new guidelines, data processes and a dedicated interface. The resulting corpus covers 505 Chinese, English and Spanish documents and yields 389 cross-document event hoppers. It also includes many negative judgements, which are useful as training signal. The authors argue that the hopper concept transfers well beyond single documents, and they outline open challenges and future directions. The data went to DEFT and TAC KBP 2017 participants and is slated for the LDC catalog.

Key points

  • Coreference criteria: Hoppers are linked when the events are intuitively identical and share the same type and subtype.
    • Time and place need only match in general scope.
    • Arguments may mismatch or conflict, and realis (Actual / Other / Generic) may differ.
  • Ambiguous cases: If context cannot resolve the ambiguity, annotators leave the events unlinked. Examples include possible superset/subset arguments and repeated events of the same subtype, such as multiple elections.
  • Ontology: Rich ERE events cover 9 types and 38 subtypes (e.g., Conflict.Attack, Contact.Meet, Movement.TransportPerson).
  • Corpus size: 505 documents in three languages produced 389 cross-document event hoppers.
  • Distribution: Data were supplied to DEFT performers and 2017 NIST TAC KBP participants, with public LDC release planned afterward.

Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.

Fields: Linguistics & languages

Topics: Coreference resolution

Cited by

10 papers of our library cite it (of the 140673 whose bibliographies we read); 2 of them have a page here, 8 not yet.

Cite

The paper itself, not this page. BibTeX from the library's record of the work.

@inproceedings{song2018cross,
  title = {{Cross-Document, Cross-Language Event Coreference Annotation Using Event Hoppers}},
  author = {Song, Zhiyi and Bies, Ann and Mott, Justin and Li, Xuansong and Strassel, Stephanie and Caruso, Christopher},
  booktitle = {Proceedings of the Eleventh International Conference on Language Resources and Evaluation (LREC 2018)},
  year = {2018},
  doi = {10.63317/58cyairar5w5},
  url = {https://doi.org/10.63317/58cyairar5w5}
}

References

25 references, 5 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

  1. Aguilar, J., Beller, C., McNamee, P., Van Durme, B., Strassel, S., Song, Z., & Ellis, J. (2014). A comparison of the events and relations across ACE, ERE, TAC-KBP, and FrameNet annotation standards. In Proceedings of the Second Workshop on EVENTS: Definition, Detection, Coreference, and Representati… find our card
  2. Araki, J., Liu, Z., Hovy, E. H., & Mitamura, T. (2014). Detecting Subevent Structure for Event Coreference Resolution. In Proceedings of LREC 2014. find our card
  3. Bejan, AC. and Harabagiu, S. (2008). A Linguistic Resource for Discovering Event Structures and Resolving Event Coreference. In Proceedings of LREC 2008. find our card
  4. Bies, A., Song, Z., Getman, J., Ellis, J., Mott, J., Strassel, S., Palmer, M., Mitamura, T., Freedman, M., Ji, H., and O'Gorman, T. (2016). A Comparison of Event Representations in DEFT. In Proceedings of the Fourth Workshop on Events (pp. 27-36). find
  5. Cybulska, A., & Vossen, P. (2014a ). Using a sledgehammer to crack a nut? Lexical diversity and event coreference resolution. In Proceedings of LREC 2014. find our card
  6. Cybulska, A., & Vossen, P. (2014b). Guidelines for ECB+ annotation of events and their coreference. Technical Report NWR-2014-1, VU University Amsterdam. find
  7. DARPA. 2012. Broad Agency Announcement: Deep Exploration and Filtering of Text (DEFT). Defense Advanced Research Projects Agency, DARPA-BAA-12-47. find
  8. Ellis, J., Getman, J., Fore, D., Kuster, N., Song, Z., Bies, A., & Strassel, S. (2016). Overview of Linguistic Resources for the TAC KBP 2016 Evaluations: Methodologies and Results. In Proceedings of TAC KBP 2016 Workshop, National Institute of Standards and Technology. find
  9. Getman, J., Ellis, J., Strassel, S., Song, Z., & Tracey, J. (2018). Laying the Groundwork for Knowledge Base Population: Nine Years of Linguistic Resources for TAC KBP. In Proceedings of LREC 2018. find
  10. Girardi, C., Speranza, M., Sprugnoli, R., & Tonelli, S. (2014). CROMER: a Tool for Cross-Document Event and Entity Coreference. In Proceedings of LREC 2016. find
  11. Hong, Y., Zhang, T., O’Gorman, T., Horowit-Hendler, S., Ji, H., & Palmer, M. (2016). Building a Cross-document Event-Event Relation Corpus. LAW X, 1. find
  12. Ji, H., Grishman, R., Dang, H. T., Griffitt, K., & Ellis, J. (2010). Overview of the TAC 2010 Knowledge Base Population Track. In Third Text Analysis Conference (TAC 2010). find
  13. Lee, H., Recasens, M., Chang, A., Surdeanu, M., & Jurafsky, D. (2012). Joint entity and event coreference resolution across documents. In Proceedings of the 2012 Joint Conference on Empirical Methods in Natural Language Processing and Computational Natural Language Learning (pp. 489-500). Associatio… find our card
  14. Linguistic Data Consortium. (2005). ACE (Automatic Content Extraction) English Annotation Guidelines for Events Version 5.4.3. find
  15. Mott, J., Bies, A., Song, Z., & Strassel, S. (2016). Parallel Chinese-English Entities, Relations and Events Corpora. In Proceedings of LREC 2016. find
  16. NIST. (2016). TAC 2016 Cold Start KB Track. https://tac.nist.gov/2016/KBP/ColdStart/index.html. ↗
  17. March 2017. Accessed September, 2017. find
  18. NIST. (2017). TAC 2017 Cold Start KB Track. https://tac.nist.gov/2017/KBP/ColdStart/index.html. ↗
  19. May 2017. Accessed September, 2017. find
  20. Pradhan, S. S., Ramshaw, L., Weischedel, R., MacBride, J., & Micciulla, L. (2007). Unrestricted coreference: Identifying entities and events in OntoNotes. In Semantic Computing, 2007. find
  21. ICSC 2007. International Conference on (pp. 446-453). find
  22. IEEE. Pustejovsky, J., Castano, J.M., Ingria, R., Sauri, R., Gaizauskas, R.J., Setzer, A., Katz, G. and Radev, D.R. (2003). TimeML: Robust specification of event and temporal expressions in text. New directions in question answering, 3, 28-34. find
  23. Song, Z., Bies, A., Strassel, S., Riese, T., Mott, J., Ellis, J., Wright, J., Kulick, S., Ryant, N., and Ma, X. (2015). From Light to Rich ERE: Annotation of Entities, Relations, and Events. In Proceedings of the 3rd Workshop on Events at the NAACL-HLT. \#\# 10. find
  24. Language Resource References Walker, C., Strassel, S., Medero, J., and Maeda, K. (2006). find
  25. ACE 2005 Multilingual Training Corpus. Linguistic Data Consortium, LDC Catalog No.: LDC2006T06. find

↑ back to top

This page in Markdown → https://md.research.elamur.ai/free/en/papers/song-2018-cross-document-cross-language-event/