Elamur Research⌕
☰
home / papers / 2013 / Generating Coherent Event Schemas at Scale

Generating Coherent Event Schemas at Scale

Summary

The authors automatically induce event schemas. Such a schema describes a typical situation through a set of participants (actors) and the relations between them. The earlier approach of Chambers and Jurafsky (2009) often produced incoherent schemas: unrelated events ended up in the same schema, and different roles were merged into a single actor. This happened because subject–verb and verb–object pairs were modeled separately there. This paper instead uses full triples (Arg1, Relation, Arg2) extracted by the Open IE system OLLIE from 1.8 million New York Times articles in the Gigaword corpus. Arguments are reduced to their head nouns and given semantic types from WordNet and Stanford NER. Coreferent mentions are found with Stanford Co-reference. Counts of how often triples co-occur in documents are collected in the Rel-grams database, a relational analogue of an n-gram language model. On a graph built from this database, a PageRank-based algorithm ranks the triples and merges arguments into actors. Evaluators on Amazon Mechanical Turk judged the new schemas substantially more coherent than those of Chambers. The paper's value is that the method scales and is domain-independent, and that both resources have been made publicly available.

Key points

  • Evaluators on Amazon Mechanical Turk judged 92% of the new schemas topically coherent, versus 82% for Chambers. Actors with a consistent role: 81% versus 59%. Correct assertions: 94% versus 61%.
  • The Rel-grams database contains about 320K triples with frequency ≥ 3 and 1.1M entries in the pair table (BigramCounts). It also separately records when the arguments of two triples coincide (E11–E22). Probabilities are estimated with smoothing δ = 0.05.
  • Arguments are assigned 29 semantic types from WordNet (person, organization, drug, illness, etc.). Normalizing arguments and relations increased tenfold the number of Rel-grams that pass the minimum frequency threshold.
  • More than 2K schemas were produced. The number of actors in them is not limited to two, and relation phrases keep their prepositions. The schemas and the Rel-grams database can be downloaded at relgrams.cs.washington.edu.
  • Errors in the new schemas come mostly from inconsistent actors (56% of incorrect triples) and from Open IE extraction failures (47%). For Chambers, 80% of incorrect triples involve a subject–object mismatch.

Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.

Fields: Linguistics & languages

Cited by

64 papers of our library cite it (of the 140673 whose bibliographies we read); 4 of them have a page here, 60 not yet.

Cite

The paper itself, not this page. BibTeX from the library's record of the work.

@inproceedings{balasubramanian2013generating,
  title = {{Generating Coherent Event Schemas at Scale}},
  author = {Balasubramanian, Niranjan and Soderland, Stephen and Mausam and Etzioni, Oren},
  booktitle = {Proceedings of the 2013 Conference on Empirical Methods in Natural Language Processing},
  year = {2013},
  doi = {10.18653/v1/d13-1178},
  url = {https://doi.org/10.18653/v1/d13-1178}
}

References

23 references, 4 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

  1. Eneko Agirre and Aitor Soroa. 2009. Personalizing pagerank for word sense disambiguation. In 12th Conference of the European Chapter of the Association for Computational Linguistics (EACL), pages 33–41. find
  2. ARPA. 1991. Proc. 3rd Message Understanding Conf. Morgan Kaufmann. find
  3. ARPA. 1998. Proc. 7th Message Understanding Conf. Morgan Kaufmann. find
  4. Niranjan Balasubramanian, Stephen Soderland, Oren Etzioni, et al. 2012. Rel-grams: a probabilistic model of relations in text. In Proceedings of the Joint Workshop on Automatic Knowledge Base Construction and Web-scale Knowledge Extraction, pages 101–105. Association for Computational Linguistics. find
  5. Sergey Brin and Lawrence Page. 1998. The anatomy of a large-scale hypertextual web search engine. Computer Networks, 30(1-7):107–117. find our card
  6. N. Chambers and D. Jurafsky. 2008. Unsupervised learning of narrative event chains. In Proceedings of ACL-08: HLT. find
  7. N. Chambers and D. Jurafsky. 2009. Unsupervised learning of narrative schemas and their participants. In Proceedings of ACL. find
  8. N. Chambers and D. Jurafsky. 2010. A database of narrative schemas. In Proceedings of LREC. find
  9. N. Chambers and D. Jurafsky. 2011. Template-based information extraction without the templates. In Proceedings of ACL. find
  10. J. Cheung, H. Poon, and L. Vandervende. 2013. Probabilistic frame induction. In Proceedings of NAACL HLT. find
  11. I. Dagan, O. Glickman, and B. Magnini. 2005. The PASCAL Recognising Textual Entailment Challenge. Proceedings of the PASCAL Challenges Workshop on Recognising Textual Entailment, pages 1–8. find
  12. G. Doddington, A. Mitchell, M. Przybocki, L. Ramshaw, S. Strassel, , and R. Weischedel. 2004. The automatic content extraction (ACE) program-tasks, data, and evaluation. In Procs. of LREC. find our card
  13. Jenny Rose Finkel, Trond Grenager, and Christopher Manning. 2005. Incorporating non-local information into information extraction systems by gibbs sampling. In Proceedings of the 43rd Annual Meeting on Association for Computational Linguistics, pages 363–370. Association for Computational Linguistic… find
  14. Antske Fokkens, Marieke van Erp, Piek Vossen, Sara Tonelli, Willem Robert van Hage, BV SynerScope, Luciano Serafini, Rachele Sprugnoli, and Jesper Hoeksema. 2013. Gaf: A grounded annotation framework for events. find
  15. Taher H. Haveliwala. 2002. Topic-sensitive pagerank. In WWW, pages 517–526. find
  16. Joseph Irwin, Mamoru Komachi, and Yuji Matsumoto. 2011. Narrative schema as world knowledge for coreference resolution. In Proceedings of the Fifteenth Conference on Computational Natural Language Learning: Shared Task, pages 86–92. Association for Computational Linguistics. find our card
  17. Heeyoung Lee, Angel Chang, Yves Peirsman, Nathanael Chambers, Mihai Surdeanu, and Dan Jurafsky. 2013. Deterministic coreference resolution based on entity-centric, precision-ranked rules. Computational Linguistics, (Just Accepted):1–54. find our card
  18. Mausam, Stephen Soderland, Oren Etzioni, Daniel Weld, Kobi Reiter, Michael Skinner, Marcus Sammer, and Jeff Bilmes. 2010. Panlingual lexical translation via probabilistic inference. Artificial Intelligence Journal (AIJ). find
  19. Mausam, Michael Schmitz, Robert Bart, Stephen Soderland, and Oren Etzioni. 2012. Open language learning for information extraction. In Proceedings of EMNLP. find
  20. K. Owczarzak and H.T. Dang. 2010. Overview of the tac 2010 summarization track. find
  21. S. Patwardhan and E. Riloff. 2009. A unified model of phrasal and sentential evidence for information extraction. In Proceedings of EMNLP 2009. find
  22. Rion Snow, Brendan O’Connor, Daniel Jurafsky, and Andrew Y Ng. 2008. Cheap and fast—but is it good?: evaluating non-expert annotations for natural language tasks. In Proceedings of the conference on empirical methods in natural language processing, pages 254– 263. Association for Computational Lingu… find
  23. A. Wang, C.D.V. Hoang, and M-Y. Kan. 2013. Perspectives on crowdsourcing annotations for Natural Language Processing. Language Resources and Evaluation, 47:9–31. find

↑ back to top

This page in Markdown → https://md.research.elamur.ai/free/en/papers/balasubramanian-2013-generating-coherent-event-schemas-scale/