Elamur Research⌕
☰
home / papers / 2011 / Narrative Schema as World Knowledge for Coreference Resolution

Narrative Schema as World Knowledge for Coreference Resolution

Summary

This CoNLL-2011 Shared Task entry tests whether script-like event knowledge can help coreference resolution on the OntoNotes corpus. The authors reimplemented the cluster-ranking model of Rahman and Ng (2009). It processes mentions in document order, ranks the partial clusters built so far against each new mention, and includes a null cluster so that discourse-new detection is learned jointly. The baseline relies on standard lexical, syntactic, distance, pronoun, and named-entity features. The authors then added features from narrative schema (Chambers and Jurafsky, 2009), which are sets of verbs whose argument slots tend to refer to the same participant. These features record whether two mentions, or a mention and an existing cluster, occupy the same schema role. On the development set, the schema features produced mixed results. In cross-validation on that set, however, MUC and B3 rose by about one point. Overall scores were low, partly because some baseline features were noisy. The work is an early demonstration that high-level event knowledge can be fed into an entity-based coreference model.

Key points

  • Setup: The system took part in both the closed and open tracks. It was implemented in Python with NLTK and used SVM-rank as the ranker.
  • Official overall scores: 31.88 in the closed track (baseline without schema features) and 35.84 in the open track (with schema features, but trained only on the WSJ portion).
  • Schema resource: The released schema database of Chambers and Jurafsky (2010), size-12 list. It feeds two feature types, SCHEMA_PAIR_MATCH and SCHEMA_CLUSTER_MATCH.
  • Effect of schema features:
    • On the development set, B3 F1 rose from 50.67 to 51.24, while MUC and CEAF dipped slightly.
    • In development-set cross-validation, both MUC and B3 improved by over 1 point, with p-values of 0.06 and below 0.01, respectively. CEAF gained about 0.5, which was not significant.
  • Feature ablation:
    • Removing HEAD_MATCH or DISTANCE caused large drops, about 20 MUC F1 points each.
    • Dropping the NE features improved results (MUC F1 +10.27). Dropping the pronoun or definiteness/demonstrative feature blocks also improved results, which suggests these features were noisy.

Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.

Fields: Linguistics & languages

Topics: Coreference resolution

Cited by

8 papers of our library cite it (of the 140673 whose bibliographies we read); 4 of them have a page here, 4 not yet.

Cite

The paper itself, not this page. BibTeX from the library's record of the work.

@inproceedings{irwin2011narrative,
  title = {{Narrative Schema as World Knowledge for Coreference Resolution}},
  author = {Irwin, Joseph and Komachi, Mamoru and Matsumoto, Yuji},
  booktitle = {Proceedings of the Fifteenth Conference on Computational Natural Language Learning: Shared Task},
  year = {2011},
  url = {https://aclanthology.org/W11-1913/}
}

References

15 references, 6 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

  1. Robert P. Abelson. 1981. Psychological status of the script concept. American Psychologist, 36(7):715– 729. find
  2. Nathanael Chambers and Dan Jurafsky. 2009. Unsupervised Learning of Narrative Schemas and their Participants. In Proceedings of the Joint Conference of the 47th Annual Meeting of the ACL and the 4th International Joint Conference on Natural Language Processing of the AFNLP, pages 602–610, Suntec, Si… find
  3. Nathanael Chambers and Dan Jurafsky. 2010. A database of narrative schemas. In Proceedings of the Seventh International Conference on Language Resources and Evaluation (LREC 2010), Malta. find
  4. Hal Daumé and Daniel Marcu. 2005. A large-scale exploration of effective global features for a joint entity detection and tracking model. In Proceedings of the Conference on Human Language Technology and Empirical Methods in Natural Language Processing - HLT ’05, pages 97–104, Morristown, NJ, USA. find
  5. Aria Haghighi and Dan Klein. 2010. Coreference resolution in a modular, entity-centered model. In Human Language Technologies: The 2010 Annual Conference of the North American Chapter of the Association for Computational Linguistics, pages 385–393. find our card
  6. Thorsten Joachims. 2006. Training linear SVMs in linear time. In Proceedings of the 12th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining KDD 06, pages 217–226. find
  7. Vincent Ng. 2010. Supervised noun phrase coreference research: the first fifteen years. In Proceedings of the 48th Annual Meeting of the Association for Computational Linguistics, pages 1396–1411. find our card
  8. Simone Paolo Ponzetto and Michael Strube. 2006a. Exploiting semantic role labeling, WordNet and Wikipedia for coreference resolution. In Proceedings of the main conference on Human Language Technology Conference of the North American Chapter of the Association of Computational Linguistics, pages 192… find our card
  9. Simone Paolo Ponzetto and Michael Strube. 2006b. Semantic role labeling for coreference resolution. In Proceedings of the Eleventh Conference of the European Chapter of the Association for Computational 6.74 1.47 2.00 6.77 5.47 4.08 10.27 9.66 7.06 find our card
  10. Linguistics - EACL ’06, pages 143–146, Morristown, NJ, USA. find
  11. Sameer Pradhan, Lance Ramshaw, Ralph Weischedel, Jessica MacBride, and Linnea Micciulla. 2007. Unrestricted Coreference: Identifying Entities and Events in OntoNotes. In International Conference on Semantic Computing (ICSC 2007), pages 446–453. find
  12. Sameer Pradhan, Lance Ramshaw, Mitchell Marcus, Martha Palmer, Ralph Weischedel, and Nianwen Xue. 2011. CoNLL-2011 Shared Task: Modeling Unrestricted Coreference in OntoNotes. In Proceedings of the Fifteenth Conference on Computational Natural Language Learning (CoNLL 2011), Portland, Oregon. find our card
  13. Altaf Rahman and Vincent Ng. 2009. Supervised Models for Coreference Resolution. In Proceedings of the 2009 Conference on Empirical Methods in Natural Language Processing, pages 968–977, Singapore. find our card
  14. Altaf Rahman and Vincent Ng. 2011. Narrowing the Modeling Gap: A Cluster-Ranking Approach to Coreference Resolution. Journal of Artificial Intelligence Research, 40:469–521. find
  15. Roger C. Schank and Robert P. Abelson. 1977. Scripts, plans, goals and understanding: An inquiry into human knowledge structures. Lawrence Erlbaum, Oxford, England. find

↑ back to top

This page in Markdown → https://md.research.elamur.ai/free/papers/irwin-2011-narrative-schema-as-world-knowledge/