Elamur Research⌕
☰
home / papers / 2006 / NPs for Events: Experiments in Coreference Annotation

NPs for Events: Experiments in Coreference Annotation

Summary

Research on resolving coreference between noun phrases and events is held back by a shortage of annotated data. The authors found that existing schemes (MUC-7, UCREL, TimeML, ACE) did not fully suit detailed event coreference marking. This pilot project, NP4E, therefore focused on designing annotation schemes and guidelines rather than on producing a large corpus. The team worked with English Reuters newswire on terrorism and security and annotated a sample of roughly 50,000 words in two phases:

  • NP phase: every NP was marked and linked into within-document coreference chains, using guidelines adapted from Mitkov et al. (2000).
  • Event phase: events were annotated with triggers and argument slots that reuse the NPs from the first phase, which reduces annotation effort.

The guidelines were revised several times after rounds of annotation and discussion. The PALinkA tool was substantially extended to support events. The result is a reusable methodology for later large-scale annotation, including cross-document work. The project also showed that even within-document coreference annotation is far from trivial.

Key points

  • The sample corpus (~50,000 words) was drawn from the Reuters corpus using the ht://dig search engine. It spans five topic clusters: a bombing in Bukavu, hostage-taking in Peru, hostage-taking in Tajikistan, a suicide bombing in Israel, and a hijacking involving China and Taiwan. The clusters were chosen with future cross-document coreference in mind.
  • NP annotation marks all NPs, even non-coreferential ones, because they serve as candidate event arguments. Annotators choose between a coref tag (certain links) and a ucoref tag (likely but uncertain links).
  • Event categories and arguments resemble those in ACE, which has 8 event types and 33 sub-types, but this project used fewer categories. It also treated modality and pronominal triggers differently. Demands, threats and promises became events in their own right.
  • PALinkA gained several features: multiple links between markables, plug-in support (e.g., a WordNet query plug-in) and a pluggable previewer suited to event structures.
  • All resources are available on the NP4E project page. The work was funded by the British Academy and the British Council (SYNCAR project).

Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.

Fields: Linguistics & languages

Topics: Coreference resolution

Cited by

15 papers of our library cite it (of the 140673 whose bibliographies we read); 5 of them have a page here, 10 not yet.

Cite

The paper itself, not this page. BibTeX from the library's record of the work.

@inproceedings{hasler2006nps,
  title = {{NPs for Events: Experiments in Coreference Annotation}},
  author = {Hasler, Laura and Orasan, Constantin and Naumann, Karin},
  booktitle = {Proceedings of the Fifth International Conference on Language Resources and Evaluation (LREC'06)},
  year = {2006},
  url = {https://aclanthology.org/L06-1325/}
}

References

19 references, 2 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

  1. ACE. http://www.itl.nist.gov/iaui/894.01/tests/ace/ ↗
  2. Bagga, A., Baldwin, B. (1999). Cross-document event coreference: annotations, experiments and observations. In Proceedings of the ACL’99 Workshop on Coreference and its Applications. pp. 1-8. find our card
  3. Botley, S. (1999). Corpora and discourse anaphora: using corpus evidence to test theoretical claims. PhD Thesis. University of Lancaster. find
  4. Bruneseaux, F., Romary, L. (1997). Codage des références et coréférences dans les dialogues hommemachine. In Proceedings of ACH-ALLC '97. pp. 15-17. find
  5. Davies, S., Poesio, M., Bruneseaux, F., Romary, L. (1998). Annotating coreference in dialogues: proposal for a scheme for MATE. First draft. Available at http://www.hcrc.ed.ac.uk/~poesio/MATE/anno\_manual .html de Rocha, M. (1997). Supporting anaphor resolution with a corpus-based probabilistic model… ↗
  6. Fellbaum, C. (1998). WordNet: An Electronic Lexical Database. Cambridge, MA: MIT Press. find
  7. Fligelstone, S. (1992). Developing a scheme for annotating text to show anaphoric relations. In G. Leitner (Ed.) New Directions in English Language Corpora: Methodology, Results, Software Developments. Berlin: Mouton de Gruyter, pp. 153-170. find
  8. Garside, R., Fligelstone, S., Botley, S. (1997) Discourse annotation: anaphoric relations in corpora. In R. Garside, G. Leech, A. McEnery (Eds.) Corpus Annotation: Linguistic Information from Computer Text Corpora. London: Longman, pp. 66-84. find
  9. Ge, N. (1998). Annotating the Penn Treebank with coreference information. Internal report, Department of Computer Science, Brown University. find
  10. Hirschman, L. (1997). MUC-7 coreference task definition. Version 3.0 find our card
  11. Mitkov, R., Evans, R., Orasan, C., Barbu, C., Jones, L., Sotirova, V. (2000). Coreference and anaphora: developing annotating tools, annotated resources and annotation strategies. In Proceedings of DAARC2000. pp. 49-58. find
  12. Orasan, C. (2003). PALinkA: A highly customisable tool for discourse annotation. In Proceedings of the 4 th SIGdial Workshop on Discourse and Dialogue, ACL’03. pp. 39-43. find
  13. Passonneau, R. J., Litman, D. L. (1997). Discourse segmentation by human and automated means. Computational Linguistics 23(1), pp. 103-139. find
  14. Poesio, M., Vieira, R. (1998). A corpus-based investigation of definite description use. Computational Linguistics 24(2), pp. 183-216. find
  15. Pustejovsky, J., Castaño, J., Ingria, R., Saurí, R., Gaizauskas, R., Setzer, A., Katz, G. (2003). TimeML: Robust Specification of Event and Temporal Expressions in Text. In Proceedings of IWCS-5, Fifth International Workshop on Computational Semantics. find
  16. Pustejovsky, J., Hanks, P., Sauri, R., See, A., Gaizuaskas, R., Setzer, A., Radev, D., Sundheim, B., Day, D., Ferro, L., Lazo, M. (2003). The TIMEBANK Corpus. In Proceedings of Corpus Linguistics 2003. pp. 647-656. find
  17. Rose, T.G., Stevenson, M., Whitehead, M. (2002). The Reuters Corpus Volume 1 - from Yesterday's News to Tomorrow's Language Resources. In Proceedings of LREC2002. pp. 827-833 find
  18. Setzer, A., Gaizauskas, R. (2000). Annotating Events and Temporal Information in Newswire Texts. In Proceedings of LREC2000. pp. 1287-1293. find
  19. Setzer, A., Gaizauskas, R. (2002). On the Importance of Annotating Event-Event Temporal Relations in Text. In Proceedings of the Workshop on Annotation Standards for Temporal Information in Natural Language, LREC2002. pp. 52-60. van Deemter, K., Kibble, R. (1999). What is coreference and what should… find

↑ back to top