Increasing Informativeness in Temporal Annotation

Summary

Under TimeML, linking every pair of events and times in a document is infeasible, so the guideline decides which TLINKs annotators produce. An error analysis of TimeBank TLINKs showed that the guideline leaves out useful link types and pushes annotators toward weak ones, mostly orderings of events against the Document Creation Time (DCT). The authors propose the narrative container (NC): a default, genre-dependent interval around publication time in which events with no explicit temporal anchor are assumed to occur. They also define Narrative Time (the currently active anchor, which shifts as new timexes appear) and Narrative Scope (the span from the earliest to the latest event mentioned). Three annotators re-annotated TimeBank, anchoring each event either to the closest time expression or to the NC with a reduced relation set. Agreement on anchors was fairly high. The new links placed events inside bounded intervals much more often than the original TimeBank links did. The work shows that guideline design, not only the specification, determines how informative a temporal corpus is.

Key points

The suggested default NC length depends on publication type: roughly 24 hours (one business day) for print newspapers, 2–10 hours for newswire, and a week or more for weekly or monthly outlets.
The re-annotation used only three relations (before, after, is included). Event-event links were excluded.
About 50% of events were linked to the NC with "is included." Under the original TimeBank annotation, such events would typically get only a vaguer before/overlap relation to the DCT.
Fleiss' kappa for temporal anchors averaged 0.74 (range −0.04 to 1), computed in R with the irr package. Negative values occur for articles where annotators linked everything to the NC, which is an artifact of low variability rather than real disagreement.
In the case study wsj_1042.tml (NC covering the day preceding the DCT up to and including the DCT itself), every event received at least one temporal constraint, and most received both a left and a right bound.

Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.

Fields: Linguistics & languages · Natural language processing

Cited by

43 papers of our library cite it (of the 140673 whose bibliographies we read); 4 of them have a page here, 39 not yet.

Cite

The paper itself, not this page. BibTeX from the library's record of the work.

@inproceedings{pustejovsky2011increasing,
  title = {{Increasing Informativeness in Temporal Annotation}},
  author = {Pustejovsky, James and Stubbs, Amber},
  booktitle = {Proceedings of the 5th Linguistic Annotation Workshop},
  year = {2011},
  url = {https://aclanthology.org/W11-0419/}
}

References

16 references, 2 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

James Allen. 1984. Towards a general theory of action and time. Arificial Intelligence, 23:123–154. find
Douglas Biber. 2009. Register, Genre, and Style. find
J. L. Fleiss. 1971. Measuring nominal scale agreement among many raters. Psychological Bulletin, 76(5):378–382. find
Matthias Gamer, Jim Lemon, and Ian Fellows Pus-pendra Singh ¡puspendra.pusp22@gmail.com¿, 2010. irr: Various Coefficients of Interrater Reliability and Agreement. R package version 0.83. find
Nancy Ide and Harry Bunt. 2010. Anatomy of annotation schemes: Mappings to graf. In In Proceedings 4th Linguistic Annotation Workshop (LAW IV). find
Nancy Ide. 2007. Annotation science: From theory to practice and use: Data structures for linguistics resources and applications. In In Proceedings of the Bi-enniel GLDV Conference. find
David Lee. 2001. Genres, registers, text types, domains, and styles: Clarifying the concepts and navigating a path through the bnc jungle. Language Learning & Technology, 5(3.3):37–72. find
Eleni Miltsakaki, Rashmi Prasad, Aravind Joshi, and Bonnie Webber. 2004. The penn discourse treebank. In In Proceedings of LREC 2004. find our card
Massimo Poesio. 2004. Discourse annotation and semantic annotation in the gnome corpus. In In Proceedings of the ACL Workshop on Discourse Annotation. find
Rashmi Prasad, Nikhil Dinesh, Alan Lee, Eleni Miltsakaki, Livio Robaldo, Aravind Joshi, and Bonnie Webber. 2008. The penn discourse treebank 2.0. In In Proceedings of the 6th International Conference on Language Resources and Evaluation (LREC 2008). find our card
James Pustejovsky, Patrick Hanks, Roser Saurı̀, Andrew See, Robert Gaizauskas, Andrea Setzer, Dragomir Radev, Beth Sundheim, David Day, Lisa Ferro, and Marcia Lazo. 2003. The timebank corpus. In Dawn Archer, Paul Rayson, Andrew Wilson, and Tony McEnery, editors, Proceedings of the Corpus Linguistics… find
James Pustejovsky, Robert Knippen, Jessica Littman, and Roser Saurı́. 2005. Temporal and event information in natural language text. Language Resources and Evaluation, 39:123–164, May. find
R Team, 2009. R: A Language and Environment for Statistical Computing. R Foundation for Statistical Computing, Vienna, Austria. ISBN 3-900051-07-0. find
Roser Saurı́, Jessica Littman, Bob Knippen, Robert Gaizauskas, Andrea Setzer, and James Pustejovsky, 2006. TimeML Annotation Guidelines, version 1.2.1 edition, January. find
Carlota Smith. 2003. Modes of Discourse. Cambridge University Press, Cambridge, UK. find
Marc Vilain, Henry Kautz, and Peter Beek. 1986. Constraint propagation algorithms for temporal reasoning. In Readings in Qualitative Reasoning about Physical Systems, pages 377–382. Morgan Kaufmann. find

↑ back to top

This page in Markdown → https://md.research.elamur.ai/free/papers/pustejovsky-2011-increasing-informativeness-temporal-annotation/

📄 Increasing Informativeness in Temporal Annotation