Elamur Research⌕
☰
home / papers / 2011 / Annotating and Learning Event Durations in Text

Annotating and Learning Event Durations in Text

Paper · 2011 · Feng Pan, Rutu Mulkar-Mehta and Jerry R. Hobbs

Summary

The paper tackles a largely ignored aspect of temporal understanding: estimating how long events last when a text gives no explicit duration. The authors annotated events in a subset of the TimeBank corpus (TimeML) with lower and upper duration bounds intended to cover roughly 80% of plausible cases, plus a confidence score. They found that most severe disagreements between annotators came from a small set of recurring event types, including actions versus states, aspectual events and reporting events, and they wrote guidelines to address them. Each annotated interval is modeled as a normal distribution, so agreement between annotators can be measured as the overlap of their distributions. Supervised classifiers trained on this data predict coarse durations, either shorter or longer than a day or the most likely temporal unit, clearly beating baselines and coming close to human agreement, with SVM performing best. The work matters because it supplies commonsense temporal knowledge useful for event ordering, coreference, question answering and temporal reasoning. Its interval-agreement method could also apply to other vague scalar information in text.

Key points

  • Corpus: all 48 non-WSJ TimeBank articles (2,132 events) and 10 WSJ articles (156 events) were annotated by three annotators. The annotations are merged into TimeBank and released by ISI.
  • Guidelines: annotators read the full article first and judge as contemporary readers would, without using hindsight. Coreferent events receive identical ranges. With the guidelines, agreement improved on a test set.
  • Agreement method: intervals are represented on a logarithmic scale as normal distributions, and agreement is computed from distribution overlap.
  • Generalization: the coarse-grained model reached 75.0% precision on unseen WSJ data, close to its in-domain result.
  • Temporal-unit task: this task uses seven classes, from second to year. Exact human agreement was only 44.4%, but counting adjacent units as matches raised it to 79.8% (kappa ≈ 0.76). On this approximate measure, SVM reached 67.9%, compared with a 51.5% baseline.

Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.

 Institutions

🏢MicrosoftRedmond · United States325 works · 142,931 citations🎓University of Southern CaliforniaLos Angeles · United States162 works · 35,836 citations

 Fields, topics and concepts

📚 Shelves 4The field of science this work belongs to (OpenAlex fields).Computer Science · 16,667 worksArtificial Intelligence · 16,194 worksArts and Humanities · 16,039 worksLanguage and Linguistics · 15,891 works

Cite

Cite as Feng Pan, Rutu Mulkar-Mehta and Jerry R. Hobbs. 2011. Annotating and Learning Event Durations in Text. Computational Linguistics.

The paper itself, not this page. BibTeX from the library's record of the work. DOI 10.1162/coli_a_00075

@article{pan2011annotating,
  title = {{Annotating and Learning Event Durations in Text}},
  author = {Feng Pan, Rutu Mulkar-Mehta and Jerry R. Hobbs},
  journal = {Computational Linguistics},
  year = {2011},
  doi = {10.1162/coli\_a\_00075},
  url = {https://doi.org/10.1162/coli_a_00075}
}

References

43 references, 5 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

  1. Allen, James F. 1984. Towards a general theory of action and time. Artificial Intelligence, 23(2):123–154. find
  2. Allen, James F. and George Ferguson. 1994. Actions and events in interval temporal logic. Journal of Logic and Computation, 4(5):531–579. find
  3. Bejan, Cosmin Adrian and Sanda Harabagiu. 2010. Unsupervised event coreference resolution with rich linguistic features. In Proceedings of the 48th Annual Meeting of the Association for Computational Linguistics, pages 1412–1422, Uppsala, Sweden.
  4. Boguraev, Branimir and Rie Kubota Ando. 2005. Timeml-compliant text analysis for temporal reasoning. In Proceedings of the International Joint Conference on Artificial Intelligence, pages 997–1003, Edinburgh, Scotland, UK. find
  5. Brill, Eric. 1992. A simple rule-based part of speech tagger. In Applied Natural Language Processing, pages 152–155, Trento, Italy.
  6. Carletta, Jean. 1996. Assessing agreement on classification tasks: The kappa statistic. Computational Linguistics, 22:249–254.
  7. Chittaro, Luca and Angelo Montanari. 2000. Temporal representation and reasoning in artificial intelligence: Issues and approaches. Annals of Mathematics and Artificial Intelligence, 28:47–106. find
  8. Dowty, David. 1979. Word Meaning and Montague Grammar. Reidel, Dordrecht. find
  9. Duda, R. O. and P. E. Hart. 1973. Pattern Classification and Scene Analysis. John Wiley & Sons, New York. find
  10. Di Eugenio, Barbara and Michael Glass. 2004. The kappa statistic: A second look. Computational Linguistics, 30(1):95–101. find
  11. Ferro, Lisa. 2001. Instruction manual for the annotation of temporal expressions. MITRE Technical Report MTR 01W0000046, June 2001. The MITRE Corporation, Mc Lean, VA. find
  12. Filatova, Elena and Eduard Hovy. 2001. Assigning time-stamps to event-clauses. In Proceedings of the Workshop on Temporal and Spatial Information Processing - Volume 13, pages 13:1–13:8, Toulouse, France. find
  13. Fortemps, Philippe. 1997. Jobshop scheduling with imprecise durations: A fuzzy approach. IEEE Transactions on Fuzzy Systems, 5(1997), pages 557–569. find
  14. Gildea, Daniel and Daniel Jurafsky. 2002. Automatic labeling of semantic roles. Computational Linguistics, 28(3):245–288.
  15. Giorgi, Alessandra and Fabio Pianesi. 1997. Tense and Aspect: From Semantics to Morphosyntax, Oxford University Press, Oxford Studies in Comparative Syntax, Oxford. find
  16. Godo, Lluis and Lluı́s Vila. 1995. Possibilistic temporal reasoning based on fuzzy temporal constraints. In Proceedings of the International Joint Conference on Artificial Intelligence, pages 1916–1923, Montreal, Quebec, Canada. find
  17. Han, Benjamin and Alon Lavie. 2004. A framework for resolution of time in natural language. Transactions on Asian Language Information Processing Special Issue on Spatial and Temporal Information Processing, 3(1):11–32, March. find
  18. Harabagiu, Sanda and Cosmin Adrian Bejan. 2005. Question answering based on temporal inference. In Proceedings of the AAAI-2005 Workshop on Inference for Textual Question Answering, pages 27–34, Pittsburg, PA. find
  19. Hermjakob, Ulf and Raymond J. Mooney. 1997. Learning parse and translation decisions from examples with rich context. In The 35th Annual Meeting of the Association for Computational Linguistics, pages 482–489, Madrid, Spain. find
  20. Hitzeman, Janet, Marc Moens, and Claire Grover. 1995. Algorithms for analysing the temporal structure of discourse. In Proceedings of the 7th Conference of the European Chapter of the Association for Computational Linguistics, pages 253–260, Dublin, Ireland. find
  21. Hobbs, Jerry R. 2000. Half orders of magnitude. In Proceedings of KR-2000 Workshop on Semantic Approximation, Granularity, and Vagueness, pages 28–38, Breckenridge, CO. find
  22. Hobbs, Jerry R. and Vladik Kreinovich. 2001. Optimal choice of granularity in commonsense estimation: Why half orders of magnitude? In Proceedings of Joint 9th Information Fuzzy Systems Association World Congress and 20th North American Fuzzy Information Processing Society International Conference,… find
  23. Hobbs, Jerry R. and Feng Pan. 2004. An ontology of time for the semantic web. ACM Transactions on Asian Language Information Processing, 3:66–85. find
  24. Krippendorff, Klaus. 1980. Content Analysis: An Introduction to Its Methodology. Sage Publications, Beverly Hills, CA. find
  25. Lapata, Mirella and Alex Lascarides. 2006. Learning sentence-internal temporal relations. Journal of AI Research, 27(1), pages 85–117. find
  26. Madden, Carol J. and Rolf A. Zwaan. 2003. How does verb aspect constrain event representations? Memory & Cognition, 31:663–672. find
  27. Mani, Inderjeet and Barry Schiffman. 2007. Temporally anchoring and ordering events in news. Event Recognition in Natural Language, John Benjamins. find
  28. Mani, Inderjeet, Marc Verhagen, Ben Wellner, Chong Min Lee, and James Pustejovsky. 2006. Machine learning of temporal relations. In Proceedings of the Joint Conference of the International Committee on Computational Linguistics and the Association for Computational Liguistics, pages 17–18, Sydney, A… find
  29. Mani, Inderjeet and George Wilson. 2000. Robust temporal processing of news. In Proceedings of the 38th Annual Meeting on Association for Computational Linguistics, pages 69–76, Stroudsburg, PA. find
  30. Miller, George A., Richard Beckwith, Christiane Fellbaum, Derek Gross, and Katherine Miller. 1990. Wordnet: An on-line lexical database. International Journal of Lexicography, 3:235–244. find
  31. Moens, Marc and Mark Steedman. 1988. Temporal ontology and temporal reference. Computational Linguistics, 14:15–28. find
  32. Passonneau, Rebecca J. 1988. A computational model of the semantics of tense and aspect. Computational Linguistics, 14:44–60. find
  33. Pustejovsky, James, Patrick Hanks, Roser Sauri, Andrew See, Robert Gaizauskas, Andrea Setzer, Dragomir Radev, Beth Sundheim, David Day, Lisa Ferro, and Marcia Lazo. 2003. The timebank corpus. In Proceedings of Corpus Linguistics, pages 647–656, Lancaster, UK. find
  34. Quinlan, J. Ross. 1993. C4.5: Programs for Machine Learning. Morgan Kaufmann Publishers Inc., San Francisco, CA.
  35. Rieger, Charles J. 1974. Conceptual memory: A theory and computer program for processing and meaning content of natural language utterances. Stanford Artificial Intelligence Laboratory Memo, Computer Science Department, Stanford University. find
  36. Siegel, S. and N. J. Castellan. 1988. Nonparametric statistics for the behavioral sciences. McGraw–Hill, Inc., 2nd edition, London, UK. find
  37. Smith, Carlota. 2005. Aspectual entities and tense in discourse. Aspectual Inquiries, pages 223–238, Kluwer, Dordrecht. find
  38. Tao, Cui, Harold R. Solbrig, Deepak K. Sharma, Wei-Qi Wei, Guergana K. Savova, and Christopher G. Chute. 2010. Time-oriented question answering from clinical narratives sing semantic-web techniques. In Proceedings of the 9th International Semantic Web Conference on the Semantic Web - Volume Part II,… find
  39. Vapnik, Vladimir N. 1995. The Nature of Statistical Learning Theory. Springer-Verlag, New York, NY. find
  40. Vendler, Zeno. 1967. Linguistics in philosophy. Aspectual Inquiries. Cornell University Press, Ithaca, NY. find
  41. Witten, I. H. and E. Frank. 2005. Data Mining: Practical Machine Learning Tools and Techniques. Morgan Kaufmann, 2nd edition, San Francisco, CA. find
  42. Zhou, Li and George Hripcsak. 2007. Temporal reasoning with medical data—a review with emphasis on medical natural language processing. Journal of Biomedical Informatics, 40(2):183–202. find
  43. Zhou, Qing and Qing Zhou Richard Fikes. 2002. A reusable time ontology. In Proceedings of the AAAI Workshop on Ontologies for the Semantic Web, the Eighteenth National Conference on Artificial Intelligence WS-02-11, Edmonton, Alberta, Canada. find

Updated

↑ back to top

This page in Markdown → https://md.research.elamur.ai/free/en/papers/pan-2011-annotating-learning-event-durations-text/