Elamur Research⌕
☰
home / papers / 2025 / Visual Representations of Temporal Relations between Events and Time Expressions in News Stories

Visual Representations of Temporal Relations between Events and Time Expressions in News Stories

Summary

Dense temporal relation annotations are hard to read in common tools: Brat displays the entire text together with all entities and links, so annotator errors are easy to miss. The authors propose Vitra (Visualizer of temporal relation annotations), an open-source tool for experts. It shows temporal expressions, the events linked to them and the relations between them separately, and places them on a timeline. The tool was tested on a subset of the Text2Story Lusa corpus, which consists of Portuguese news articles. Annotations made in Brat were converted to JSON using the text2story package. On this data, the authors showed that atypical patterns in a multilayer scheme become more noticeable when the visualization focuses on a single layer. The resulting observations have already led to changes in the annotation guidelines. A small survey of linguist annotators showed that Vitra is on par with Brat and was even rated slightly higher for detecting errors and temporal patterns. The work matters because it supports iterative refinement of annotation schemes and the quality of corpora for model training.

Key points

  • Data: 67 news texts in European Portuguese, mostly from October–December 2020, from the Text2Story Lusa corpus. The annotation scheme is based on ISO-TimeML and taken from Silvano et al. (2023a).
  • Vitra's code is open source and available on GitHub (evelinamorim/sentencevisual); there is also an online demo. Conversion from the Brat standoff format to JSON is done by the text2story package.
  • The questionnaire contained six statements. Vitra was rated by 3 experts, Brat by 4. For the ability to detect errors, Vitra received scores of 4, 4 and 5, and Brat from 2 to 5. For detecting temporal patterns, Vitra received 5, 5 and 4, and Brat 4, 4, 4 and 2.
  • The authors themselves acknowledge limitations: the respondent sample is too small for statistical conclusions (they recommend at least 15), the tool does not allow correcting annotations, and it is tied to a single scheme.
  • In their future plans, the authors mention support for other formats and schemes, as well as integrating Vitra into the INCEpTION annotation tool.

Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.

Fields: Linguistics & languages

Cite

The paper itself, not this page. BibTeX from the library's record of the work. DOI 10.18653/v1/2025.law-1.20

@inproceedings{amorim2025visual,
  title = {{Visual Representations of Temporal Relations between Events and Time Expressions in News Stories}},
  author = {Amorim, Evelin and Leal, António and Yu, Nana and Silvano, Purificação Moura and Jorge, Alipio Mario},
  booktitle = {Proceedings of the 19th Linguistic Annotation Workshop (LAW-XIX-2025)},
  year = {2025},
  doi = {10.18653/v1/2025.law-1.20},
  url = {https://doi.org/10.18653/v1/2025.law-1.20}
}

References

51 references, 2 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

  1. Evelin Amorim, Ricardo Campos, Alípio Jorge, Pedro Mota, and Rúben Almeida. 2024. text2story: A python toolkit to extract and visualize story components of narrative text. In Proceedings of the 2024 Joint International Conference on Computational Linguistics, Language Resources and Evaluation (LREC-… find
  2. Evelin Amorim, Alexandre Ribeiro, Inês Cantante, Alípio Jorge, Brenda Santana, Sérgio Nunes, Purificação Silvano, António Leal, and Ricardo Campos. 2021. Brat2viz: a tool and pipeline for visualizing narratives from annotated texts. In Proceedings of Text2Story-Fourth Workshop on Narrative Extractio… find
  3. Allan Bell. 1997. The Language of News Media. Blackwell. find
  4. Steven Bethard, Guergana Savova, Wei-Te Chen, Leon Derczynski, James Pustejovsky, and Marc Verhagen. 2016. SemEval-2016 task 12: Clinical TempEval. In Proceedings of the 10th International Workshop on Semantic Evaluation (SemEval-2016), pages 1052– 1062, San Diego, California. Association for Comput… find
  5. André Bittar. 2009. Annotation of events and temporal expressions in French texts. In Proceedings of the Third Linguistic Annotation Workshop (LAW III), pages 48–51, Suntec, Singapore. Association for Computational Linguistics. find
  6. Tommaso Caselli, Valentina Bartalesi Lenzi, Rachele Sprugnoli, Emanuele Pianta, and Irina Prodanof. 2011. Annotating events, temporal expressions and relations in Italian: the it-timeml experience for the ita-TimeBank. In Proceedings of the 5th Linguistic Annotation Workshop, pages 143–151, Portland… find
  7. Tommaso Caselli and Piek Vossen. 2017. The event storyline corpus: A new benchmark for causal and temporal relation extraction. In Proceedings of the Events and Stories in the News Workshop, pages 77– 86. find
  8. Taylor Cassidy, Bill McDowell, Nathanael Chambers, and Steven Bethard. 2014. An annotation framework for dense event ordering. In Proceedings of the 52nd Annual Meeting of the Association for Computational Linguistics (Volume 2: Short Papers), pages 501– 506. find
  9. Qing Chen, Shixiong Cao, Jiazhe Wang, and Nan Cao. 2023. How does automation shape the process of narrative visualization: A survey of tools. IEEE Transactions on Visualization and Computer Graphics. find
  10. Zhutian Chen, Yun Wang, Qianwen Wang, Yong Wang, and Huamin Qu. 2019. Towards automated info-graphic design: Deep learning-based auto-extraction of extensible timeline. IEEE transactions on visualization and computer graphics, 26(1):917–926. find
  11. Francisco Costa and António Branco. 2012. Time-BankPT: A TimeML annotated corpus of Portuguese. In Proceedings of the Eighth International Conference on Language Resources and Evaluation (LREC’12), pages 3727–3734, Istanbul, Turkey. European Language Resources Association (ELRA). find
  12. George R Doddington, Alexis Mitchell, Mark A Przybocki, Lance A Ramshaw, Stephanie M Strassel, and Ralph M Weischedel. 2004. The automatic content extraction (ace) program-tasks, data, and evaluation. In Lrec, volume 2, pages 837–840. Lisbon. find our card
  13. Catarina Justo dos Santos Fernandes. 2023. Visualizing news stories from annotated text. Master’s thesis, Faculdade de Enegenharia da Universidade do Porto. find
  14. Extracting message sequence charts from hindi narrative text. In Proceedings of the First Joint Workshop on Narrative Understanding, Storylines, and Events, pages 87–96. find
  15. Seohyun Im, Hyunjo You, Hayun Jang, Seungho Nam, and Hyopil Shin. 2009. KTimeML: Specification of temporal and event expressions in Korean text. In Proceedings of the 7th Workshop on Asian Language Resources (ALR7), pages 115–122, Suntec, Singapore. Association for Computational Linguistics. find
  16. ISO-24617-1. 2012. Language resource management - semantic annotation framework (semaf) - part 1: Time and events (semaf-time, iso-timeml). Standard, Geneva, CH. find
  17. ISO-24617-4. 2014. Language resource management-semantic annotation framework (semaf) - part 4: Semantic roles (semaf-sr). Standard, Geneva, CH. find
  18. ISO-24617-7. 2020. Language resource management-semantic annotation framework (semaf) - part 7: Spatial information. Standard, Geneva, CH. find
  19. ISO-24617-9. 2019. Language resource management-semantic annotation framework (semaf) - - part 9: Reference annotation framework (raf). Standard, Geneva, CH. find
  20. Adam Jatowt, Antoine Doucet, and Ricardo Campos. 2022. Diachronic analysis of time references in news articles. pages 918–923. find
  21. Jan-Christoph Klie, Michael Bugert, Beto Boullosa, Richard Eckart de Castilho, and Iryna Gurevych. 2018. The INCEpTION platform: Machine-assisted and knowledge-oriented interactive annotation. In Proceedings of the 27th International Conference on Computational Linguistics: System Demonstrations, pa… find
  22. Ryan Ka Yau Lai. 2023. From annotation to analysis: Exploring conversational dynamics with rezonater. In Proceedings of the 37th Pacific Asia Conference on Language, Information and Computation, pages 303–313. find
  23. Elena Filatova and Eduard Hovy. 2001. Assigning timestamps to event-clauses. In Proceedings of the Workshop on Temporal and Spatial Information Processing - Volume 13, TASIP ’01, USA. Association for Computational Linguistics. find
  24. L. Lange, A. Iurshina, H. Adel, and J. Strötgen. 2020. Adversarial alignment of multilingual models for extracting temporal expressions from text. In Proceedings of the 5th Workshop on Representation Learning for NLP, pages 103–109. find
  25. Francisco Gonçalves, Ricardo Campos, and Alípio Jorge. 2023. Text2storyline: generating enriched storylines from text. In European Conference on Information Retrieval, pages 248–254. Springer. find
  26. António Leal, Purificação Silvano, Evelin Amorim, Inês Cantante, Fátima Silva, Alípio Mario Jorge, and Ricardo Campos. 2022. The place of ISO-space in Text2Story multilayer annotation scheme. In Proceedings of the 18th Joint ACL - ISO Workshop on Interoperable Semantic Annotation within LREC2022, pa… find
  27. Swapnil Hingmire, Nitin Ramrakhiyani, Avinash Kumar Singh, Sangameshwar Patil, Girish Palshikar, Pushpak Bhattacharyya, and Vasudeva Varma. 2020. find
  28. Anne-Lyse Minard, Manuela Speranza, Ruben Urizar, Begoña Altuna, Marieke van Erp, Anneleen Schoen, and Chantal van Son. 2016. MEANTIME, the NewsReader multilingual event and time corpus. In Proceedings of the Tenth International Conference on Language Resources and Evaluation (LREC’16), pages 4417–4… find
  29. Phong H Nguyen, Kai Xu, Rick Walker, and BL William Wong. 2016. Timesets: Timeline visualization with set relations. Information Visualization, 15(3):253– 269. find
  30. Sérgio Nunes, Alípio Mario Jorge, Evelin Amorim, Hugo Sousa, António Leal, Purificação Silvano, Inês Cantante, and Ricardo Campos. 2024. Text2Story lusa: A dataset for narrative analysis in European Portuguese news articles. In Proceedings of the 2024 Joint International Conference on Computational… find
  31. Helene Olsen, Étienne Simon, Erik Velldal, and Lilja Øvrelid. 2024. Socio-political events of conflict and unrest: A survey of available datasets. In Proceedings of the 7th Workshop on Challenges and Applications of Automated Extraction of Socio-political Events from Text (CASE 2024), pages 40–53. find
  32. James Pustejovsky, José Castaño, Robert Ingria, Roser Saurí, Rob Gaizauskas, Andrea Setzer, Graham Katz, and Dragomir Radev. 2003. Timeml: Robust specification of event and temporal expressions in text. pages 28–34. find
  33. James Pustejovsky and Marc Verhagen. 2009. SemEval-2010 task 13: Evaluating events, time expressions, and temporal relations (TempEval-2). In Proceedings of the Workshop on Semantic Evaluations: Recent Achievements and Future Directions (SEW-2009), pages 112–116, Boulder, Colorado. Association for C… find
  34. Jeff Sauro and James R Lewis. 2016. Quantifying the user experience: Practical statistics for user research. Morgan Kaufmann. find
  35. Christina Schokkenbroek. 1999. News stories: Structure, time and evaluation. Time & Society, 8(1):59– 98. find
  36. Purificação Silvano, Evelin Amorim, António Leal, Inês Cantante, Maria de Fátima Henriques da Silva, Alípio Jorge, Ricardo Campos, and Sérgio Sobral Nunes. 2023a. Annotation and visualisation of reporting events in textual narratives. In Proceedings of Text2Story 2023: Sixth Workshop on Narrative Ex… find
  37. Purificação Silvano, António Leal, Fátima Silva, Inês Cantante, Fatima Oliveira, and Alípio Mario Jorge. 2021. Developing a multilayer semantic annotation scheme based on ISO standards for the visualization of a newswire corpus. In Proceedings of the 17th Joint ACL - ISO Workshop on Interoperable Se… find
  38. Purificação Silvano, Evelin Amorim, António Leal, Inês Cantante, Alípio Jorge, Ricardo Campos, and Nana Yu. 2024. Untangling a web of temporal relations in news articles. In Proceedings of Text2Story 2024 - Seventh Workshop on Narrative Extraction From Texts, volume 3671 of CEUR Workshop Proceedings… find
  39. Purificação Silvano, Alípio Jorge, António Leal, Evelin Amorim, Hugo Sousa, Inês Cantante, Ricardo Campos, and Sérgio Nunes. 2023b. Text2story lusa annotated corpus. Data set. find
  40. Hugo Sousa, Ricardo Campos, and Alípio Jorge. 2023. Tei2go: A multilingual approach for fast temporal expression identification. In Proceedings of the 32nd ACM International Conference on Information and Knowledge Management, CIKM ’23, page 5401–5406, New York, NY, USA. Association for Computing Mac… find
  41. Laura South, David Saffo, Olga Vitek, Cody Dunne, and Michelle A Borkin. 2022. Effective use of likert scales in visualization evaluations: A systematic review. In Computer Graphics Forum, volume 41, pages 43–55. Wiley Online Library. find
  42. Pontus Stenetorp, Sampo Pyysalo, Goran Topić, Tomoko Ohta, Sophia Ananiadou, and Jun’ichi Tsujii. 2012. Brat: a web-based tool for nlp-assisted text annotation. In Proceedings of the Demonstrations at the 13th Conference of the European Chapter of the Association for Computational Linguistics, page… find
  43. Weiyi Sun, Anna Rumshisky, and Ozlem Uzuner. 2013. Evaluating temporal relations in clinical text: 2012 i2b2 challenge. Journal of the American Medical Informatics Association : JAMIA, 20. find
  44. Tan Tang, Renzhong Li, Xinke Wu, Shuhan Liu, Johannes Knittel, Steffen Koch, Thomas Ertl, Lingyun Yu, Peiran Ren, and Yingcai Wu. 2020. Plotthread: Creating expressive storyline visualizations using reinforcement learning. IEEE Transactions on Visualization and Computer Graphics, 27(2):294–303. find
  45. Tan Tang, Sadia Rubab, Jiewen Lai, Weiwei Cui, Lingyun Yu, and Yingcai Wu. 2018. istoryline: Effective convergence to hand-drawn storylines. IEEE transactions on visualization and computer graphics, 25(1):769–778. find
  46. Xiaozhi Wang, Yulin Chen, Ning Ding, Hao Peng, Zimu Wang, Yankai Lin, Xu Han, Lei Hou, Juanzi Li, Zhiyuan Liu, et al. 2022. Maven-ere: A unified large-scale dataset for event coreference, temporal, causal, and subevent relation extraction. arXiv preprint arXiv:2211.07342. find our card
  47. Yunchao Wang, Guodao Sun, Zihao Zhu, Tong Li, Ling Chen, and Ronghua Liang. 2024. E2 storyline: visualizing the relationship with triplet entities and event discovery. ACM Transactions on Intelligent Systems and Technology, 15(1):1–26. find
  48. Attribute Name Class Li Ye, Lei Wang, Shaolun Ruan, Yuwei Meng, Yi-gang Wang, Wei Chen, and Zhiguang Zhou. 2024. Storyexplorer: A visualization framework for storyline generation of textual narratives. arXiv preprint arXiv:2411.05435. — [arXiv:2411.05435](https://arxiv.org/abs/2411.05435) find
  49. Jian Zhao, Shenyu Xu, Senthil Chandrasegaran, Chris Bryan, Fan Du, Aditi Mishra, Xin Qian, Yiran Li, and Kwan-Liu Ma. 2021. Chartstory: Automated partitioning, layout, and captioning of charts into comic-style narratives. IEEE Transactions on Visualization and Computer Graphics, 29(2):1384–1399. find
  50. Type Pos Xiaoshi Zhong and Erik Cambria. 2023. Time expression recognition and normalization: a survey. Artificial Intelligence Review, 56(9):9115–9140. find
  51. Tense Xiaoshi Zhong, Chenyu Jin, Mengyu An, and Erik Cambria. 2024. Xtime: A general rule-based method for time expression recognition and normalization. Knowledge-Based Systems, 297:111921. find

↑ back to top

This page in Markdown → https://md.research.elamur.ai/free/en/papers/amorim-2025-visual-representations-temporal-relations-between/