Elamur Research⌕
☰
home / papers / 2026 / HistoryBankQA: Multilingual Temporal Question Answering on Historical Events

HistoryBankQA: Multilingual Temporal Question Answering on Historical Events

Summary

Large language models are increasingly expected to reason about time, yet existing benchmarks for this skill tend to be small, English-only, synthetic, or focused on recent happenings. This paper addresses that gap for historical events. The authors automatically build HistoryBank, a database of more than 10 million dated events. These events are drawn from Wikipedia's "On This Day" date pages and from article infoboxes across ten language editions: six European ones (Spanish, Russian, Portuguese, Italian, German and French), two South Asian ones (Hindi and Bengali), plus Indonesian and English. From this database they generate HistoryBankQA, a benchmark of six question families. These cover date recall, chronological sorting, interval arithmetic, relations between event spans, counting events within a period, and finding the previous or next edition of a recurring event. The authors test LLaMA-3-8B, Mistral-7B, Gemma-2-9B, Qwen3-8B and GPT-4o, mainly zero-shot. GPT-4o comes out strongest, Gemma-2 leads the smaller models, and scores vary considerably by task and language. The resource helps researchers judge whether models truly reason about time or rely on recalled facts and English-specific cues. Code and data are publicly released.

Key points

  • Scale of HistoryBank: 10M+ events in 10 languages. English dominates with about 8.2M events from roughly 4.26M articles, and German is next with about 1.52M events.
  • Six HistoryBankQA tasks: FactQA, SequenceQA, DurationQA, RelationQA (with subtypes such as Overlap, Inclusion, Gap Between, Order Span Start/End), CountQA and RecurrenceQA.
  • Event quality: In a quality check, 84% of extracted events were rated interesting. The rest were mostly low-salience infobox entries such as census or administrative records, not generation errors.
  • GPT-4o error analysis (English test set): Three recurring failure modes appear:
    • missing or wrong temporal knowledge;
    • correct knowledge but faulty reasoning, such as mishandled century boundaries or inclusive/exclusive ranges;
    • final answers that contradict otherwise correct reasoning.
  • Acknowledged limitations:
    • Western-centric Wikipedia coverage;
    • LLM-generated questions;
    • a narrow evaluation setup;
    • no human evaluation of model answers.

Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.

Fields: Linguistics & languages

Cite

The paper itself, not this page. BibTeX from the library's record of the work. DOI 10.18653/v1/2026.starsem-conference.33

@inproceedings{mandal2026historybankqa,
  title = {{HistoryBankQA: Multilingual Temporal Question Answering on Historical Events}},
  author = {Mandal, Biswadip and Khandelwal, Anant and Gupta, Manish},
  booktitle = {Proceedings of the 15th Joint Conference on Lexical and Computational Semantics (*SEM 2026)},
  year = {2026},
  doi = {10.18653/v1/2026.starsem-conference.33},
  url = {https://doi.org/10.18653/v1/2026.starsem-conference.33}
}

References

29 references, 2 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

  1. Abdelrahman Abdallah, Mohammed Ali, Muhammad Abdul-Mageed, and Adam Jatowt. 2026. Tempo: A realistic multi-domain benchmark for temporal reasoning-intensive retrieval. arXiv preprint arXiv:2601.09523. — [arXiv:2601.09523](https://arxiv.org/abs/2601.09523) find
  2. James F Allen. 1983. Maintaining knowledge about temporal intervals. Communications of the ACM, 26(11):832–843. find
  3. Sören Auer, Christian Bizer, Georgi Kobilarov, Jens Lehmann, Richard Cyganiak, and Zachary Ives. 2007. Dbpedia: A nucleus for a web of open data. In international semantic web conference, pages 722– 735. Springer. find
  4. Claire Barale, Leslie Barrett, Vikram Sunil Bajaj, and Michael Rovatsos. 2025. Lextime: A benchmark for temporal ordering of legal events. arXiv preprint arXiv:2506.04041. find
  5. Nadav Borenstein, Natalia da Silva Perez, and Isabelle Augenstein. 2023. Multilingual event extraction from historical newspaper adverts. arXiv preprint arXiv:2305.10928. find
  6. Aleksandar Botev, Soham De, Samuel L Smith, Anushan Fernando, George-Cristian Muraru, Ruba Haroun, Leonard Berrada, Razvan Pascanu, Pier Giuseppe Sessa, Robert Dadashi, and 1 others. 2024. Recurrentgemma: Moving past transformers for efficient open language models. arXiv preprint arXiv:2404.07839. — [arXiv:2404.07839](https://arxiv.org/abs/2404.07839) find
  7. Chia-Hui Chang, Yuan-Hao Lin, and Hsiu-Min Chuang. 2020. Eventgo! exploring event dynamics from social-media posts. In 2020 International Computer Symposium (ICS), pages 548–552. IEEE. find
  8. Hyung Won Chung, Le Hou, Shayne Longpre, Barret Zoph, Yi Tay, William Fedus, Yunxuan Li, Xuezhi Wang, Mostafa Dehghani, Siddhartha Brahma, and 1 others. 2024. Scaling instruction-finetuned language models. Journal of Machine Learning Research, 25(70):1–53. find
  9. Simon Gottschalk and Elena Demidova. 2018. Eventkg: A multilingual event-centric temporal knowledge graph. In European semantic web conference, pages 272–287. Springer. find
  10. Aaron Grattafiori, Abhimanyu Dubey, Abhinav Jauhri, and et al. 2024. The llama 3 herd of models. arXiv preprint arXiv:2407.21783. find
  11. Raphael Gruber, Abdelrahman Abdallah, Michael Färber, and Adam Jatowt. 2025. Complextempqa: A 100m dataset for complex temporal question answering. In Proceedings of the 2025 Conference on Empirical Methods in Natural Language Processing, pages 9111–9123. find
  12. Johannes Hoffart, Fabian M Suchanek, Klaus Berberich, and Gerhard Weikum. 2013. Yago2: A spatially and temporally enhanced knowledge base from wikipedia. Artificial intelligence, 194:28–61. find
  13. Duygu Sezen Islakoglu and Jan-Christoph Kalo. 2025. Chronosense: Exploring temporal understanding in large language models with time intervals of events. arXiv preprint arXiv:2501.03040. find
  14. Albert Q. Jiang, Alexandre Sablayrolles, Arthur Mensch, and et al. 2023. Mistral 7b. arXiv preprint arXiv:2310.06825. find
  15. Kalev Leetaru and Philip A Schrodt. 2013. Gdelt: Global data on events, location, and tone, 1979–2012. In ISA annual convention, volume 2, pages 1–49. Citeseer. find
  16. Zhuoyuan Liu and Yilin Luo. 2024. Document-level event extraction with definition-driven icl. arXiv preprint arXiv:2408.05566. — [arXiv:2408.05566](https://arxiv.org/abs/2408.05566) find
  17. Jiayu Song, Mahmud Akhter, Dana Atzil Slonim, and Maria Liakata. 2024. Temporal reasoning for timeline summarisation in social media. arXiv preprint arXiv:2501.00152. find
  18. Qingyu Tan, Hwee Tou Ng, and Lidong Bing. 2023. Towards benchmarking and improving the temporal reasoning capability of large language models. arXiv preprint arXiv:2306.08952. find
  19. Gemma Team, Aishwarya Kamath, Johan Ferret, Shreya Pathak, Nino Vieillard, Ramona Merhej, Sarah Perrin, Tatiana Matejovicova, Alexandre Ramé, Morgane Rivière, and 1 others. 2025. Gemma 3 technical report. arXiv preprint arXiv:2503.19786. find
  20. Denny Vrandečić and Markus Krötzsch. 2014. Wikidata: a free collaborative knowledgebase. Communications of the ACM, 57(10):78–85. find
  21. Qifan Wang, Bhargav Kanagal, Vijay Garg, and D Sivakumar. 2019. Constructing a comprehensive events database from the web. In Proceedings of the 28th ACM International Conference on Information and Knowledge Management, pages 229–238. find
  22. Yuqing Wang and Yun Zhao. 2024. TRAM: Benchmarking temporal reasoning for large language models. In Findings of the Association for Computational Linguistics: ACL 2024, pages 6389–6415, Bangkok, Thailand. Association for Computational Linguistics. find
  23. Siheng Xiong, Ali Payani, Ramana Kompella, and Faramarz Fekri. 2024. Large language models can learn temporal reasoning. In Proceedings of the 62nd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 10452– 10470, Bangkok, Thailand. Association for Computat… find
  24. An Yang, Anfeng Li, Baosong Yang, and et al. 2025. Qwen3 technical report. arXiv preprint arXiv:2505.09388. find our card
  25. Ben Zhou, Daniel Khashabi, Qiang Ning, and Dan Roth. 2019. “going on a vacation” takes longer than “going for a walk”: A study of temporal commonsense understanding. In Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference o… find
  26. Overview of Appendices • Appendix A: Related Work. find
  27. OpenAI, Aaron Hurst, Adam Lerer, and et al. 2024. Gpt-4o system card. arXiv preprint arXiv:2410.21276. find our card
  28. Gemma Team: Morgane Riviere, Shreya Pathak, Pier Giuseppe Sessa, and et al. 2024. Gemma 2: Improving open language models at a practical size. arXiv preprint arXiv:2408.00118. find
  29. Appendix B: Detailed Dataset Analysis. • Appendix C: Prompt for Event Extraction. • Appendix D: Prompt for FactQA question generation. find

↑ back to top

This page in Markdown → https://md.research.elamur.ai/free/papers/mandal-2026-historybankqa-multilingual-temporal-question-answering/