Elamur Research⌕
☰
home / papers / 2021 / Question Answering Over Temporal Knowledge Graphs

Question Answering Over Temporal Knowledge Graphs

Paper · 2021 · Apoorv Saxena, Soumen Chakrabarti and Partha Talukdar

Summary

The paper addresses question answering over Temporal Knowledge Graphs, where each fact has a time span. Progress here has been held back by a lack of large datasets. The authors release CronQuestions, a resource built from a WikiData subset. It has two parts:

  • a temporal KG with year-level timestamps, to which important world events such as wars were added as extra facts;
  • 410k natural language questions, generated from seed templates and then paraphrased by humans and by machine.

The questions are grouped by reasoning structure, from simple entity or time lookups to before/after, first/last and time-join questions. Strong baselines (T5, BERT, KnowBERT, Entities-as-Experts variants and EmbedKGQA) handle the temporal aspect poorly. The authors therefore propose CronKGQA, an extension of EmbedKGQA that pairs a fine-tuned BERT question encoder with TComplEx temporal KG embeddings. CronKGQA solves simple questions almost perfectly and more than doubles the overall accuracy of the best baseline. Complex reasoning, however, remains far from solved. The work matters because it supplies the first training-scale benchmark for temporal KGQA and shows that temporal embeddings are useful for the task.

Key points

  • CronQuestions KG and question set: the KG contains 328k facts (about 5k of them event facts), 125k entities and 203 relations. The questions come from 30 seed templates covering five relations and five reasoning types. They are split into 350k train, 30k dev and 30k test questions.
  • Scale: the dataset is about 340× larger than TempQuestions (1,271 questions), which was the only earlier temporal KGQA set and is based on the non-temporal FreeBase.
  • Overall result: CronKGQA reaches 0.647 overall hits@1, compared with 0.288 for the best baselines (EmbedKGQA, T-EaE-replace). This is roughly a 120% gain.
  • By question type: CronKGQA scores about 0.99 hits@1 on simple questions. Before/after questions remain the hardest, with a best score of 0.288.
  • Temporal vs. non-temporal embeddings: replacing TComplEx with the non-temporal ComplEx lowers CronKGQA to 0.288 overall. Accuracy on questions with time answers falls to 0.057.
  • Availability: code and data are available at github.com/apoorvumang/CronKGQA.

Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.

 Institutions

🏢GoogleMountain View · United States533 works · 443,884 citations🎓Indian Institute of Technology BombayMumbai · India32 works · 2,535 citations🎓Indian Institute of Science BangaloreBengaluru · India16 works · 1,462 citations

 Fields, topics and concepts

📚 Shelves 4The field of science this work belongs to (OpenAlex fields).Computer Science · 19,503 worksArtificial Intelligence · 18,805 worksArts and Humanities · 17,959 worksLanguage and Linguistics · 17,789 works

🧭 Topics 2Research subjects and languages the work is about.Question answering · 491 worksKnowledge graphs & ontologies · 218 works

Cite

Cite as Apoorv Saxena, Soumen Chakrabarti and Partha Talukdar. 2021. Question Answering Over Temporal Knowledge Graphs. Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Conference on Natural Language Processing (Volume 1: Long Papers).

The paper itself, not this page. BibTeX from the library's record of the work. DOI 10.18653/v1/2021.acl-long.520 · arXiv 2106.01515

@inproceedings{saxena2021question,
  title = {{Question Answering Over Temporal Knowledge Graphs}},
  author = {Apoorv Saxena, Soumen Chakrabarti and Partha Talukdar},
  booktitle = {Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Conference on Natural Language Processing (Volume 1: Long Papers)},
  year = {2021},
  doi = {10.18653/v1/2021.acl-long.520},
  eprint = {2106.01515},
  archivePrefix = {arXiv},
  url = {https://doi.org/10.18653/v1/2021.acl-long.520}
}

References

2 references, 1 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

  1. Junwei Bao, Nan Duan, Zhao Yan, Ming Zhou, and Tiejun Zhao. 2016. Constraint-based question answering with knowledge graph. In Proceedings of COLING 2016, the 26th International Conference on Computational Linguistics: Technical Papers, pages 2503–2514, Osaka, Japan.
  2. The COLING 2016 Organizing Committee. *Table 7: Hits@1 for C RON KGQA and T-EaE-replace using ComplEx(CX) and TComplEx(TCX) KG embeddings. Please refer to Section 6.5 for more details.* find

Updated

This page in Markdown → https://md.research.elamur.ai/free/en/papers/saxena-2021-question-answering-over-temporal-knowledge/