Elamur Research⌕
☰
home / papers / 2025 / RTQA : Recursive Thinking for Complex Temporal Knowledge Graph Question Answering with Large Language Models

RTQA : Recursive Thinking for Complex Temporal Knowledge Graph Question Answering with Large Language Models

Paper · 2025 · Zhaoyan Gong, Juan Li, Zhiqiang Liu et al.

Summary

Questions over temporal knowledge graphs (TKGs) often combine several time conditions, such as an implicit anchor event plus an ordering requirement like "last." Existing TKGQA systems, including LLM-based ones, tend to hallucinate on such questions. Decomposition-based systems also let a mistake in an early sub-step carry through to the final answer.

RTQA is a training-free, plug-and-play framework that addresses both problems with three components:

  • The Temporal Question Decomposer splits a question into a tree of simpler sub-questions and rewrites implicit constraints as explicit time expressions.
  • The Recursive Solver answers the leaves first, using an LLM supported by retrieved TKG quadruples. It then substitutes those answers into placeholders in the parent questions.
  • The Answer Aggregator works on every non-atomic node. It weighs the answer derived directly from retrieved facts against the one assembled from the child nodes and keeps the more trustworthy option, which limits error propagation.

On MultiTQ and TimelineKGQA, RTQA outperforms state-of-the-art baselines in Hits@1, most clearly on the "Multiple" and "Complex" question categories. It also works with different LLMs. This matters for applications that need dependable time-aware reasoning without fine-tuning.

Key points

  • The framework has three modules (Temporal Question Decomposer, Recursive Solver, Answer Aggregator); code and data are released at github.com/zjukg/RTQA.
  • Benchmarks are MultiTQ and TimelineKGQA, with Hits@1 as the main metric; the largest gains are in the "Multiple" and "Complex" categories.
  • Overhead is low. MultiTQ questions average depth 1.37, branching 1.60 and 3.96 API calls. TimelineKGQA questions average depth 1.57, branching 1.81 and 5.38 calls.
  • Retrieval size shows a trade-off. Using the top n=50 facts gives the best Hits@1 (77.8%, Recall@n 71.78%). At n=60, recall rises to 73.95% but Hits@1 drops to 76.0% because of added noise.
  • The error analysis names five failure types: evaluation (alias) errors, annotation errors, retrieval errors, temporal reasoning hallucinations and faulty decomposition. The listed limitations are dependence on the LLM's size and capability and on retriever quality.

Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.

 Institutions

🎓Zhejiang UniversityHangzhou · China172 works · 12,988 citations🏢Ant GroupHangzhou · China42 works · 1,131 citations

 Fields, topics and concepts

📚 Shelves 4The field of science this work belongs to (OpenAlex fields).Computer Science · 16,667 worksArtificial Intelligence · 16,194 worksArts and Humanities · 16,039 worksLanguage and Linguistics · 15,891 works

🧭 Topics 2Research subjects and languages the work is about.Question answering · 441 worksKnowledge graphs & ontologies · 192 works

Cite

Cite as Zhaoyan Gong, Juan Li, Zhiqiang Liu, Lei Liang, Huajun Chen and Wen Zhang. 2025. RTQA : Recursive Thinking for Complex Temporal Knowledge Graph Question Answering with Large Language Models. Proceedings of the 2025 Conference on Empirical Methods in Natural Language Processing.

The paper itself, not this page. BibTeX from the library's record of the work. DOI 10.18653/v1/2025.emnlp-main.499

@inproceedings{gong2025rtqa,
  title = {{RTQA : Recursive Thinking for Complex Temporal Knowledge Graph Question Answering with Large Language Models}},
  author = {Zhaoyan Gong, Juan Li, Zhiqiang Liu, Lei Liang, Huajun Chen and Wen Zhang},
  booktitle = {Proceedings of the 2025 Conference on Empirical Methods in Natural Language Processing},
  year = {2025},
  doi = {10.18653/v1/2025.emnlp-main.499},
  url = {https://doi.org/10.18653/v1/2025.emnlp-main.499}
}

References

32 references, 7 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

  1. James F. Allen. 1984. Towards a general theory of action and time. Artif. Intell., 23(2):123–154. find
  2. Amos Azaria, Rina Azoulay, and Shulamit Reches. 2024. Chatgpt is a remarkable tool—for experts. Data Intelligence, 6(1):240–296. find
  3. Elizabeth Boschee, Jennifer Lautenschlager, Sean O’Brien, Steve Shellman, James Starz, and Michael Ward. 2015. ICEWS Coded Event Data. find
  4. Shulin Cao, Jiaxin Shi, Liangming Pan, Lunyiu Nie, Yutong Xiang, Lei Hou, Juanzi Li, Bin He, and Hanwang Zhang. 2022. KQA pro: A dataset with explicit compositional programs for complex question answering over knowledge base. In ACL (1), pages 6101–6119. Association for Computational Linguistics. find
  5. Shulin Cao, Jiajie Zhang, Jiaxin Shi, Xin Lv, Zijun Yao, Qi Tian, Lei Hou, and Juanzi Li. 2023. Probabilistic tree-of-thought reasoning for answering knowledge-intensive complex questions. In EMNLP (Findings), pages 12541–12560. Association for Computational Linguistics. find
  6. Huajun Chen. 2024. Large knowledge model: Perspectives and challenges. Data Intelligence, 6(3):587–620. find
  7. Jianlv Chen, Shitao Xiao, Peitian Zhang, Kun Luo, Defu Lian, and Zheng Liu. 2024a. BGE m3-embedding: Multi-lingual, multi-functionality, multi-granularity text embeddings through self-knowledge distillation. CoRR, abs/2402.03216.
  8. Zhuo Chen, Zhao Zhang, Zixuan Li, Fei Wang, Yutao Zeng, Xiaolong Jin, and Yongjun Xu. 2024b. Self-improvement programming for temporal knowledge graph question answering. In LREC/COLING, pages 14579–14594. ELRA and ICCL. find
  9. Ziyang Chen, Dongfang Li, Xiang Zhao, Baotian Hu, and Min Zhang. 2024c. Temporal knowledge question answering via abstract reasoning induction. In ACL (1), pages 4872–4889. Association for Computational Linguistics. find
  10. Ziyang Chen, Jinzhi Liao, and Xiang Zhao. 2023. Multi-granularity temporal question answering over knowledge graphs. In ACL (1), pages 11378–11392. Association for Computational Linguistics. find
  11. Ziyang Chen, Xiang Zhao, Jinzhi Liao, Xinyi Li, and Evangelos Kanoulas. 2022. Temporal knowledge graph question answering via subgraph reasoning. Knowl. Based Syst., 251:109134. find
  12. Jacob Devlin, Ming-Wei Chang, Kenton Lee, and Kristina Toutanova. 2019. BERT: pre-training of deep bidirectional transformers for language understanding. In NAACL-HLT (1), pages 4171–4186. Association for Computational Linguistics.
  13. Wentao Ding, Hao Chen, Huayu Li, and Yuzhong Qu. 2022. Semantic framework based query generation for temporal question answering over knowledge graphs. In EMNLP, pages 1867–1877. Association for Computational Linguistics. find
  14. Matthijs Douze, Alexandr Guzhva, Chengqi Deng, Jeff Johnson, Gergely Szilvasy, Pierre-Emmanuel Mazaré, Maria Lomeli, Lucas Hosseini, and Hervé Jégou. 2024. The faiss library. CoRR, abs/2401.08281.
  15. Yifu Gao, Linbo Qiao, Zhigang Kan, Zhihua Wen, Yongquan He, and Dongsheng Li. 2024. Two-stage generative question answering on temporal knowledge graph using large language models. In ACL (Findings), pages 6719–6734. Association for Computational Linguistics. find
  16. Costas Mavromatis, Prasanna Lakkur Subramanyam, Vassilis N. Ioannidis, Adesoji Adeshina, Phillip Ryan Howard, Tetiana Grinberg, Nagib Hakim, and George Karypis. 2022. Tempoqr: Temporal question reasoning over knowledge graphs. In AAAI, pages 5825–5833. AAAI Press. find
  17. Sumit Neelam, Udit Sharma, Hima Karanam, Shajith Ikbal, Pavan Kapanipathi, Ibrahim Abdelaziz, Nandana Mihindukulasooriya, Young-Suk Lee, Santosh K. Srivastava, Cezar Pendus, Saswati Dana, Dinesh Garg, Achille Fokoue, G. P. Shrivatsa Bhargav, Dinesh Khandelwal, Srinivas Ravishankar, Sairam Gurajada,… — [arXiv:2109.13430](https://arxiv.org/abs/2109.13430) find
  18. Xinying Qian, Ying Zhang, Yu Zhao, Baohang Zhou, Xuhui Sui, Li Zhang, and Kehui Song. 2024. Timer4 : Time-aware retrieval-augmented large language models for temporal knowledge graph question answering. In EMNLP, pages 6942–6952. Association for Computational Linguistics. find
  19. Victor Sanh, Lysandre Debut, Julien Chaumond, and Thomas Wolf. 2019. Distilbert, a distilled version of BERT: smaller, faster, cheaper and lighter. CoRR, abs/1910.01108. find
  20. Apoorv Saxena, Soumen Chakrabarti, and Partha P. Talukdar. 2021. Question answering over temporal knowledge graphs. In ACL/IJCNLP (1), pages 6663– 6676. Association for Computational Linguistics. find
  21. Alberto García-Durán, Sebastijan Dumancic, and Mathias Niepert. 2018. Learning sequence encoders for temporal knowledge graph completion. In EMNLP, pages 4816–4821. Association for Computational Linguistics. find
  22. Apoorv Saxena, Aditay Tripathi, and Partha P. Talukdar. 2020. Improving multi-hop question answering over knowledge graphs using knowledge base embeddings. In ACL, pages 4498–4507. Association for Computational Linguistics.
  23. Zhen Jia, Abdalghani Abujabal, Rishiraj Saha Roy, Jannik Strötgen, and Gerhard Weikum. 2018. TEQUILA: temporal question answering over knowledge bases. In CIKM, pages 1807–1810. ACM. find
  24. Aditya Sharma, Apoorv Saxena, Chitrank Gupta, Seyed Mehran Kazemi, Partha P. Talukdar, and Soumen Chakrabarti. 2023. Twirgcn: Temporally weighted graph convolution for question answering over temporal knowledge graphs. In EACL, pages 2041–2052. Association for Computational Linguistics. find
  25. Zhen Jia, Philipp Christmann, and Gerhard Weikum. 2024. Faithful temporal question answering over heterogeneous sources. In WWW, pages 2052–2063. ACM. find
  26. Tushar Khot, Harsh Trivedi, Matthew Finlayson, Yao Fu, Kyle Richardson, Peter Clark, and Ashish Sabharwal. 2023. Decomposed prompting: A modular approach for solving complex tasks. In ICLR. OpenReview.net. find
  27. Zhenzhong Lan, Mingda Chen, Sebastian Goodman, Kevin Gimpel, Piyush Sharma, and Radu Soricut. 2020. ALBERT: A lite BERT for self-supervised learning of language representations. In ICLR. OpenReview.net.
  28. Yonghao Liu, Di Liang, Mengyu Li, Fausto Giunchiglia, Ximing Li, Sirui Wang, Wei Wu, Lan Huang, Xiaoyue Feng, and Renchu Guan. 2023. Local and global: Temporal question answering via information fusion. In IJCAI, pages 5141–5149. ijcai.org. find
  29. Qiang Sun, Sirui Li, Du Huynh, Mark Reynolds, and Wei Liu. 2025. Timelinekgqa: A comprehensive question-answer pair generator for temporal knowledge graphs. CoRR, abs/2501.04343. — [arXiv:2501.04343](https://arxiv.org/abs/2501.04343) find
  30. Hugo Touvron, Louis Martin, Kevin Stone, Peter Albert, Amjad Almahairi, Yasmine Babaei, Nikolay Bashlykov, Soumya Batra, Prajjwal Bhargava, Shruti Bhosale, Dan Bikel, Lukas Blecher, Cristian Canton-Ferrer, Moya Chen, Guillem Cucurull, David Esiobu, Jude Fernandes, Jeremy Fu, Wenyin Fu, and 49 others… — Llama 2: Open Foundation and Fine-Tuned Chat Models
  31. Harsh Trivedi, Niranjan Balasubramanian, Tushar Khot, and Ashish Sabharwal. 2022. Musique: Multihop questions via single-hop question composition. Trans. Assoc. Comput. Linguistics, 10:539–554.
  32. Suifeng Zhao, Tong Zhou, Zhuoran Jin, Hongbang Yuan, Yubo Chen, Kang Liu, and Sujian Li. 2024. Awecita: Generating answer with appropriate and well-grained citations using llms. Data Intelligence, 6(4):1134– 1157. find

Updated

↑ back to top

This page in Markdown → https://md.research.elamur.ai/free/en/papers/gong-2025-rtqa-recursive-thinking-complex-temporal/