面向时序知识图谱推理的本体-图谱双智能体强化学习方法

Summary

The paper addresses temporal knowledge graph (TKG) extrapolation, which means forecasting missing subjects or objects of quadruples at future timestamps. Existing reinforcement-learning (RL) path-search methods operate only on entities. As a result, their action spaces grow rapidly with path length and their paths lack concept-level logic. Ontologies could supply that logic, but they are usually built once from training-set counts and never revised. The authors propose ODRT, which pairs two cooperating agents. An upper ontology agent walks over a concept graph that mirrors the TKG. Its history state and suggested target concepts or relations softly guide a lower graph agent, which searches entity-level paths over historical snapshots under hierarchical soft rewards. The ontology is seeded by clustering entities using LLM semantic embeddings combined with structural embeddings from a pretrained TKG model. Its edge weights are then refined end-to-end during training. On four public benchmarks, ODRT outperforms all RL-based baselines and sets the state of the art on the two encyclopedic datasets. It still trails some embedding methods on event-style data, where reusing history matters most.

Key points

• Concepts are obtained by clustering entities. Their features combine LLM text embeddings, projected down to the structural dimension, with embeddings from a pretrained TKG reasoner such as TTransE.
• The ontology graph (concepts, the original relation set, and aggregated concept-level facts) carries no timestamps. This filters out short-lived noise and keeps patterns that stay stable over time.
• During training, the entity-to-concept mapping stays fixed, and only learnable frequency weights on ontology edges are updated. These weights start as normalized training-set counts.
• Evaluation uses link prediction on 4 public TKG datasets. ODRT is best among RL methods (e.g., TITer-style baselines) and best overall on 2 encyclopedic datasets, but weaker than some embedding models on event-based data.
• The work was published in Journal of Computer Research and Development, 63(9): 2248–2263, 2026, by authors from the National University of Defense Technology and Nanjing University.