Elamur Research⌕
☰
home / papers / 2026 / Think Parallax: Solving Multi-Hop Problems via Multi-View Knowledge-Graph-Based Retrieval-Augmented Generation

Think Parallax: Solving Multi-Hop Problems via Multi-View Knowledge-Graph-Based Retrieval-Augmented Generation

Paper · 2026 · Jinliang Liu, Jiale Bai and Shaoning Zeng

Summary

Knowledge-graph-based RAG (KG-RAG) systems often fail on multi-hop questions. They encode the question and all graph triples into one shared vector space, which blends together the semantics that each reasoning step needs. The authors observe that Transformer attention heads tend to take on different relations at different stages of reasoning, a "relay" in which some heads lead early hops and others lead later ones. They propose ParallaxRAG to exploit this. The question's [CLS] representation is split into per-head slices. Entity and relation texts pass through the same projection, so each head scores triples within its own aligned subspace. A Pairwise Similarity Regularization (PSR) term keeps the heads distinct. A weakly supervised, query-aware gate then weights the per-head scores to select a compact subgraph for the LLM generator. The method reports state-of-the-art retrieval and QA results on WebQSP and CWQ, with less hallucination. It also transfers zero-shot to the biomedical BioASQ benchmark. This suggests that modelling head-level specialization is a practical design choice for multi-hop KG retrieval.

Key points

  • Core design: query and KG triples are symmetrically decomposed into H head-specific views of dimension d/H. A shared linear projection aligns them, so each head compares query and graph in the same subspace, unlike MRAG's independent channels.
  • Full model results (Llama3.1-8B generator): WebQSP reaches 71.73 Macro-F1 and 86.85 Hit; CWQ reaches 48.33 Macro-F1 and 58.41 Hit.
  • Zero-shot transfer: on BioASQ, ParallaxRAG beats the previous SOTA by 7.32 Macro-F1.
  • Ablations:
    • Replacing the query-aware gate with simple averaging causes the largest drop, cutting Hit by 12.16 (WebQSP) and 20.15 (CWQ).
    • Removing PSR costs 0.73 and 1.21 Macro-F1.
    • Single-vector and split-vector baselines also underperform the full model.
  • Limitations and code: training uses shortest-path triples as weak supervision, and the method assumes static KGs. Multi-view retrieval helps less on shallow 1-hop queries. Code is available at github.com/LucaLiu1313/ParallaxRAG.

Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.

 Institutions

🎓University of Electronic Science and Technology of ChinaChengdu · China38 works · 2,088 citations

 Fields, topics and concepts

📚 Shelves 6The field of science this work belongs to (OpenAlex fields).Computer Science · 18,614 worksArtificial Intelligence · 17,965 worksArts and Humanities · 17,450 worksLanguage and Linguistics · 17,296 worksComputer Vision and Pattern Recognition · 3,486 worksInformation Systems · 1,730 works

🧭 Topics 2Research subjects and languages the work is about.Information retrieval & search · 552 worksKnowledge graphs & ontologies · 211 works

Cite

Cite as Jinliang Liu, Jiale Bai and Shaoning Zeng. 2026. Think Parallax: Solving Multi-Hop Problems via Multi-View Knowledge-Graph-Based Retrieval-Augmented Generation. Proceedings of the 64th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers).

The paper itself, not this page. BibTeX from the library's record of the work. DOI 10.18653/v1/2026.acl-long.1226

@inproceedings{liu2026think,
  title = {{Think Parallax: Solving Multi-Hop Problems via Multi-View Knowledge-Graph-Based Retrieval-Augmented Generation}},
  author = {Jinliang Liu, Jiale Bai and Shaoning Zeng},
  booktitle = {Proceedings of the 64th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers)},
  year = {2026},
  doi = {10.18653/v1/2026.acl-long.1226},
  url = {https://doi.org/10.18653/v1/2026.acl-long.1226}
}

References

42 references, 19 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

  1. Akari Asai, Zeqiu Wu, Yizhong Wang, Avirup Sil, and Hannaneh Hajishirzi. 2024. Self-rag: Learning to retrieve, generate, and critique through self-reflection.
  2. Lorenzo Basile, Valentino Maiorca, Diego Doimo, Francesco Locatello, and Alberto Cazzaniga. 2025. Head pursuit: Probing attention specialization in multimodal transformers. In Advances in Neural Information Processing Systems, volume 38. Spotlight. find
  3. Maciej Besta, Ales Kubicek, Robert Gerstenberger, Marcin Chrapek, Roman Niggli, Patrik Okanovic, Yi Zhu, Patrick Iff, Michal Podstawski, Lucas Weitzendorf, and 1 others. 2024. Multi-head rag: Solving multi-aspect problems with llms. arXiv preprint arXiv:2406.05085. — [arXiv:2406.05085](https://arxiv.org/abs/2406.05085) find
  4. Srinadh Bhojanapalli, Ayan Chakrabarti, Andreas Veit, Michal Lukasik, Himanshu Jain, Frederick Liu, Yin-Wen Chang, and Sanjiv Kumar. 2021. Leveraging redundancy in attention with reuse transformers. arXiv preprint arXiv:2110.06821. — [arXiv:2110.06821](https://arxiv.org/abs/2110.06821) find
  5. Yuchen Bian, Jiaji Huang, Xingyu Cai, Jiahong Yuan, and Kenneth Church. 2021. On attention redundancy: A comprehensive study. In Proceedings of the 2021 conference of the north american chapter of the association for computational linguistics: human language technologies, pages 930–945. find
  6. Bin Fu, Yunqi Qiu, Chengguang Tang, Yang Li, Haiyang Yu, and Jian Sun. 2020. A survey on complex question answering over knowledge base: Recent advances and challenges. arXiv preprint arXiv:2007.13069. find
  7. Ruiliu Fu, Han Wang, Xuejun Zhang, Jun Zhou, and Yonghong Yan. 2021. Decomposing complex questions makes multi-hop qa easier and more interpretable. arXiv preprint arXiv:2110.13472. find
  8. Yunfan Gao, Yun Xiong, Xinyu Gao, Kangxiang Jia, Jinliu Pan, Yuxi Bi, Yixin Dai, Jiawei Sun, Haofen Wang, and Haofen Wang. 2023. Retrieval-augmented generation for large language models: A survey. arXiv preprint arXiv:2312.10997, 2(1).
  9. Kai Guo, Harry Shomer, Shenglai Zeng, Haoyu Han, Yu Wang, and Jiliang Tang. 2025. Empowering GraphRAG with knowledge filtering and integration. In Proceedings of EMNLP 2025.
  10. Mufei Li, Siqi Miao, and Pan Li. 2024. Simple is effective: The roles of graphs and large language models in knowledge-graph-based retrieval-augmented generation. arXiv preprint arXiv:2410.20724. — [arXiv:2410.20724](https://arxiv.org/abs/2410.20724) find
  11. Haoyu Han, Kai Guo, Harry Shomer, Yu Wang, Yucheng Chu, Hang Li, Li Ma, and Jiliang Tang. 2026. Reasoning by exploration: A unified approach to retrieval and generation over graphs. In Proceedings of the ACM Web Conference 2026, WWW ’26. find
  12. Yuejie Li, Ke Yang, Tao Wang, Bolin Chen, Bowen Li, and Chengjun Mao. 2026. Deep GraphRAG: A balanced approach to hierarchical retrieval and adaptive integration. arXiv preprint arXiv:2601.11144. — [arXiv:2601.11144](https://arxiv.org/abs/2601.11144) find
  13. Xiaoxin He, Yijun Tian, Yifei Sun, Nitesh Chawla, Thomas Laurent, Yann LeCun, Xavier Bresson, and Bryan Hooi. 2024. G-retriever: Retrieval-augmented generation for textual graph understanding and question answering. Advances in Neural Information Processing Systems, 37:132876–132907.
  14. Gautier Izacard and Edouard Grave. 2020. Leveraging passage retrieval with generative models for open domain question answering. arXiv preprint arXiv:2007.01282.
  15. Gautier Izacard, Patrick Lewis, Maria Lomeli, Lucas Hosseini, Fabio Petroni, Timo Schick, Jane Dwivedi-Yu, Armand Joulin, Sebastian Riedel, and Edouard Grave. 2022. Few-shot learning with retrieval augmented language models. arXiv preprint arXiv:2208.03299, 1(2):4.
  16. Shaoxiong Ji, Shirui Pan, Erik Cambria, Pekka Marttinen, and Philip S Yu. 2021. A survey on knowledge graphs: Representation, acquisition, and applications. IEEE transactions on neural networks and learning systems, 33(2):494–514. find
  17. Jinhao Jiang, Kun Zhou, Zican Dong, Keming Ye, Wayne Xin Zhao, and Ji-Rong Wen. 2023. Structgpt: A general framework for large language model to reason over structured data. arXiv preprint arXiv:2305.09645. find
  18. Jinhao Jiang, Kun Zhou, Wayne Xin Zhao, and Ji-Rong Wen. 2022. Unikgqa: Unified retrieval and reasoning for solving multi-hop question answering over knowledge graph. arXiv preprint arXiv:2212.00959. find
  19. Vladimir Karpukhin, Barlas Oguz, Sewon Min, Patrick SH Lewis, Ledell Wu, Sergey Edunov, Danqi Chen, and Wen-tau Yih. 2020. Dense passage retrieval for open-domain question answering. In EMNLP (1), pages 6769–6781.
  20. Omar Khattab and Matei Zaharia. 2020. Colbert: Efficient and effective passage search via contextualized late interaction over bert. In Proceedings of the 43rd International ACM SIGIR conference on research and development in Information Retrieval, pages 39– 48.
  21. Patrick Lewis, Ethan Perez, Aleksandra Piktus, Fabio Petroni, Vladimir Karpukhin, Naman Goyal, Heinrich Küttler, Mike Lewis, Wen-tau Yih, Tim Rocktäschel, and 1 others. 2020. Retrieval-augmented generation for knowledge-intensive nlp tasks. Advances in neural information processing systems, 33:9459–…
  22. Tsung-Yi Lin, Priya Goyal, Ross Girshick, Kaiming He, and Piotr Dollár. 2017. Focal loss for dense object detection. In Proceedings of the IEEE International Conference on Computer Vision (ICCV), pages 2980– 2988.
  23. Guangyi Liu, Yongqi Zhang, Yong Li, and Quanming Yao. 2024. Explore then determine: A gnn-llm synergy framework for reasoning over knowledge graph. arXiv e-prints, pages arXiv–2406. find
  24. Nelson F Liu, Kevin Lin, John Hewitt, Ashwin Paranjape, Michele Bevilacqua, Fabio Petroni, and Percy Liang. 2023. Lost in the middle: How language models use long contexts. arXiv preprint arXiv:2307.03172.
  25. Linhao Luo, Yuan-Fang Li, Gholamreza Haffari, and Shirui Pan. 2023. Reasoning on graphs: Faithful and interpretable large language model reasoning. arXiv preprint arXiv:2310.01061. find
  26. Costas Mavromatis and George Karypis. 2024. Gnn-rag: Graph neural retrieval for large language model reasoning. arXiv preprint arXiv:2405.20139. find
  27. Geondo Park, Chihye Han, Wonjun Yoon, and Daeshik Kim. 2020. Mhsan: Multi-head self-attention network for visual semantic embedding. In Proceedings of the IEEE/CVF winter conference on applications of computer vision, pages 1518–1526. find
  28. Boci Peng, Yun Zhu, Yongchao Liu, Xiaohe Bo, Haizhou Shi, Chuntao Hong, Yan Zhang, and Siliang Tang. 2024. Graph retrieval-augmented generation: A survey. arXiv preprint arXiv:2408.08921.
  29. Ethan Perez, Patrick Lewis, Wen-tau Yih, Kyunghyun Cho, and Douwe Kiela. 2020. Unsupervised question decomposition for question answering. arXiv preprint arXiv:2002.09758.
  30. Fabio Petroni, Aleksandra Piktus, Angela Fan, Patrick Lewis, Majid Yazdani, Nicola De Cao, James Thorne, Yacine Jernite, Vladimir Karpukhin, Jean Maillard, and 1 others. 2020. Kilt: a benchmark for knowledge intensive language tasks. arXiv preprint arXiv:2009.02252.
  31. Jiashuo Sun, Chengjin Xu, Lumingyuan Tang, Saizhuo Wang, Chen Lin, Yeyun Gong, Lionel M Ni, Heung-Yeung Shum, and Jian Guo. 2023. Think-on-graph: Deep and responsible reasoning of large language model on knowledge graph. arXiv preprint arXiv:2307.07697. find
  32. Alon Talmor and Jonathan Berant. 2018. The web as a knowledge-base for answering complex questions. In Proceedings of the 2018 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long Papers), pages 641–651. prompting of l…
  33. Harsh Trivedi, Niranjan Balasubramanian, Tushar Khot, and Ashish Sabharwal. 2022. Interleaving retrieval with chain-of-thought reasoning for knowledge-intensive multi-step questions. arXiv preprint arXiv:2212.10509.
  34. Zifan Zheng, Yezhaohui Wang, Yuxin Huang, Shichao Song, Mingchuan Yang, Bo Tang, Feiyu Xiong, and Zhiyu Li. 2025. Attention heads of large language models: a survey. Patterns, 6(2):101176.
  35. George Tsatsaronis, Georgios Balikas, Prodromos Malakasiotis, and 1 others. 2015. An overview of the BIOASQ large-scale biomedical semantic indexing and question answering competition. BMC Bioinformatics, 16(1):138. find
  36. Varun Vashisht, Samar Singh, Mihir Konduskar, Jaskaran Singh Walia, and Vukosi Marivate. 2025. Mage: Multi-head attention guided embeddings for low resource sentiment classification. arXiv preprint arXiv:2502.17987. — [arXiv:2502.17987](https://arxiv.org/abs/2502.17987) find
  37. Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N Gomez, Łukasz Kaiser, and Illia Polosukhin. 2017. Attention is all you need. Advances in neural information processing systems, 30.
  38. Jason Wei, Xuezhi Wang, Dale Schuurmans, Maarten Bosma, Fei Xia, Ed Chi, Quoc V Le, Denny Zhou, and 1 others. 2022. Chain-of-thought prompting elicits reasoning in large language models. Advances in neural information processing systems, 35:24824– 24837.
  39. Yike Wu, Nan Hu, Sheng Bi, Guilin Qi, Jie Ren, An-huan Xie, and Wei Song. 2023. Retrieve-rewrite-answer: A kg-to-text enhanced llms framework for knowledge graph question answering. arXiv preprint arXiv:2309.11206. find
  40. Tianjun Yao, Haoxuan Li, Zhiqiang Shen, Pan Li, Tongliang Liu, and Kun Zhang. 2025. Learning efficient and generalizable graph retriever for knowledge-graph question answering. Preprint, arXiv:2506.09645. — [arXiv:2506.09645](https://arxiv.org/abs/2506.09645) find
  41. Wen-tau Yih, Ming-Wei Chang, Xiaodong He, and Jianfeng Gao. 2016. Value-based neighborhood expansion: A survey. In Proceedings of the Thirtieth AAAI Conference on Artificial Intelligence, pages 1780– 1786. find
  42. Jing Zhang, Xiaokang Zhang, Jifan Yu, Jian Tang, Jie Tang, Cuiping Li, and Hong Chen. 2022. Subgraph retrieval enhanced model for multi-hop knowledge base question answering. In Proceedings of the 60th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 577… find

Updated

↑ back to top

This page in Markdown → https://md.research.elamur.ai/free/en/papers/liu-2026-think-parallax-solving-multi-hop/