Elamur Research⌕
☰
home / papers / 2026 / NeuRAG: End-to-End Neural Knowledge Augmentation via Hyper-Neurons

NeuRAG: End-to-End Neural Knowledge Augmentation via Hyper-Neurons

Paper · 2026 · Liwei Zheng, Xuemin Liu and Jie Liu

Summary

Standard RAG pipelines keep retrieval and generation apart. Retrieved passages are simply placed in the prompt, so the model has to re-read them on every query. This is limited by context length and handles questions that combine evidence from several sources poorly. NeuRAG (Nankai University) moves retrieval inside the model. Offline, each document is turned into its own small LoRA module (a "Hyper-Neuron"), following the PRAG approach. Each module is keyed by the base LLM's last-layer hidden state for the document's final token. During decoding, a "Knowledge Hyper-Layer" uses the current hidden state to score these modules, picks the top-k, merges them in parameter space and injects them into the FFN layers. Re-selection happens at the first output token and afterwards only when a combined signal of hidden-state drift and predictive entropy passes a threshold. The authors report strong, consistent results across several datasets and LLMs; the exact scores are not in the extracted text. The work matters because it treats retrieval as on-the-fly reconfiguration of model parameters, trainable end to end.

Key points

  • Trigger score: S_t blends normalized hidden-state drift and token entropy through a mixing factor λ. The best value is around λ ≈ 0.6, and using either signal alone performs worse.
  • Threshold τ: values of 0.8 and above (but below 1.0) work best. Setting τ = 1.0 turns reconfiguration off and lowers performance, while low thresholds cause unstable, frequent switching between modules.
  • Top-k: varying the number of fused Hyper-Neurons from 1 to 5 changes results only mildly. Fusing many modules can dilute the contribution of individual documents.
  • Evaluation setup: analyses cover 2WikiMultihopQA and ComplexWebQuestions with LLaMA3.2-1B-Instruct. Extra latency versus PRAG is under one second in most settings.
  • Limitations: the method needs access to hidden states, so it cannot be used with closed-source LLMs. Storing one LoRA per document raises memory and scalability concerns.

Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.

 Institutions

🎓Nankai UniversityTianjin · China36 works · 762 citations

 Fields, topics and concepts

📚 Shelves 6The field of science this work belongs to (OpenAlex fields).Computer Science · 18,614 worksArtificial Intelligence · 17,965 worksArts and Humanities · 17,450 worksLanguage and Linguistics · 17,296 worksComputer Vision and Pattern Recognition · 3,486 worksInformation Systems · 1,730 works

Cite

Cite as Liwei Zheng, Xuemin Liu and Jie Liu. 2026. NeuRAG: End-to-End Neural Knowledge Augmentation via Hyper-Neurons. Findings of the Association for Computational Linguistics: ACL 2026.

The paper itself, not this page. BibTeX from the library's record of the work. DOI 10.18653/v1/2026.findings-acl.1516

@inproceedings{zheng2026neurag,
  title = {{NeuRAG: End-to-End Neural Knowledge Augmentation via Hyper-Neurons}},
  author = {Liwei Zheng, Xuemin Liu and Jie Liu},
  booktitle = {Findings of the Association for Computational Linguistics: ACL 2026},
  year = {2026},
  doi = {10.18653/v1/2026.findings-acl.1516},
  url = {https://doi.org/10.18653/v1/2026.findings-acl.1516}
}

References

26 references, 17 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

  1. Akari Asai, Zeqiu Wu, Yizhong Wang, Avirup Sil, and Hannaneh Hajishirzi. 2024. Self-rag: Learning to retrieve, generate, and critique through self-reflection.
  2. Sebastian Borgeaud, Arthur Mensch, Jordan Hoffmann, Trevor Cai, Eliza Rutherford, Katie Millican, George Bm Van Den Driessche, Jean-Baptiste Lespiau, Bogdan Damoc, Aidan Clark, and 1 others. 2022. Improving language models by retrieving from trillions of tokens. In International conference on machin… find
  3. Tom Brown, Benjamin Mann, Nick Ryder, Melanie Subbiah, Jared D Kaplan, Prafulla Dhariwal, Arvind Neelakantan, Pranav Shyam, Girish Sastry, Amanda Askell, and 1 others. 2020. Language models are few-shot learners. Advances in neural information processing systems, 33:1877–1901.
  4. Jianlv Chen, Shitao Xiao, Peitian Zhang, Kun Luo, Defu Lian, and Zheng Liu. 2024. Bge m3-embedding: Multi-lingual, multi-functionality, multi-granularity text embeddings through self-knowledge distillation. arXiv preprint arXiv:2402.03216.
  5. Aakanksha Chowdhery, Sharan Narang, Jacob Devlin, Maarten Bosma, Gaurav Mishra, Adam Roberts, Paul Barham, Hyung Won Chung, Charles Sutton, Sebastian Gehrmann, and 1 others. 2023. Palm: Scaling language modeling with pathways. Journal of Machine Learning Research, 24(240):1–113.
  6. James Ferguson, Matt Gardner, Hannaneh Hajishirzi, Tushar Khot, and Pradeep Dasigi. 2020. IIRC: A dataset of incomplete information reading comprehension questions. In Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing (EMNLP), pages 1137–1147, Online. find
  7. William Fleshman and Benjamin Van Durme. 2025. Lora-augmented generation (lag) for knowledge-intensive language tasks. arXiv preprint arXiv:2507.05346. — [arXiv:2507.05346](https://arxiv.org/abs/2507.05346) find
  8. Mor Geva, Daniel Khashabi, Elad Segal, Tushar Khot, Dan Roth, and Jonathan Berant. 2021. Did aristotle use a laptop? a question answering benchmark with implicit reasoning strategies. Transactions of the Association for Computational Linguistics, 9:346– 361. find
  9. Kelvin Guu, Kenton Lee, Zora Tung, Panupong Pasupat, and Mingwei Chang. 2020. Retrieval augmented language model pre-training. In International conference on machine learning, pages 3929–3938. PMLR.
  10. Xanh Ho, Anh-Khoa Duong Nguyen, Saku Sugawara, and Akiko Aizawa. 2020. Constructing a multi-hop qa dataset for comprehensive evaluation of reasoning steps. arXiv preprint arXiv:2011.01060.
  11. Gautier Izacard and Edouard Grave. 2021. Leveraging passage retrieval with generative models for open domain question answering. In Proceedings of the 16th conference of the european chapter of the association for computational linguistics: main volume, pages 874–880.
  12. Soyeong Jeong, Jinheon Baek, Sukmin Cho, Sung Ju Hwang, and Jong C Park. 2024. Adaptive-rag: Learning to adapt retrieval-augmented large language models through question complexity. arXiv preprint arXiv:2403.14403.
  13. Zhengbao Jiang, Frank F Xu, Luyu Gao, Zhiqing Sun, Qian Liu, Jane Dwivedi-Yu, Yiming Yang, Jamie Callan, and Graham Neubig. 2023. Active retrieval augmented generation. In Proceedings of the 2023 Conference on Empirical Methods in Natural Language Processing, pages 7969–7992.
  14. Vladimir Karpukhin, Barlas Oguz, Sewon Min, Patrick SH Lewis, Ledell Wu, Sergey Edunov, Danqi Chen, and Wen-tau Yih. 2020. Dense passage retrieval for open-domain question answering. In EMNLP (1), pages 6769–6781.
  15. Patrick Lewis, Ethan Perez, Aleksandra Piktus, Fabio Petroni, Vladimir Karpukhin, Naman Goyal, Heinrich Küttler, Mike Lewis, Wen-tau Yih, Tim Rocktäschel, and 1 others. 2020. Retrieval-augmented generation for knowledge-intensive nlp tasks. Advances in neural information processing systems, 33:9459–…
  16. Meta. 2024a. Llama-3.2-1b-instruct. https://huggingface.co/meta-llama/Llama-3. 2-1B-Instruct. Accessed: 2024-09. ↗
  17. Meta. 2024b. Meta-llama-3-8b-instruct. https://huggingface.co/meta-llama/ Meta-Llama-3-8B-Instruct. Accessed: 2024-04. ↗
  18. Harsh Trivedi, Niranjan Balasubramanian, Tushar Khot, and Ashish Sabharwal. 2023. Interleaving retrieval with chain-of-thought reasoning for knowledge-intensive multi-step questions. In Proceedings of the 61st annual meeting of the association for computational linguistics (volume 1: long papers), p…
  19. An Yang, Anfeng Li, Baosong Yang, Beichen Zhang, Binyuan Hui, Bo Zheng, Bowen Yu, Chang Gao, Chengen Huang, Chenxu Lv, and 1 others. 2025. Qwen3 technical report. arXiv preprint arXiv:2505.09388.
  20. Zhilin Yang, Peng Qi, Saizheng Zhang, Yoshua Bengio, William Cohen, Ruslan Salakhutdinov, and Christopher D Manning. 2018. Hotpotqa: A dataset for diverse, explainable multi-hop question answering. In Proceedings of the 2018 conference on empirical methods in natural language processing, pages 2369–…
  21. Stephen E. Robertson, Steve Walker, Susan Jones, Micheline Hancock-Beaulieu, and Mike Gatford. 1994. Okapi at TREC-3. In Proceedings of The Third Text REtrieval Conference, TREC 1994, Gaithersburg, Maryland, USA, November 2-4, 1994, volume 500-225 of NIST Special Publication, pages 109– 126. Nationa…
  22. Weihang Su, Yichen Tang, Qingyao Ai, Zhijing Wu, and Yiqun Liu. 2024. Dragin: dynamic retrieval augmented generation based on the information needs of large language models. arXiv preprint arXiv:2403.10081. — [arXiv:2403.10081](https://arxiv.org/abs/2403.10081) find
  23. Weihang Su, Yichen Tang, Qingyao Ai, Junxi Yan, Changyue Wang, Hongning Wang, Ziyi Ye, Yujia Zhou, and Yiqun Liu. 2025. Parametric retrieval augmented generation. In Proceedings of the 48th International ACM SIGIR Conference on Research and Development in Information Retrieval, pages 1240–1250. find
  24. Alon Talmor and Jonathan Berant. 2018. The web as a knowledge-base for answering complex questions. In Proceedings of the 2018 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long Papers), pages 641–651, New Orleans, L…
  25. Yuqiao Tan, Shizhu He, Huanxuan Liao, Jun Zhao, and Kang Liu. 2025. Dynamic parametric retrieval augmented generation for test-time knowledge enhancement. arXiv preprint arXiv:2503.23895. — [arXiv:2503.23895](https://arxiv.org/abs/2503.23895) find
  26. Hugo Touvron, Thibaut Lavril, Gautier Izacard, Xavier Martinet, Marie-Anne Lachaux, Timothée Lacroix, Baptiste Rozière, Naman Goyal, Eric Hambro, Faisal Azhar, and 1 others. 2023. Llama: Open and efficient foundation language models. arXiv preprint arXiv:2302.13971.

Updated

↑ back to top

This page in Markdown → https://md.research.elamur.ai/free/en/papers/zheng-2026-neurag-end-end-neural-knowledge/