Elamur Research⌕
☰
home / papers / 2024 / RQ-RAG: Learning to Refine Queries for Retrieval Augmented Generation

RQ-RAG: Learning to Refine Queries for Retrieval Augmented Generation

Paper · 2024 · Chi-Min Chan, Chunpu Xu, Ruibin Yuan et al.

Summary

Retrieval-augmented LLMs usually send the user's raw input straight to the search step. This works poorly in three cases. Simple inputs like greetings need no retrieval at all. Multi-step questions need splitting into parts. Vague questions have several valid readings. RQ-RAG addresses this by teaching a 7B Llama2 model to improve its own search queries before answering. The authors assembled about 40k training examples from multi-turn dialogue, decomposition and disambiguation datasets. ChatGPT wrote refined queries, DuckDuckGo supplied documents for them, and ChatGPT then rewrote the target answers so they match the retrieved evidence. Special control tokens let the model decide at each step whether to rewrite, decompose, disambiguate, or stop and respond. Because this produces several possible reasoning paths, the final answer is chosen by perplexity, confidence, or an ensemble of the two, with no external LLM judge. The model beats Self-RAG by 1.9% on average on single-hop QA and also performs strongly on multi-hop QA. This shows that refining queries, plus answers grounded in retrieved context, makes RAG more reliable.

Key points

  • Base model: Llama2 7B, fine-tuned end-to-end on about 40k ChatGPT-annotated instances (gpt-3.5-turbo-0125) built with DuckDuckGo search. Code is released at github.com/chanchimin/RQ-RAG.
  • Benchmarks: ARC_C, PopQA and OBQA (single-hop), and HotpotQA, 2Wiki and MuSiQue (multi-hop). RQ-RAG exceeds the previous SOTA, Self-RAG, by 1.9% on average on the single-hop sets.
  • Trajectory selection uses PPL, confidence or an ensemble. An oracle "upper bound" checks whether any generated path contains the right answer, and it is notably high, so better selection could bring large further gains.
  • Ablation on HotpotQA: the model does best when 0% of the original dataset answers are kept. Performance falls as the retained share rises toward 100%.
  • Robustness to the retrieval source (DuckDuckGo, Wikipedia, Bing) on single-hop tasks: RQ-RAG averages 67.6 with variance 0.7, while Self-RAG averages 65.5 with variance 1.8. Self-RAG loses 3–5% with Bing.

Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.

 Institutions

🎓Massachusetts Institute of TechnologyCambridge · United States243 works · 76,506 citations🎓Hong Kong University of Science and TechnologyHong Kong · China170 works · 26,341 citations🎓Hong Kong Polytechnic UniversityHong Kong · China106 works · 5,190 citations

 Fields, topics and concepts

📚 Shelves 6The field of science this work belongs to (OpenAlex fields).Computer Science · 18,614 worksArtificial Intelligence · 17,965 worksArts and Humanities · 17,450 worksLanguage and Linguistics · 17,296 worksComputer Vision and Pattern Recognition · 3,486 worksInformation Systems · 1,730 works

🧭 Topics 1Research subjects and languages the work is about.Information retrieval & search · 552 works

Cite

Cite as Chi-Min Chan, Chunpu Xu, Ruibin Yuan, Hongyin Luo, Wei Xue, Yike Guo and Jie Fu. 2024. RQ-RAG: Learning to Refine Queries for Retrieval Augmented Generation. arXiv:2404.00610.

The paper itself, not this page. BibTeX from the library's record of the work. DOI 10.48550/arxiv.2404.00610 · arXiv 2404.00610

@misc{chan2024rq,
  title = {{RQ-RAG: Learning to Refine Queries for Retrieval Augmented Generation}},
  author = {Chi-Min Chan, Chunpu Xu, Ruibin Yuan, Hongyin Luo, Wei Xue, Yike Guo and Jie Fu},
  year = {2024},
  doi = {10.48550/arxiv.2404.00610},
  eprint = {2404.00610},
  archivePrefix = {arXiv},
  url = {https://doi.org/10.48550/arxiv.2404.00610}
}

References

40 references, 29 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

  1. Asai, A., Wu, Z., Wang, Y., Sil, A., and Hajishirzi, H. (2024). Self-rag: Learning to retrieve, generate, and critique through self-reflection. In The Twelfth International Conference on Learning Representations.
  2. Brown, T., Mann, B., Ryder, N., Subbiah, M., Kaplan, J. D., Dhariwal, P., Neelakantan, A., Shyam, P., Sastry, G., Askell, A., et al. (2020). Language models are few-shot learners. Advances in neural information processing systems, 33:1877–1901.
  3. Clark, P., Cowhey, I., Etzioni, O., Khot, T., Sabharwal, A., Schoenick, C., and Tafjord, O. (2018). Think you have solved question answering? try arc, the ai2 reasoning challenge. arXiv preprint arXiv:1803.05457.
  4. Ho, X., Nguyen, A.-K. D., Sugawara, S., and Aizawa, A. (2020). Constructing a multi-hop qa dataset for comprehensive evaluation of reasoning steps. arXiv preprint arXiv:2011.01060.
  5. Ji, Z., Lee, N., Frieske, R., Yu, T., Su, D., Xu, Y., Ishii, E., Bang, Y. J., Madotto, A., and Fung, P. (2023). Survey of hallucination in natural language generation. ACM Computing Surveys, 55(12):1–38.
  6. Karpukhin, V., Oğuz, B., Min, S., Lewis, P., Wu, L., Edunov, S., Chen, D., and Yih, W.-t. (2020). Dense passage retrieval for open-domain question answering. arXiv preprint arXiv:2004.04906.
  7. Keskar, N. S., McCann, B., Varshney, L. R., Xiong, C., and Socher, R. (2019). Ctrl: A conditional transformer language model for controllable generation. arXiv preprint arXiv:1909.05858.
  8. Köpf, A., Kilcher, Y., von Rütte, D., Anagnostidis, S., Tam, Z. R., Stevens, K., Barhoum, A., Nguyen, D., Stanley, O., Nagyfi, R., et al. (2024). Openassistant conversations-democratizing large language model alignment. Advances in Neural Information Processing Systems, 36.
  9. Lewis, P., Perez, E., Piktus, A., Petroni, F., Karpukhin, V., Goyal, N., Küttler, H., Lewis, M., Yih, W.-t., Rocktäschel, T., et al. (2020). Retrieval-augmented generation for knowledge-intensive nlp tasks. Advances in Neural Information Processing Systems, 33:9459–9474.
  10. Lu, X., Welleck, S., Hessel, J., Jiang, L., Qin, L., West, P., Ammanabrolu, P., and Choi, Y. (2022). Quark: Controllable text generation with reinforced unlearning. Advances in neural information processing systems, 35:27591–27609.
  11. Luo, H., Zhang, T., Chuang, Y.-S., Gong, Y., Kim, Y., Wu, X., Meng, H. M., and Glass, J. R. (2023). Search augmented instruction learning. In The 2023 Conference on Empirical Methods in Natural Language Processing. find
  12. Ma, X., Gong, Y., He, P., Zhao, H., and Duan, N. (2023). Query rewriting for retrieval-augmented large language models. arXiv preprint arXiv:2305.14283. — [arXiv:2305.14283](https://arxiv.org/abs/2305.14283) find
  13. Mallen, A., Asai, A., Zhong, V., Das, R., Khashabi, D., and Hajishirzi, H. (2022). When not to trust language models: Investigating effectiveness of parametric and non-parametric memories. arXiv preprint arXiv:2212.10511. find
  14. Mihaylov, T., Clark, P., Khot, T., and Sabharwal, A. (2018). Can a suit of armor conduct electricity? a new dataset for open book question answering. arXiv preprint arXiv:1809.02789. find
  15. Mukherjee, S., Mitra, A., Jawahar, G., Agarwal, S., Palangi, H., and Awadallah, A. (2023). Orca: Progressive learning from complex explanation traces of gpt-4. arXiv preprint arXiv:2306.02707.
  16. OpenAI (2023). Gpt-4 technical report. arXiv preprint arXiv:2303.08774.
  17. Ouyang, L., Wu, J., Jiang, X., Almeida, D., Wainwright, C., Mishkin, P., Zhang, C., Agarwal, S., Slama, K., Ray, A., et al. (2022). Training language models to follow instructions with human feedback. Advances in neural information processing systems, 35:27730–27744.
  18. Raffel, C., Shazeer, N., Roberts, A., Lee, K., Narang, S., Matena, M., Zhou, Y., Li, W., and Liu, P. J. (2020). Exploring the limits of transfer learning with a unified text-to-text transformer. Journal of machine learning research, 21(140):1–67.
  19. Robertson, S., Zaragoza, H., et al. (2009). The probabilistic relevance framework: Bm25 and beyond. Foundations and Trends® in Information Retrieval, 3(4):333–389. find
  20. Shi, F., Chen, X., Misra, K., Scales, N., Dohan, D., Chi, E. H., Schärli, N., and Zhou, D. (2023a). Large language models can be easily distracted by irrelevant context. In International Conference on Machine Learning, pages 31210–31227. PMLR.
  21. Shi, W., Min, S., Yasunaga, M., Seo, M., James, R., Lewis, M., Zettlemoyer, L., and Yih, W.-t. (2023b). Replug: Retrieval-augmented black-box language models. arXiv preprint arXiv:2301.12652.
  22. Shuster, K., Poff, S., Chen, M., Kiela, D., and Weston, J. (2021). Retrieval augmentation reduces hallucination in conversation. arXiv preprint arXiv:2104.07567.
  23. Stelmakh, I., Luan, Y., Dhingra, B., and Chang, M.-W. (2022). Asqa: Factoid questions meet long-form answers. arXiv preprint arXiv:2204.06092.
  24. Sun, Z., Wang, X., Tay, Y., Yang, Y., and Zhou, D. (2022). Recitation-augmented language models. arXiv preprint arXiv:2210.01296. find
  25. Taori, R., Gulrajani, I., Zhang, T., Dubois, Y., Li, X., Guestrin, C., Liang, P., and Hashimoto, T. B. (2023). Stanford alpaca: An instruction-following llama model. find
  26. Touvron, H., Martin, L., Stone, K., Albert, P., Almahairi, A., Babaei, Y., Bashlykov, N., Batra, S., Bhargava, P., Bhosale, S., et al. (2023). Llama 2: Open foundation and fine-tuned chat models. arXiv preprint arXiv:2307.09288.
  27. Trivedi, H., Balasubramanian, N., Khot, T., and Sabharwal, A. (2022). Musique: Multihop questions via single-hop question composition. Transactions of the Association for Computational Linguistics, 10:539–554.
  28. Trivedi, H., Balasubramanian, N., Khot, T., and Sabharwal, A. (2023). Interleaving retrieval with chain-of-thought reasoning for knowledge-intensive multi-step questions. The 61st Annual Meeting of the Association for Computational Linguistics.
  29. Vu, T., Iyyer, M., Wang, X., Constant, N., Wei, J., Wei, J., Tar, C., Sung, Y.-H., Zhou, D., Le, Q., et al. (2023). Freshllms: Refreshing large language models with search engine augmentation. arXiv preprint arXiv:2310.03214.
  30. Wang, X., Wei, J., Schuurmans, D., Le, Q., Chi, E., Narang, S., Chowdhery, A., and Zhou, D. (2022). Self-consistency improves chain of thought reasoning in language models. arXiv preprint arXiv:2203.11171.
  31. Wei, J., Wang, X., Schuurmans, D., Bosma, M., Xia, F., Chi, E., Le, Q. V., Zhou, D., et al. (2022). Chain-of-thought prompting elicits reasoning in large language models. Advances in neural information processing systems, 35:24824–24837.
  32. Xu, C., Chern, S., Chern, E., Zhang, G., Wang, Z., Liu, R., Li, J., Fu, J., and Liu, P. (2023a). Align on the fly: Adapting chatbot behavior to established norms. arXiv preprint arXiv:2312.15907. — [arXiv:2312.15907](https://arxiv.org/abs/2312.15907) find
  33. Xu, C., Sun, Q., Zheng, K., Geng, X., Zhao, P., Feng, J., Tao, C., and Jiang, D. (2023b). Wizardlm: Empowering large language models to follow complex instructions. arXiv preprint arXiv:2304.12244.
  34. Xu, F., Shi, W., and Choi, E. (2023c). Recomp: Improving retrieval-augmented lms with compression and selective augmentation. arXiv preprint arXiv:2310.04408.
  35. Yang, Z., Qi, P., Zhang, S., Bengio, Y., Cohen, W. W., Salakhutdinov, R., and Manning, C. D. (2018). Hotpotqa: A dataset for diverse, explainable multi-hop question answering. arXiv preprint arXiv:1809.09600.
  36. Yao, S., Yu, D., Zhao, J., Shafran, I., Griffiths, T., Cao, Y., and Narasimhan, K. (2024). Tree of thoughts: Deliberate problem solving with large language models. Advances in Neural Information Processing Systems, 36. find
  37. Yoran, O., Wolfson, T., Ram, O., and Berant, J. (2023). Making retrieval-augmented language models robust to irrelevant context. arXiv preprint arXiv:2310.01558.
  38. Yu, W., Zhang, H., Pan, X., Ma, K., Wang, H., and Yu, D. (2023). Chain-of-note: Enhancing robustness in retrieval-augmented language models. arXiv preprint arXiv:2311.09210.
  39. Zhou, C., Liu, P., Xu, P., Iyer, S., Sun, J., Mao, Y., Ma, X., Efrat, A., Yu, P., Yu, L., et al. (2024). Lima: Less is more for alignment. Advances in Neural Information Processing Systems, 36. find
  40. Zhou, S., Alon, U., Xu, F. F., Wang, Z., Jiang, Z., and Neubig, G. (2022). Docprompting: Generating code by retrieving the docs. arXiv preprint arXiv:2207.05987. find

Updated

↑ back to top

This page in Markdown → https://md.research.elamur.ai/free/en/papers/chan-2024-rq-rag-learning-refine-queries/