Elamur Research⌕
☰
home / papers / 2026 / AccurateRAG: A Framework for Building Accurate Retrieval-Augmented Question-Answering Applications

AccurateRAG: A Framework for Building Accurate Retrieval-Augmented Question-Answering Applications

Summary

Production question-answering systems built on retrieval-augmented generation (RAG) need more than a clever retrieval method. They also require tooling for document parsing, training-data creation, model adaptation and evaluation, and most prior RAG work does not supply these. The authors propose AccurateRAG, a locally runnable framework with four components:

  • Preprocessor: turns PDF/DOCX files into structure-aware Markdown by merging Unstructured and LlamaParse outputs, then splits them into context chunks with overlap.
  • Fine-tuning Data Generator: has an LLM write simple and complex questions for each chunk and then filters out unanswerable ones.
  • Retriever: combines a contrastively fine-tuned embedding model, BM25 and a hybrid fusion option, choosing among them on validation data.
  • Answer Generator: a LoRA-tuned LLM trained on contexts enlarged with retrieved distractors, plus an LLM-based judge.

On FinanceBench, accuracy rises from 19% for a GPT-4-turbo baseline to 42%. The framework also sets new best scores on HotpotQA, PubMedQA and APIBench. This makes it a practical template for domain-specific QA over proprietary documents.

Key points

  • Models: Llama-3.1-8B-Instruct generates the synthetic (context, question, answer) data, bge-large-en-v1.5 is fine-tuned with hard and in-batch negatives, and GLM-4-9B-Chat answers questions on FinanceBench.
  • FinanceBench ablations: removing embedding fine-tuning lowers accuracy from 42% to 38.7%. Swapping in Unstructured's "fast" mode for the custom Preprocessor gives 34.7%, and its "hi-res" mode gives 26.7%.
  • Other benchmarks: with fine-tuned Llama-3-8B or CodeGemma, AccurateRAG scores 48.71 on HotpotQA, 82.4 on PubMedQA, 77.21 on HF, 93.55 on Torch Hub and 88.91 on TF Hub. It beats both RAFT and RankRAG.
  • Llama-2-7B comparison: using the same base model as RAFT, AccurateRAG wins on 4 of 5 datasets, even though RAFT was trained with GPT-4 Chain-of-Thought answers. Against RAFT without CoT, the gap is large, e.g. 45.71 vs. 25.62 on HotpotQA.
  • Limitations: the authors note that the 8B judge model may be poorly calibrated and suggest human-in-the-loop checks.

Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.

Fields: Linguistics & languages

Cite

The paper itself, not this page. BibTeX from the library's record of the work. DOI 10.63317/2ygvnkbv24j6

@inproceedings{nguyen2026accuraterag,
  title = {{AccurateRAG: A Framework for Building Accurate Retrieval-Augmented Question-Answering Applications}},
  author = {Nguyen, Linh The and Tran, Chi and Nguyen, Dung Ngoc and Pham, Van-Cuong and Ngo, Hoang and Nguyen, Dat Quoc},
  booktitle = {Proceedings of the Fifteenth Language Resources and Evaluation Conference},
  year = {2026},
  doi = {10.63317/2ygvnkbv24j6},
  url = {https://doi.org/10.63317/2ygvnkbv24j6}
}

References

28 references, 11 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

  1. Ting Chen, Simon Kornblith, Mohammad Norouzi, and Geoffrey Hinton. 2020. A simple framework for contrastive learning of visual representations. In Proceedings of the 37th International Conference on Machine Learning, pages 1597–1607. find our card
  2. Gordon V. Cormack, Charles L A Clarke, and Stefan Buettcher. 2009. Reciprocal rank fusion outperforms condorcet and individual rank learning methods. In Proceedings of the 32nd International ACM SIGIR Conference on Research and Development in Information Retrieval, page 758–759. find our card
  3. Wenqi Fan, Yujuan Ding, Liangbo Ning, Shijie Wang, Hengyun Li, Dawei Yin, Tat-Seng Chua, and Qing Li. 2024. A Survey on RAG Meeting LLMs: Towards Retrieval-Augmented Large Language Models. In Proceedings of the 30th ACM SIGKDD Conference on Knowledge Discovery and Data Mining, page 6491–6501. find our card
  4. Yunfan Gao, Yun Xiong, Xinyu Gao, Kangxiang Jia, Jinliu Pan, Yuxi Bi, Yi Dai, Jiawei Sun, Meng Wang, and Haofen Wang. 2024. Retrieval-Augmented Generation for Large Language Models: A Survey. arXiv preprint, arXiv:2312.10997. find our card
  5. Team GLM, Aohan Zeng, Bin Xu, Bowen Wang, Chenhui Zhang, Da Yin, Diego Rojas, Guanyu Feng, Hanlin Zhao, Hanyu Lai, Hao Yu, Hongning Wang, Jiadai Sun, Jiajie Zhang, Jiale Cheng, Jiayi Gui, Jie Tang, Jing Zhang, Juanzi Li, et al. 2024. Chatglm: A family of large language models from glm-130b to glm-4… find our card
  6. Aaron Grattafiori, Abhimanyu Dubey, Abhinav Jauhri, Abhinav Pandey, Abhishek Kadian, et al. 2024. The Llama 3 Herd of Models. arXiv preprint, arXiv:2407.21783. find
  7. Kelvin Guu, Kenton Lee, Zora Tung, Panupong Pasupat, and Mingwei Chang. 2020. REALM: Retrieval-Augmented Language Model Pre-Training. In Proceedings of the 37th International Conference on Machine Learning, pages 3929– 3938. find
  8. Edward J Hu, yelong shen, Phillip Wallis, Zeyuan Allen-Zhu, Yuanzhi Li, Shean Wang, Lu Wang, and Weizhu Chen. 2022. LoRA: Low-Rank Adaptation of Large Language Models. In Proceedings of the 10th International Conference on Learning Representations. find
  9. Pranab Islam, Anand Kannappan, Douwe Kiela, Rebecca Qian, Nino Scherrer, and Bertie Vidgen. 2023. FinanceBench: A New Benchmark for Financial Question Answering. arXiv preprint, arXiv:2311.11944. — [arXiv:2311.11944](https://arxiv.org/abs/2311.11944) find
  10. Qiao Jin, Bhuwan Dhingra, Zhengping Liu, William Cohen, and Xinghua Lu. 2019. PubMedQA: A Dataset for Biomedical Research Question Answering. In Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Proce… find our card
  11. Vladimir Karpukhin, Barlas Oguz, Sewon Min, Patrick Lewis, Ledell Wu, Sergey Edunov, Danqi Chen, and Wen-tau Yih. 2020. Dense Passage Retrieval for Open-Domain Question Answering. In Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing, pages 6769–6781. find our card
  12. Patrick Lewis, Ethan Perez, Aleksandra Piktus, Fabio Petroni, Vladimir Karpukhin, Naman Goyal, Heinrich Küttler, Mike Lewis, Wen-tau Yih, Tim Rocktäschel, Sebastian Riedel, and Douwe Kiela. 2020. Retrieval-augmented generation for knowledge-intensive NLP tasks. In Proceedings of the 34th Internation… find
  13. Jiarui Li, Ye Yuan, and Zehua Zhang. 2024. Enhancing LLM Factual Accuracy with RAG to Counter Hallucinations: A Case Study on Domain-Specific Queries in Private Knowledge-Bases. arXiv preprint, arXiv:2403.10446. — [arXiv:2403.10446](https://arxiv.org/abs/2403.10446) find
  14. Ilya Loshchilov and Frank Hutter. 2019. Decoupled Weight Decay Regularization. In Proceedings of the 7th International Conference on Learning Representations. find
  15. Linh The Nguyen and Dat Quoc Nguyen. 2025. Pre-training of Foundation Adapters for LLM Fine-tuning. In Proceedings of the 4th Blogpost Track at ICLR 2025. find
  16. OpenAI, Josh Achiam, Steven Adler, Sandhini Agarwal, Lama Ahmad, Ilge Akkaya, et al. 2024. GPT-4 Technical Report. find our card
  17. Shishir G. Patil, Tianjun Zhang, Xin Wang, and Joseph E. Gonzalez. 2023. Gorilla: Large Language Model Connected with Massive APIs. arXiv preprint, arXiv:2305.15334. find our card
  18. Stephen Robertson and Hugo Zaragoza. 2009. The Probabilistic Relevance Framework: BM25 and Beyond. Found. Trends Inf. Retr., 3(4):333–389. find
  19. Aditi Singh, Abul Ehtesham, Saket Kumar, and Tala Talaei Khoei. 2025. Agentic Retrieval-Augmented Generation: A Survey on Agentic RAG. find
  20. CodeGemma Team, Heri Zhao, Jeffrey Hui, Joshua Howland, Nam Nguyen, Siqi Zuo, Andrea Hu, Christopher A. Choquette-Choo, Jingyue Shen, Joe Kelley, Kshitij Bansal, Luke Vilnis, Mateo Wirth, Paul Michel, Peter Choy, Pratik Joshi, Ravin Kumar, Sarmad Hashmi, Shubham Agrawal, Zhitao Gong, Jane Fine, Tris… find
  21. Hugo Touvron, Louis Martin, Kevin Stone, et al. 2023. Llama 2: Open Foundation and Fine-Tuned Chat Models. arXiv preprint, arXiv:2307.09288. find
  22. Ashok Urlana, Charaka Vinayak Kumar, Ajeet Kumar Singh, Bala Mallikarjunarao Garlapati, Srinivasa Rao Chalamala, and Rahul Mishra. 2024. LLMs with Industrial Lens: Deciphering the Challenges and Prospects – A Survey. find
  23. Shitao Xiao, Zheng Liu, Peitian Zhang, and Niklas Muennighoff. 2023. C-Pack: Packaged Resources To Advance General Chinese Embedding. arXiv preprint. find
  24. Zhilin Yang, Peng Qi, Saizheng Zhang, Yoshua Bengio, William Cohen, Ruslan Salakhutdinov, and Christopher D. Manning. 2018. HotpotQA: A Dataset for Diverse, Explainable Multi-hop Question Answering. In Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing, pages 2369… find our card
  25. Yue Yu, Wei Ping, Zihan Liu, Boxin Wang, Jiaxuan You, Chao Zhang, Mohammad Shoeybi, and Bryan Catanzaro. 2024. RankRAG: Unifying Context Ranking with Retrieval-Augmented Generation in LLMs. In Proceedings of the 38th Annual Conference on Neural Information Processing Systems. find
  26. Shenglai Zeng, Jiankun Zhang, Pengfei He, Yiding Liu, Yue Xing, Han Xu, Jie Ren, Yi Chang, Shuaiqiang Wang, Dawei Yin, and Jiliang Tang. 2024. The Good and The Bad: Exploring Privacy Issues in Retrieval-Augmented Generation (RAG). In Findings of the Association for Computational Linguistics: ACL 202… find
  27. Tianjun Zhang, Shishir G Patil, Naman Jain, Sheng Shen, Matei Zaharia, Ion Stoica, and Joseph E. Gonzalez. 2024. RAFT: Adapting Language Model to Domain Specific RAG. In Proceedings of the First Conference on Language Modeling. find our card
  28. Lianmin Zheng, Wei-Lin Chiang, Ying Sheng, Siyuan Zhuang, Zhanghao Wu, Yonghao Zhuang, Zi Lin, Zhuohan Li, Dacheng Li, Eric Xing, Hao Zhang, Joseph E. Gonzalez, and Ion Stoica. 2023. Judging LLM-as-a-Judge with MT-Bench and Chatbot Arena. In Proceedings of the 37th Conference on Neural Information P… find

↑ back to top

This page in Markdown → https://md.research.elamur.ai/free/en/papers/nguyen-2026-accuraterag-framework-building-accurate-retrieval/