Elamur Research⌕
☰
home / papers / 2026 / Sifei at SemEval-2026 Task 8: Hybrid Retrieval and Query Rewriting for Multi-Turn RAG

Sifei at SemEval-2026 Task 8: Hybrid Retrieval and Query Rewriting for Multi-Turn RAG

Paper · 2026 · Sifei Meng and Dmitry Ilvovsky

Summary

Multi-turn RAG is difficult because user intent shifts across turns, dialogue history adds noise, and context budgets are tight. The authors built a system for SemEval-2026 Task 8 (MTRAGEval), entering the retrieval subtask (Task A) and the retrieve-and-generate subtask (Task C) on the MTRAG corpora CLAPNQ, FiQA, Govt and Cloud. No model is fine-tuned. The pipeline first cleans the corpus while keeping qrels-referenced passages. It then forms the query by joining the final user turn with a GPT-5.2 standalone rewrite. Candidates come from Jasper-Token-Compression-600M (dense) and SPLADE-v3 (sparse), are merged with reciprocal rank fusion, reranked by BAAI/bge-reranker-v2-m3, and finally blended with retrieval scores (α=0.15). For Task C, the top-5 passages are passed to DeepSeek-V3.2 with the official prompt. Task A reached 0.5453 nDCG@5, placing third of 38 teams and beating the strongest baseline (0.4795). Task C scored 0.5312, placing 15th of 29. The work shows that a training-free, largely open-source retrieval stack can compete. Its error analysis identifies first-stage recall as the main remaining weakness.

Key points

  • Data: 110 conversations (7.7 turns on average, 842 turn-level tasks), of which 777 answerable tasks were used for development. The official 512-token passages with 100-token overlap were kept, since re-chunking alternatives performed worse.
  • Concatenating the last turn with the rewrite beat either input alone in both tasks. In Task C it raised the harmonic mean from 0.5813 to 0.5922 on the training split. Keyword-style and HyDE-like expansions hurt results.
  • Hybrid configuration: top-200 candidates from each retriever, RRF with k=10, and reranking of the top-100 fused candidates. Very large pools (500–1000) added noise and lowered nDCG@5.
  • The custom rewrites had lower perplexity than the official rewrites in all four domains (e.g., FiQA 78.42→60.64).
  • Test error analysis: 16.8% of queries scored nDCG@5 = 0. Among hard cases, 85.7% were recall failures, concentrated in the Cloud and FiQA domains and in turns after the fifth.

Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.

 Institutions

🎓National Research University Higher School of EconomicsMoscow · Russia24 works · 39,585 citations

 Fields, topics and concepts

📚 Shelves 5The field of science this work belongs to (OpenAlex fields).Computer Science · 18,614 worksArtificial Intelligence · 17,965 worksArts and Humanities · 17,450 worksLanguage and Linguistics · 17,296 worksInformation Systems · 1,730 works

🧭 Topics 1Research subjects and languages the work is about.Information retrieval & search · 552 works

Cite

Cite as Sifei Meng and Dmitry Ilvovsky. 2026. Sifei at SemEval-2026 Task 8: Hybrid Retrieval and Query Rewriting for Multi-Turn RAG. Proceedings of the 20th International Workshop on Semantic Evaluation (2026).

The paper itself, not this page. BibTeX from the library's record of the work. DOI 10.18653/v1/2026.semeval-1.32

@inproceedings{meng2026sifei,
  title = {{Sifei at SemEval-2026 Task 8: Hybrid Retrieval and Query Rewriting for Multi-Turn RAG}},
  author = {Sifei Meng and Dmitry Ilvovsky},
  booktitle = {Proceedings of the 20th International Workshop on Semantic Evaluation (2026)},
  year = {2026},
  doi = {10.18653/v1/2026.semeval-1.32},
  url = {https://doi.org/10.18653/v1/2026.semeval-1.32}
}

References

16 references, 5 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

  1. Ashish Bansal. 2023. Optimizing rag with hybrid search and contextual chunking. Journal of Engineering and Applied Sciences Technology. find
  2. Jianlv Chen, Shitao Xiao, Peitian Zhang, Kun Luo, Defu Lian, and Zheng Liu. 2025a. M3-embedding: Multilinguality, multi-functionality, multi-granularity text embeddings through self-knowledge distillation. Preprint, arXiv:2402.03216.
  3. Zhiyu Chen, Biancen Xie, Sidarth Srinivasan, Qun Liu, Manikandarajan Ramanathan, and Raj Maragoud. 2025b. Building multi-turn rag for customer support with llm labeling. find
  4. Kenneth Enevoldsen, Isaac Chung, Imene Kerboua, Márton Kardos, Ashwin Mathur, David Stap, Jay Gala, Wissam Siblini, Dominik Krzemiński, Genta Indra Winata, Saba Sturua, Saiteja Utpala, Mathieu Ciancone, Marion Schaeffer, Gabriel Sequeira, Diganta Misra, Shreeya Dhakal, Jonathan Rystrøm, Roman Solom… — MMTEB: Massive Multilingual Text Embedding Benchmark
  5. Rishi Kalra, Zekun Wu, Ayesha Gulley, Airlie Hilliard, Xin Guan, Adriano Koshiyama, and Philip Treleaven. 2025. Hypa-rag: A hybrid parameter adaptive retrieval-augmented generation system for ai legal and policy applications. Preprint, arXiv:2409.09046. find
  6. Yannis Katsis, Sara Rosenthal, Kshitij Fadnis, Chulaka Gunasekara, Young-Suk Lee, Lucian Popa, Vraj Shah, Huaiyu Zhu, Danish Contractor, and Marina Danilevsky. 2025. mtrag: A multi-turn conversational benchmark for evaluating retrieval-augmented generation systems. Transactions of the Association fo… find
  7. Carlos Lassance, Hervé Déjean, Thibault Formal, and Stéphane Clinchant. 2024. Splade-v3: New baselines for splade. Preprint, arXiv:2403.06789. — [arXiv:2403.06789](https://arxiv.org/abs/2403.06789) find
  8. Joohyun Lee and Minji Roh. 2024. Multi-reranker: Maximizing performance of retrieval-augmented generation in the financerag challenge. Preprint, arXiv:2411.16732. — [arXiv:2411.16732](https://arxiv.org/abs/2411.16732) find
  9. Carlo Merola and Jaspinder Singh. 2025. Reconstructing context: Evaluating advanced chunking strategies for retrieval-augmented generation. Preprint, arXiv:2504.19754. — [arXiv:2504.19754](https://arxiv.org/abs/2504.19754) find
  10. Priyaranjan Pattnayak, Amit Agarwal, Hansa Meghwani, Hitesh Laxmichand Patel, and Srikant Panda. 2025. Hybrid ai for responsive multi-turn online conversations with novel dynamic routing and feedback adaptation. Preprint, arXiv:2506.02097. find
  11. Sara Rosenthal, Yannis Katsis, Vraj Shah, Lihong He, Lucian Popa, and Marina Danilevsky. 2026a. Mtrag-un: A benchmark for open challenges in multi-turn rag conversations. Preprint, arXiv:2602.23184.
  12. Sara Rosenthal, Vraj Shah, Yannis Katsis, and Marina Danilevsky. 2026b. Semeval-2026 task 8: Mtrageval: Evaluating multi-turn rag conversations. In Proceedings of the 20th International Workshop on Semantic Evaluation (SemEval-2026), San Diego, California. Association for Computational Linguistics.…
  13. Xiaohua Wang, Zhenghua Wang, Xuan Gao, Feiran Zhang, Yixin Wu, Zhibo Xu, Tianyuan Shi, Zhengyuan Wang, Shizheng Li, Qi Qian, Ruicheng Yin, Changze Lv, Xiaoqing Zheng, and Xuanjing Huang. 2024. Searching for best practices in retrieval-augmented generation. Preprint, arXiv:2407.01219.
  14. Conversation history: \> who takes photos of planes in the air \> No , I meant photos in the air . Rewrite: \> Who takes photos of airplanes while they are in the air ? find
  15. Dun Zhang, Ziyang Zeng, Yudong Zhou, and Shuyang Lu. 2025. Jasper-token-compression-600m technical report. Preprint, arXiv:2511.14405. — [arXiv:2511.14405](https://arxiv.org/abs/2511.14405) find
  16. Jihao Zhao, Zhiyuan Ji, Yuchen Feng, Pengnian Qi, Simin Niu, Bo Tang, Feiyu Xiong, and Zhiyu Li. 2025. Meta-chunking: Learning text segmentation and semantic completion via logical perception. Preprint, arXiv:2410.12788. — [arXiv:2410.12788](https://arxiv.org/abs/2410.12788) find

Updated

↑ back to top

This page in Markdown → https://md.research.elamur.ai/free/en/papers/meng-2026-sifei-semeval-2026-task-8/