Elamur Research⌕
☰
home / papers / 2026 / RAGTUM at SemEval-2026 Task 8: Contextual Query Rewriting and Dense Retrieval for Multi-Turn RAG

RAGTUM at SemEval-2026 Task 8: Contextual Query Rewriting and Dense Retrieval for Multi-Turn RAG

Paper · 2026 · Finn Wigger, Maximilian Podolsky, Merle Wilmink et al.

Summary

RAGTUM is a Technical University of Munich student team's entry to SemEval-2026 Task 8 (MTRAGEval). The task tests retrieval-augmented generation over multi-turn dialogues, where later questions rely on earlier turns. The benchmark has 110 human-written conversations over four corpora: ClapNQ, FiQA, Govt and Cloud.

The authors trained no models. Instead, they built a rewrite–retrieve–filter–generate pipeline:

  • Rewriting: after cleaning the noisy corpora, Gemini 2.5 Flash, prompted with two examples, converts each turn and its history into a self-contained query.
  • Retrieval: queries and passages are embedded with OpenAI text-embedding-3-large and stored in Milvus. The top five passages are fetched by inner product.
  • Filtering: Gemini rates each passage's relevance on a five-point scale and discards weak ones.
  • Generation: Gemini answers from the remaining passages, or replies "I don't know" when evidence is lacking.

The system competed in all three subtasks. It placed roughly in the top third each time and beat the official baselines. The authors conclude that query rewriting and evidence filtering matter as much as raw model capacity, although reliance on proprietary APIs limits reproducibility.

Key points

  • Subtask A (retrieval): nDCG@5 of 0.4883, ranked 11/38. This is above the ELSER-based baseline of 0.4795, though the results section names a GPT-OSS-20b baseline instead.
  • Subtask B (generation): harmonic mean of 0.7453, ranked 11/26, against a GPT-OSS-120b baseline of 0.639. Subtask C (full RAG): 0.5421, ranked 10/29, slightly above the Qwen-30b-a3b-thinking baseline (0.5366).
  • Few-shot query rewriting raised retrieval Recall@5 on test data from 0.53 to 0.58, an improvement of about 10%.
  • Dense retrieval consistently outperformed sparse and hybrid alternatives. The relevance filter keeps passages scoring ≥ 2, a threshold chosen to protect faithfulness.
  • Corpus sizes: ClapNQ 183,408 passages, FiQA 61,022, Govt 49,607, Cloud 72,442. No GPUs were used; all inference ran through cloud APIs.

Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.

 Institutions

🎓Technical University of MunichMunich · Germany55 works · 48,474 citations

 Fields, topics and concepts

📚 Shelves 5The field of science this work belongs to (OpenAlex fields).Computer Science · 16,667 worksArtificial Intelligence · 16,194 worksArts and Humanities · 16,039 worksLanguage and Linguistics · 15,891 worksInformation Systems · 1,437 works

🧭 Topics 1Research subjects and languages the work is about.Information retrieval & search · 473 works

Cite

Cite as Finn Wigger, Maximilian Podolsky, Merle Wilmink and Zelong Peng. 2026. RAGTUM at SemEval-2026 Task 8: Contextual Query Rewriting and Dense Retrieval for Multi-Turn RAG. Proceedings of the 20th International Workshop on Semantic Evaluation (2026).

The paper itself, not this page. BibTeX from the library's record of the work. DOI 10.18653/v1/2026.semeval-1.227

@inproceedings{wigger2026ragtum,
  title = {{RAGTUM at SemEval-2026 Task 8: Contextual Query Rewriting and Dense Retrieval for Multi-Turn RAG}},
  author = {Finn Wigger, Maximilian Podolsky, Merle Wilmink and Zelong Peng},
  booktitle = {Proceedings of the 20th International Workshop on Semantic Evaluation (2026)},
  year = {2026},
  doi = {10.18653/v1/2026.semeval-1.227},
  url = {https://doi.org/10.18653/v1/2026.semeval-1.227}
}

References

14 references, 8 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

  1. Ahmed Elgohary, Denis Peskov, and Jordan Boyd-Graber. 2019. Can you unpack that? learning to rewrite questions-in-context. In Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing (EMNLP-IJCNLP…
  2. Thibault Formal, Benjamin Piwowarski, and Stéphane Clinchant. 2021. Splade: Sparse lexical and expansion model for first stage ranking.
  3. Google. 2024. Gemini 2.5 flash. Google Vertex AI Model Documentation. find
  4. Vladimir Karpukhin, Barlas Oguz, Sewon Min, Patrick Lewis, Ledell Wu, Sergey Edunov, Danqi Chen, and Wen-tau Yih. 2020. Dense passage retrieval for open-domain question answering. In Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing (EMNLP), pages 6769–6781, Onli…
  5. Sara Rosenthal, Vraj Shah, Yannis Katsis, and Marina Danilevsky. 2026b. Semeval-2026 task 8: Mtrageval: Evaluating multi-turn rag conversations. In Proceedings of the 20th International Workshop on Semantic Evaluation (SemEval-2026), San Diego, California. Association for Computational Linguistics.
  6. Kurt Shuster, Spencer Poff, Moya Chen, Douwe Kiela, and Jason Weston. 2021. Retrieval augmentation reduces hallucination in conversation.
  7. Cem Üyük, Danica Rovó, Shaghayeghkolli, Rabia Varol, Georg Groh, and Daryna Dementieva. 2024. Crafting tomorrow’s headlines: Neural news generation and detection in English, Turkish, Hungarian, and Persian. In Proceedings of the Third Workshop on NLP for Positive Impact, pages 271–307, Miami, Florid… find
  8. Yannis Katsis, Sara Rosenthal, Kshitij Fadnis, Chulaka Gunasekara, Young-Suk Lee, Lucian Popa, Vraj Shah, Huaiyu Zhu, Danish Contractor, and Marina Danilevsky. 2025. Mtrag: A multi-turn conversational benchmark for evaluating retrieval-augmented generation systems. Preprint, arXiv:2501.03468. — [arXiv:2501.03468](https://arxiv.org/abs/2501.03468) find
  9. Omar Khattab and Matei Zaharia. 2020. Colbert: Efficient and effective passage search via contextualized late interaction over bert.
  10. Shaghayegh Kolli, Richard Rosenbaum, Timo Cavelius, Lasse Strothe, Andrii Lata, and Jana Diesner. 2025. Hybrid fact-checking that integrates knowledge graphs, large language models, and search-based retrieval agents improves interpretable claim verification. In Proceedings of the 9th Widening NLP Wo… find
  11. Patrick Lewis, Ethan Perez, Aleksandra Piktus, Fabio Petroni, Vladimir Karpukhin, Naman Goyal, Heinrich Küttler, Mike Lewis, Wen-tau Yih, Tim Rocktäschel, Sebastian Riedel, and Douwe Kiela. 2020. Retrieval-augmented generation for knowledge-intensive nlp tasks.
  12. Xiao Ma, Mingchen Chen, Wenhao Wang, Yuxuan Wang, Beidi Chen, and Tianqi Chen. 2023. Query rewriting for retrieval-augmented large language models. find
  13. OpenAI. 2024. text-embedding-3-large. OpenAI Model Documentation. find
  14. Sara Rosenthal, Yannis Katsis, Vraj Shah, Lihong He, Lucian Popa, and Marina Danilevsky. 2026a. Mtrag-un: A benchmark for open challenges in multi-turn rag conversations. Preprint, arXiv:2602.23184.

Updated

↑ back to top

This page in Markdown → https://md.research.elamur.ai/free/en/papers/wigger-2026-ragtum-semeval-2026-task-8/