Elamur Research⌕
☰
home / papers / 2026 / IIMAS-RAG at SemEval-2026 Task 8: Hybrid Sparse-Dense Retrieval and Answerability-Conditioned Generation for Multi-Turn RAG

IIMAS-RAG at SemEval-2026 Task 8: Hybrid Sparse-Dense Retrieval and Answerability-Conditioned Generation for Multi-Turn RAG

Paper · 2026 · Vania Raya-Rios, Helena Gomez-Adorno, Leon Hecht et al.

Summary

This paper describes the IIMAS-RAG entry to SemEval-2026 Task 8 (MTRAGEval), a shared task on retrieval-augmented generation in multi-turn English conversations. The data are 110 conversations grounded in four corpora: ClapNQ, Cloud, FiQA and Govt. The team built a three-stage modular pipeline. First, command-r-08-2024 rewrites each dialogue turn into a self-contained query. Second, SPLADE and Voyage-3-large retrieve passages from per-domain indices, and their rankings are merged with Reciprocal Rank Fusion. Third, a Command-A-03-2025 classifier labels each turn as answerable, unanswerable, partial or conversational, and that label steers GPT-4.1 when it writes the response.

Retrieval was the strongest part, placing 4th of 38 teams. Ablations show that standalone query rewriting accounts for most of the retrieval gains. End-to-end scores were lower, mainly because the answerability step tended to abstain too often on partially answerable turns. Human judges, however, rated the answers highly. The work shows that strong retrieval does not by itself guarantee strong conversational RAG.

Key points

  • Official rankings:
    • Subtask A: 4/38, nDCG@5 = 0.5445 (the winner scored 0.5776).
    • Subtask B: 22/26, composite 0.5638.
    • Subtask C: 13/29, composite 0.5397.
  • Effect of query rewriting: rewriting improved nDCG@10 by 16.3% relative to hybrid retrieval without it. The rewrite-based hybrid reached 0.528 nDCG@10, while HyDE scored 0.383 and multi-query expansion scored 0.401.
  • Prompt variants: the three rewriting prompts (V1–V3) did not differ significantly (p = 0.642). Domain-tailored prompts gave only small gains on Cloud and FiQA.
  • Reranking: adding bge-reranker-v2-m3 lowered nDCG@5 by 0.029 and was therefore dropped. RRF used k = 60 with the top 300 candidates from each retriever.
  • Human evaluation in Subtask C: IIMAS-RAG placed second overall (3.33). It had the best FANC score (3.80), above the Reference Gold baseline (3.65).

Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.

 Institutions

🎓Universidad Nacional Autónoma de MéxicoMexico City · Mexico8 works · 288 citations

Also on the byline: Facultad de Estudios Superiores Aragón.

 Fields, topics and concepts

📚 Shelves 6The field of science this work belongs to (OpenAlex fields).Computer Science · 16,667 worksArtificial Intelligence · 16,194 worksArts and Humanities · 16,039 worksLanguage and Linguistics · 15,891 worksComputer Vision and Pattern Recognition · 2,869 worksInformation Systems · 1,437 works

🧭 Topics 1Research subjects and languages the work is about.Information retrieval & search · 473 works

Cite

Cite as Vania Raya-Rios, Helena Gomez-Adorno, Leon Hecht, Pedro Vázquez-Osorio, Erick Fabián-Sandoval, Jesús Vázquez-Osorio and Diego Hernández-Bustamante. 2026. IIMAS-RAG at SemEval-2026 Task 8: Hybrid Sparse-Dense Retrieval and Answerability-Conditioned Generation for Multi-Turn RAG. Proceedings of the 20th International Workshop on Semantic Evaluation (2026).

The paper itself, not this page. BibTeX from the library's record of the work. DOI 10.18653/v1/2026.semeval-1.345

@inproceedings{rayarios2026iimas,
  title = {{IIMAS-RAG at SemEval-2026 Task 8: Hybrid Sparse-Dense Retrieval and Answerability-Conditioned Generation for Multi-Turn RAG}},
  author = {Vania Raya-Rios, Helena Gomez-Adorno, Leon Hecht, Pedro Vázquez-Osorio, Erick Fabián-Sandoval, Jesús Vázquez-Osorio and Diego Hernández-Bustamante},
  booktitle = {Proceedings of the 20th International Workshop on Semantic Evaluation (2026)},
  year = {2026},
  doi = {10.18653/v1/2026.semeval-1.345},
  url = {https://doi.org/10.18653/v1/2026.semeval-1.345}
}

References

19 references, 10 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

  1. Jianlyu Chen, Shitao Xiao, Peitian Zhang, Kun Luo, Defu Lian, and Zheng Liu. 2024. M3-embedding: Multi-linguality, multi-functionality, multi-granularity text embeddings through self-knowledge distillation. In Findings of the Association for Computational Linguistics: ACL 2024, pages 2318–2335, Bang…
  2. Cohere. 2025. Command a: An enterprise-ready large language model. arXiv preprint arXiv:2504.00698. — [arXiv:2504.00698](https://arxiv.org/abs/2504.00698) find
  3. Gordon V. Cormack, Charles L A Clarke, and Stefan Buettcher. 2009a. Reciprocal rank fusion outperforms condorcet and individual rank learning methods. In Proceedings of the 32nd International ACM SIGIR Conference on Research and Development in Information Retrieval, SIGIR ’09, page 758–759, New York…
  4. Gordon V. Cormack, Charles L. A. Clarke, and Stefan Büttcher. 2009b. Reciprocal rank fusion outperforms condorcet and individual rank learning methods. Proceedings of the 32nd international ACM SIGIR conference on Research and development in information retrieval.
  5. Jeffrey Dalton, Chenyan Xiong, and Jamie Callan. 2020. find
  6. TREC CAsT 2019: The Conversational Assistance Track Overview. arXiv e-prints, arXiv:2003.13624. — [arXiv:2003.13624](https://arxiv.org/abs/2003.13624) find
  7. Ahmed Elgohary, Denis Peskov, and Jordan Boyd-Graber. 2019. Can you unpack that? learning to rewrite questions-in-context. In Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing (EMNLP-IJCNLP…
  8. Thibault Formal, Benjamin Piwowarski, and Stéphane Clinchant. 2021. Splade: Sparse lexical and expansion model for first stage ranking. In Proceedings of the 44th International ACM SIGIR Conference on Research and Development in Information Retrieval, pages 2288–2292.
  9. Luyu Gao, Xueguang Ma, Jimmy Lin, and Jamie Callan. 2023. Precise zero-shot dense retrieval without relevance labels. In Proceedings of the 61st Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 1762–1777, Toronto, Canada. Association for Computational Li…
  10. Jeff Johnson, Matthijs Douze, and Hervé Jégou. 2021. Billion-scale similarity search with gpus. IEEE Transactions on Big Data, 7(3):535–547. find
  11. Vladimir Karpukhin, Barlas Oguz, Sewon Min, Patrick Lewis, Ledell Wu, Sergey Edunov, Danqi Chen, and Wen-tau Yih. 2020. Dense passage retrieval for open-domain question answering. In Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing (EMNLP), pages 6769–6781, Onli…
  12. Yannis Katsis, Sara Rosenthal, Kshitij Fadnis, Chulaka Gunasekara, Young-Suk Lee, Lucian Popa, Vraj Shah, Huaiyu Zhu, Danish Contractor, and Marina Danilevsky. 2025. mtrag: A multi-turn conversational benchmark for evaluating retrieval-augmented generation systems. Transactions of the Association fo… find
  13. Patrick Lewis, Ethan Perez, Aleksandra Piktus, Fabio Petroni, Vladimir Karpukhin, Naman Goyal, Heinrich Küttler, Mike Lewis, Wen-tau Yih, Tim Rocktäschel, Sebastian Riedel, and Douwe Kiela. 2020. Retrieval-augmented generation for knowledge-intensive nlp tasks. In Advances in Neural Information Proc…
  14. OpenAI. 2025. Introducing gpt-4.1 in the api. Accessed: 2026-03-02. find
  15. Stephen Robertson and Hugo Zaragoza. 2009. The probabilistic relevance framework: Bm25 and beyond. 3(4):333–389. find
  16. Sara Rosenthal, Yannis Katsis, Vraj Shah, Lihong He, Lucian Popa, and Marina Danilevsky. 2026a. Mtrag-un: A benchmark for open challenges in multi-turn rag conversations. Preprint, arXiv:2602.23184.
  17. Sara Rosenthal, Vraj Shah, Yannis Katsis, and Marina Danilevsky. 2026b. Semeval-2026 task 8: Mtrageval: Evaluating multi-turn rag conversations. In Proceedings of the 20th International Workshop on Semantic Evaluation (SemEval-2026), San Diego, California. Association for Computational Linguistics.
  18. Nandan Thakur, Nils Reimers, Johannes Daxenberger, and Iryna Gurevych. 2021. Beir: A heterogeneous benchmark for zero-shot evaluation of information retrieval models. In Proceedings of the Neural Information Retrieval Series at NeurIPS 2021. find
  19. Voyage AI. 2025. voyage-3-large: The new state-ofthe-art general-purpose embedding model. Accessed: 2026-02-24. find

Updated

↑ back to top

This page in Markdown → https://md.research.elamur.ai/free/en/papers/rios-2026-iimas-rag-semeval-2026-task/