Elamur Research⌕
☰
home / papers / 2026 / SlugRAG at SemEval-2026 Task 8: Domain-Specific Fine-Tuning and Model Scaling for Multi-Turn RAG Retrieval

SlugRAG at SemEval-2026 Task 8: Domain-Specific Fine-Tuning and Model Scaling for Multi-Turn RAG Retrieval

Paper · 2026 · Pratibha Revankar, Jihye Kim and Umit Azirakhmet

Summary

The paper describes SlugRAG, an entry to SemEval-2026 Task 8 (MTRAGEval), Subtask A. The subtask asks systems to retrieve relevant passages for conversational queries whose meaning often depends on earlier turns. The authors fine-tune BGE dense retrievers (bge-base-en-v1.5 and bge-large-en-v1.5) with MultipleNegativesRankingLoss. Training uses roughly 2,900 query–passage pairs from four English domains: CLAPNQ, FIQA, GOVT and CLOUD.

They test improvements in four stages:

  • basic training settings;
  • learning-rate tuning and data augmentation;
  • BM25 hybrid fusion and cross-encoder reranking;
  • per-domain models and hard negative mining.

They also try LLM query rewriting with Qwen2.5-3B-Instruct. Augmentation and per-domain training gave the largest gains. The best offline configuration was a multi-domain fine-tuned BGE-large, reaching nDCG@10 0.5101 and Recall@10 0.6221, against 0.3000 and 0.3800 for the untuned model. Reranking and hard negatives did not help. The official submission ranked 25th of 38 teams. The study is a practical guide to which inexpensive retriever adaptations pay off in conversational RAG.

Key points

  • Domain-specific models: average gains over pre-trained BGE are 44.3% in Recall@10 and 47.8% in nDCG@10. CLAPNQ is strongest (R@10 0.6016, nDCG@10 0.4981), and CLOUD benefits most relative to the multi-domain model (+15.1%).
  • Data augmentation: back-translation paraphrases, synonym swaps and context expansion raised Recall@10 by 34.2%.
  • Hard negative mining: the mined-negative variants collapsed to about 0.14 nDCG@10. The authors blame using CosineSimilarityLoss instead of a contrastive loss, plus unlabeled relevant passages acting as false negatives.
  • Reranking: cross-encoder reranking left nDCG@10 almost unchanged (+0.05%) but cut Recall@10 by 6.5%.
  • Query rewriting: only "minimal clarification" rewriting helped on the baseline, by a small margin.
  • Leaderboard: the official score was nDCG@5 0.3847, versus 0.5776 for the top system. Offline scores come from a different test split, so the two sets of numbers are not comparable.
  • Error analysis: 100 failures were examined. Errors are dominated by coreference ambiguity (41%), temporal dependencies (28%) and multi-hop reasoning (19%).

Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.

 Institutions

🎓University of California, Santa CruzSanta Cruz · United States36 works · 3,515 citations

 Fields, topics and concepts

📚 Shelves 6The field of science this work belongs to (OpenAlex fields).Computer Science · 16,667 worksArtificial Intelligence · 16,194 worksArts and Humanities · 16,039 worksLanguage and Linguistics · 15,891 worksComputer Vision and Pattern Recognition · 2,869 worksInformation Systems · 1,437 works

🧭 Topics 1Research subjects and languages the work is about.Information retrieval & search · 473 works

Cite

Cite as Pratibha Revankar, Jihye Kim and Umit Azirakhmet. 2026. SlugRAG at SemEval-2026 Task 8: Domain-Specific Fine-Tuning and Model Scaling for Multi-Turn RAG Retrieval. Proceedings of the 20th International Workshop on Semantic Evaluation (2026).

The paper itself, not this page. BibTeX from the library's record of the work. DOI 10.18653/v1/2026.semeval-1.135

@inproceedings{revankar2026slugrag,
  title = {{SlugRAG at SemEval-2026 Task 8: Domain-Specific Fine-Tuning and Model Scaling for Multi-Turn RAG Retrieval}},
  author = {Pratibha Revankar, Jihye Kim and Umit Azirakhmet},
  booktitle = {Proceedings of the 20th International Workshop on Semantic Evaluation (2026)},
  year = {2026},
  doi = {10.18653/v1/2026.semeval-1.135},
  url = {https://doi.org/10.18653/v1/2026.semeval-1.135}
}

References

7 references, 4 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

  1. Beijing Academy of Artificial Intelligence. 2023. BAAI General Embedding (BGE) Model. https: Ac-//github.com/FlagOpen/FlagEmbedding. cessed: January 2025. find
  2. Vladimir Karpukhin, Barlas Oğuz, Sewon Min, Patrick Lewis, Ledell Wu, Sergey Edunov, Danqi Chen, and Wen-tau Yih. 2020. Dense Passage Retrieval for Open-Domain Question Answering. In Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing (EMNLP), pages 6769– 6781, On…
  3. Yannis Katsis, Sara Rosenthal, Kshitij Fadnis, Chulaka Gunasekara, Young-Suk Lee, Lucian Popa, Vraj Shah, Huaiyu Zhu, Danish Contractor, and Marina Danilevsky. 2025. mtrag: A multi-turn conversational benchmark for evaluating retrieval-augmented generation systems. Transactions of the Association fo… find
  4. Nils Reimers and Iryna Gurevych. 2019. Sentence-BERT: Sentence Embeddings using Siamese BERT-Networks. In Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing (EMNLP-IJCNLP), pages 3982–3992,…
  5. Sara Rosenthal, Yannis Katsis, Vraj Shah, Lihong He, Lucian Popa, and Marina Danilevsky. 2026a. Mtrag-un: A benchmark for open challenges in multi-turn rag conversations.
  6. Sara Rosenthal, Vraj Shah, Yannis Katsis, and Marina Danilevsky. 2026b. Semeval-2026 task 8: Mtrageval: Evaluating multi-turn rag conversations. In Proceedings of the 20th International Workshop on Semantic Evaluation (SemEval-2026), San Diego, California. Association for Computational Linguistics.…
  7. In all cases, the user message was formatted as Question: \<query\>. find

Updated

↑ back to top

This page in Markdown → https://md.research.elamur.ai/free/en/papers/revankar-2026-slugrag-semeval-2026-task-8/