Elamur Research⌕
☰
home / papers / 2026 / SEMIROUTER: Sparse-Data Enhanced Routing for Adaptive Multi-LLM System

SEMIROUTER: Sparse-Data Enhanced Routing for Adaptive Multi-LLM System

Paper · 2026 · Zijie Wang, Xinyu Yan, Che Wang et al.

Summary

Multi-LLM systems send each query to whichever model gives the best balance of answer quality and cost. Existing routers, however, need training data in which every candidate model has answered every query, and they usually must be retrained from scratch whenever a model is added or removed. SEMIROUTER tackles both issues. A Backbone router is trained on complete data for a set of "anchor" LLMs. A lightweight R-Adapter is then trained on sparse data to bring in new models. The adapter learns a trainable embedding for each new LLM while the backbone stays frozen. Contrastive objectives help separate new models from the anchors and from each other. Response quality labels come from LLM-as-a-judge scoring with DeepSeek-R1 and GPT-4, plus ground-truth answers where they exist. The authors also use per-query energy consumption as a routing cost, which they describe as a first for this task. In their experiments, SEMIROUTER improves data efficiency and adaptability while keeping routing quality competitive. This matters for real deployments, where model pools change often and annotation is expensive.

Key points

  • The R-Adapter holds only about 2% of the router's parameters. New LLMs can be added or dropped by retraining the adapter or deleting a model embedding, which avoids catastrophic forgetting in the backbone.
  • Evaluation covers benchmarks such as MBPP, MMLU, ARC, HellaSwag, WinoGrande, GSM8K, Accounting Audit and abstract2title. The model pool includes GPT-4 Turbo, GPT-3.5 Turbo, Llama 2-70B-Chat, Mixtral-8x7B, Mistral-7B, Code Llama-34B and WizardLM-13B, along with Claude models.
  • Energy-aware routing cuts consumption compared with sending every query to GPT-4 (13.529 Wh in total). Savings range from 69.38% on MMLU to 98.78% on abstract2title. Using GPT-4 as a backup still costs less, at roughly 6.8–8.8 Wh.
  • The router is robust to its hyperparameters. A contrastive weight λ between 0.1 and 0.3 and α = 0.3 work well, and the contrastive loss should stay smaller than the main prediction loss.
  • There are limitations. The sparse data was simulated by resampling existing datasets, Claude was excluded from energy routing, and GPT-3.5 energy use was assumed to be 0.2 Wh per query.

Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.

 Institutions

🏢Alibaba GroupHangzhou · China168 works · 15,386 citations🎓Nanyang Technological UniversitySingapore · Singapore156 works · 25,644 citations

 Fields, topics and concepts

📚 Shelves 4The field of science this work belongs to (OpenAlex fields).Computer Science · 16,667 worksArtificial Intelligence · 16,194 worksArts and Humanities · 16,039 worksLanguage and Linguistics · 15,891 works

🧭 Topics 1Research subjects and languages the work is about.Language models & pretraining · 3,320 works

Cite

Cite as Zijie Wang, Xinyu Yan, Che Wang, Zeng Zihao, Lei Xiao and Wei Yang Bryan Lim. 2026. SEMIROUTER: Sparse-Data Enhanced Routing for Adaptive Multi-LLM System. Proceedings of the 19th Conference of the European Chapter of the Association for Computational Linguistics (Volume 1: Long Papers).

The paper itself, not this page. BibTeX from the library's record of the work. DOI 10.18653/v1/2026.eacl-long.228

@inproceedings{wang2026semirouter,
  title = {{SEMIROUTER: Sparse-Data Enhanced Routing for Adaptive Multi-LLM System}},
  author = {Zijie Wang, Xinyu Yan, Che Wang, Zeng Zihao, Lei Xiao and Wei Yang Bryan Lim},
  booktitle = {Proceedings of the 19th Conference of the European Chapter of the Association for Computational Linguistics (Volume 1: Long Papers)},
  year = {2026},
  doi = {10.18653/v1/2026.eacl-long.228},
  url = {https://doi.org/10.18653/v1/2026.eacl-long.228}
}

References

29 references, 14 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

  1. Josh Achiam, Steven Adler, Sandhini Agarwal, Lama Ahmad, Ilge Akkaya, Florencia Leoni Aleman, Diogo Almeida, Janko Altenschmidt, Sam Altman, Shyamal Anadkat, and 1 others. 2023. Gpt-4 technical report. arXiv preprint arXiv:2303.08774.
  2. AIEnergyScore. 2025. Ai energy score leaderboard. https://huggingface.co/spaces/ AIEnergyScore/Leaderboard. Accessed: 2025-10-03. ↗
  3. Jacob Austin, Augustus Odena, Maxwell Nye, Maarten Bosma, Henryk Michalewski, David Dohan, Ellen Jiang, Carrie Cai, Michael Terry, Quoc Le, and 1 others. 2021. Program synthesis with large language models. arXiv preprint arXiv:2108.07732.
  4. Tom Brown, Benjamin Mann, Nick Ryder, Melanie Subbiah, Jared D Kaplan, Prafulla Dhariwal, Arvind Neelakantan, Pranav Shyam, Girish Sastry, Amanda Askell, and 1 others. 2020. Language models are few-shot learners. Advances in neural information processing systems, 33:1877–1901.
  5. Lingjiao Chen, Matei Zaharia, and James Zou. 2023. Frugalgpt: How to use large language models while reducing cost and improving performance. arXiv preprint arXiv:2305.05176.
  6. Shuhao Chen, Weisen Jiang, Baijiong Lin, James Kwok, and Yu Zhang. 2024. Routerdc: Query-based router by dual contrastive learning for assembling large language models. Advances in Neural Information Processing Systems, 37:66305–66328. find
  7. Wei-Lin Chiang, Lianmin Zheng, Ying Sheng, Anastasios N. Angelopoulos, Tianle Li, Dacheng Li, Banghua Zhu, Hao Zhang, Michael I. Jordan, Joseph E. Gonzalez, and Ion Stoica. 2024. Chatbot arena: An open platform for evaluating LLMs by human preference. In Proceedings of the 41st International Confere…
  8. Peter Clark, Isaac Cowhey, Oren Etzioni, Tushar Khot, Ashish Sabharwal, Carissa Schoenick, and Oyvind Tafjord. 2018. Think you have solved question answering? try arc, the ai2 reasoning challenge. arXiv preprint arXiv:1803.05457.
  9. Karl Cobbe, Vineet Kosaraju, Mohammad Bavarian, Mark Chen, Heewoo Jun, Lukasz Kaiser, Matthias Plappert, Jerry Tworek, Jacob Hilton, Reiichiro Nakano, and 1 others. 2021. Training verifiers to solve math word problems. arXiv preprint arXiv:2110.14168.
  10. Jacob Devlin, Ming-Wei Chang, Kenton Lee, and Kristina Toutanova. 2019. Bert: Pre-training of deep bidirectional transformers for language understanding. In Proceedings of the 2019 conference of the North American chapter of the association for computational linguistics: human language technologies,…
  11. Dujian Ding, Ankur Mallick, Chi Wang, Robert Sim, Subhabrata Mukherjee, Victor Ruhle, Laks VS Lakshmanan, and Ahmed Hassan Awadallah. 2024. Hybrid llm: Cost-efficient and quality-aware query routing. arXiv preprint arXiv:2404.14618. find
  12. Aaron Grattafiori, Abhimanyu Dubey, Abhinav Jauhri, Abhinav Pandey, Abhishek Kadian, Ahmad Al-Dahle, Aiesha Letman, Akhil Mathur, Alan Schelten, Alex Vaughan, and 1 others. 2024. The llama 3 herd of models. arXiv preprint arXiv:2407.21783.
  13. Pengcheng He, Jianfeng Gao, and Weizhu Chen. 2021. Debertav3: Improving deberta using electra-style pre-training with gradient-disentangled embedding sharing. arXiv preprint arXiv:2111.09543.
  14. Dan Hendrycks, Collin Burns, Steven Basart, Andy Zou, Mantas Mazeika, Dawn Song, and Jacob Steinhardt. 2020. Measuring massive multitask language understanding. arXiv preprint arXiv:2009.03300.
  15. Edward J Hu, Yelong Shen, Phillip Wallis, Zeyuan Allen-Zhu, Yuanzhi Li, Shean Wang, Lu Wang, Weizhu Chen, and 1 others. 2022. Lora: Low-rank adaptation of large language models. ICLR, 1(2):3.
  16. Qitian Jason Hu, Jacob Bieker, Xiuyu Li, Nan Jiang, Benjamin Keigwin, Gaurav Ranganath, Kurt Keutzer, and Shriyash Kaustubh Upadhyay. 2024. Routerbench: A benchmark for multi-llm routing system. arXiv preprint arXiv:2403.12031. — [arXiv:2403.12031](https://arxiv.org/abs/2403.12031) find
  17. Nidhal Jegham, Marwan Abdelatti, Lassad Elmoubarki, and Abdeltawab Hendawi. 2025. How hungry is ai? benchmarking energy, water, and carbon footprint of llm inference. arXiv preprint arXiv:2505.09598. — [arXiv:2505.09598](https://arxiv.org/abs/2505.09598) find
  18. Albert Q. Jiang, Alexandre Sablayrolles, Arthur Mensch, Chris Bamford, Devendra Singh Chaplot, Diego de las Casas, Florian Bressand, Gianna Lengyel, Guillaume Lample, Lucile Saulnier, Lélio Renard Lavaud, Marie-Anne Lachaux, Pierre Stock, Teven Le Scao, Thibaut Lavril, Thomas Wang, Timothée Lacroix,… find
  19. OpenRouter, Inc. 2025. Openrouter: The unified interface for llms. https://openrouter.ai/. Accessed: 2025-10-03. ↗
  20. Baptiste Roziere, Jonas Gehring, Fabian Gloeckle, Sten Sootla, Itai Gat, Xiaoqing Ellen Tan, Yossi Adi, Jingyu Liu, Romain Sauvestre, Tal Remez, and 1 others. 2023. Code llama: Open foundation models for code. arXiv preprint arXiv:2308.12950.
  21. Keisuke Sakaguchi, Ronan Le Bras, Chandra Bhagavatula, and Yejin Choi. 2021. Winogrande: An adversarial winograd schema challenge at scale. Communications of the ACM, 64(9):99–106. find
  22. Hugo Touvron, Louis Martin, Kevin Stone, Peter Albert, Amjad Almahairi, Yasmine Babaei, Nikolay Bashlykov, Soumya Batra, Prajjwal Bhargava, Shruti Bhosale, and 1 others. 2023. Llama 2: Open foundation and fine-tuned chat models. arXiv preprint arXiv:2307.09288.
  23. Asterios Tsiourvas, Wei Sun, and Georgia Perakis. 2025. Causal llm routing: End-to-end regret minimization from observational data. arXiv preprint arXiv:2505.16037. — [arXiv:2505.16037](https://arxiv.org/abs/2505.16037) find
  24. Rowan Zellers, Ari Holtzman, Yonatan Bisk, Ali Farhadi, and Yejin Choi. 2019. Hellaswag: Can a machine really finish your sentence? arXiv preprint arXiv:1905.07830. find
  25. Ziyu Zhao, Leilei Gan, Guoyin Wang, Wangchunshu Zhou, Hongxia Yang, Kun Kuang, and Fei Wu. 2024. Loraretriever: Input-aware lora retrieval and composition for mixed tasks in the wild. arXiv preprint arXiv:2402.09997. find
  26. Ruihan Jin, Pengpeng Shao, Zhengqi Wen, Jinyang Wu, Mingkuan Feng, Shuai Zhang, and Jianhua Tao. 2025. Radialrouter: Structured representation for efficient and robust large language models routing. arXiv preprint arXiv:2506.03880. find
  27. Keming Lu, Hongyi Yuan, Runji Lin, Junyang Lin, Zheng Yuan, Chang Zhou, and Jingren Zhou. 2023. Routing to the expert: Efficient reward-guided ensemble of large language models. arXiv preprint arXiv:2311.08692. find
  28. Isaac Ong, Amjad Almahairi, Vincent Wu, Wei-Lin Chiang, Tianhao Wu, Joseph E Gonzalez, M Waleed Kadous, and Ion Stoica. 2024. Routellm: Learning to route llms with preference data. arXiv preprint arXiv:2406.18665. find
  29. OpenAI. 2025. Introducing gpt-5. https://openai.com/index/introducing-gpt-5/. Accessed: 2025-10-03. ↗

Updated

↑ back to top

This page in Markdown → https://md.research.elamur.ai/free/en/papers/wang-2026-semirouter-sparse-data-enhanced-routing/