Elamur Research⌕
☰
home / papers / 2026 / What the Router Sees Matters: Funnel Pooling for Fast, Content Driven Expert Routing

What the Router Sees Matters: Funnel Pooling for Fast, Content Driven Expert Routing

Paper · 2026 · Josef Pichlmeier, Sebastian Nicolas Mueller, Jakob Sturm et al.

Summary

Systems that send prompts to specialized experts usually make that choice from one vector per input, but the effect of how this vector is built is rarely tested in isolation. The authors keep a centroid-based routing pipeline constant and change only the representation. The pipeline uses L2 normalization, UMAP to 256 dimensions, K-means with K = 8, and top-1 nearest-centroid assignment. It runs on 70,000 multilingual samples drawn from seven source datasets. They compare four representations:

  • an external encoder, Qwen3-Embedding-8B;
  • mean pooling over hidden states of a frozen Llama-3.1-8B;
  • attention pooling over the same hidden states;
  • their new Funnel pooling.

Funnel is a small trainable readout that compresses token states step by step with cross-attention. It is trained contrastively so that each question's vector sits close to an encoding of its gold answer. The parameter-free readouts tend to group prompts by language or source format, while Funnel yields clusters that follow topic more closely. Routing accuracy with LoRA experts remains competitive, and prefill time grows only slightly. The study argues that routing quality depends on representations as well as on router design, and that no separate embedding encoder is required.

Key points

  • The setup is controlled: datasets and splits, L2 normalization, UMAP (256 dimensions) and K-means (K = 8, one more than the 7 source datasets) are fixed. Only the representation changes, and the extraction layer is ablated for the in-model methods.
  • Funnel pooling uses three cross-attention blocks whose learnable queries shrink from 64 to 16 to 1. It has a student path for the question and an EMA-updated teacher path for the answer. Training uses a cosine InfoNCE loss (τ = 0.25, β = 0.95) for 3 epochs, and only the student path runs at inference.
  • With mean and attention pooling, clusters often mirror language or format. For example, function-calling prompts split by language rather than by endpoint type. Funnel reduces this language- and dataset-driven clustering.
  • Funnel adds about 3 ms to prefill latency compared with average and attention pooling, and it avoids running a standalone encoder.
  • Stated limitations:
    • the study uses a single backbone (Llama-3.1-8B) and seven datasets, two of them synthetic;
    • accuracy is judged by two LLM judges;
    • there is no sensitivity analysis over UMAP settings or K.

Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.

 Institutions

🎓Ludwig-Maximilians-Universität MünchenMunich · Germany113 works · 56,817 citations🎓Technical University of MunichMunich · Germany62 works · 48,970 citations

Also on the byline: BMW · BMW Group.

 Fields, topics and concepts

📚 Shelves 5The field of science this work belongs to (OpenAlex fields).Computer Science · 18,614 worksArtificial Intelligence · 17,965 worksArts and Humanities · 17,450 worksLanguage and Linguistics · 17,296 worksInformation Systems · 1,730 works

Cite

Cite as Josef Pichlmeier, Sebastian Nicolas Mueller, Jakob Sturm, Josef Dräxl and Andre Luckow. 2026. What the Router Sees Matters: Funnel Pooling for Fast, Content Driven Expert Routing. Proceedings of the 19th Conference of the European Chapter of the Association for Computational Linguistics (Volume 4: Student Research Workshop).

The paper itself, not this page. BibTeX from the library's record of the work. DOI 10.18653/v1/2026.eacl-srw.66

@inproceedings{pichlmeier2026what,
  title = {{What the Router Sees Matters: Funnel Pooling for Fast, Content Driven Expert Routing}},
  author = {Josef Pichlmeier, Sebastian Nicolas Mueller, Jakob Sturm, Josef Dräxl and Andre Luckow},
  booktitle = {Proceedings of the 19th Conference of the European Chapter of the Association for Computational Linguistics (Volume 4: Student Research Workshop)},
  year = {2026},
  doi = {10.18653/v1/2026.eacl-srw.66},
  url = {https://doi.org/10.18653/v1/2026.eacl-srw.66}
}

References

35 references, 11 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

  1. Parishad BehnamGhader, Vaibhav Adlakha, Marius Mosbach, Dzmitry Bahdanau, Nicolas Chapados, and Siva Reddy. 2024. LLM2vec: Large language models are secretly powerful text encoders. In First Conference on Language Modeling.
  2. Qian Chen, Wen Wang, Qinglin Zhang, Siqi Zheng, Chong Deng, Hai Yu, Jiaqing Liu, Yukun Ma, and Chong Zhang. 2023. Ditto: A simple and efficient approach to improve sentence embeddings. Preprint, arXiv:2305.10786. find
  3. Yu-Neng Chuang, Prathusha Kameswara Sarma, Parikshit Gopalan, John Boccio, Sara Bolouki, Xia Hu, and Helen Zhou. 2025. Learning to route llms with confidence tokens. Preprint, arXiv:2410.13284. — [arXiv:2410.13284](https://arxiv.org/abs/2410.13284) find
  4. Yu Fan, Yang Tian, Shauli Ravfogel, Mrinmaya Sachan, Elliott Ash, and Alexander Hoyle. 2025. The medium is not the message: Deconfounding document embeddings via linear concept erasure. Preprint, arXiv:2507.01234. find
  5. Wenfeng Feng, Chuzhan Hao, Yuewei Zhang, Yu Han, and Hao Wang. 2024. Mixture-of-loras: An efficient multitask tuning for large language models. Preprint, arXiv:2403.03432. — [arXiv:2403.03432](https://arxiv.org/abs/2403.03432) find
  6. Aaron Grattafiori, Abhimanyu Dubey, Abhinav Jauhri, et al. 2024. The llama 3 herd of models. Preprint, arXiv:2407.21783.
  7. Maarten Grootendorst. 2022. Bertopic: Neural topic modeling with a class-based tf-idf procedure. Preprint, arXiv:2203.05794.
  8. Huggingface. MTEB leaderboard. https://huggingface.co/spaces/mteb/leaderboard. Accessed 2025-09-21. ↗
  9. IAB Technology Laboratory. 2025. Iab tech lab taxonomies. GitHub repository. Accessed 2025-12-21. find
  10. Di Jin, Eileen Pan, Nassim Oufattole, Wei-Hung Weng, Hanyi Fang, and Peter Szolovits. 2020. What disease does this patient have? a large-scale open domain question answering dataset from medical exams. Preprint, arXiv:2009.13081.
  11. Eunsu Kim, Haneul Yoo, Guijin Son, Hitesh Patel, Amit Agarwal, and Alice Oh. 2025. Benchhub: A unified benchmark suite for holistic and customizable llm evaluation. Preprint, arXiv:2506.00482. — [arXiv:2506.00482](https://arxiv.org/abs/2506.00482) find
  12. Chankyu Lee, Rajarshi Roy, Mengyao Xu, Jonathan Raiman, Mohammad Shoeybi, Bryan Catanzaro, and Wei Ping. Nv-embed: Improved techniques for training llms as generalist embedding models. In The Thirteenth International Conference on Learning Representations.
  13. Haonan Li, Yixuan Zhang, Fajri Koto, Yifei Yang, Hai Zhao, Yeyun Gong, Nan Duan, and Timothy Baldwin. 2023. Cmmlu: Measuring massive multitask language understanding in chinese. Preprint, arXiv:2306.09212.
  14. Yang Li. 2025. Rethinking predictive modeling for llm routing: When simple knn beats complex learned routers. Preprint, arXiv:2505.12601. — [arXiv:2505.12601](https://arxiv.org/abs/2505.12601) find
  15. Mohammed Muqeeth, Haokun Liu, Yufan Liu, and Colin Raffel. 2024b. Learning to route among specialized experts for zero-shot generalization. In Proceedings of the 41st International Conference on Machine Learning, pages 36829–36846. find
  16. Isaac Ong, Amjad Almahairi, Vincent Wu, Wei-Lin Chiang, Tianhao Wu, Joseph E. Gonzalez, M Waleed Kadous, and Ion Stoica. 2025. Routellm: Learning to route llms with preference data. Preprint, arXiv:2406.18665.
  17. OpenAI. 2025. Introducing gpt-5. https://openai.com/index/introducing-gpt-5/. ↗
  18. Oleksiy Ostapenko, Zhan Su, Edoardo Maria Ponti, Laurent Charlin, Nicolas Le Roux, Lucas Caccia, and Alessandro Sordoni. 2024. Towards modular llms by building and reusing a library of loras. In Proceedings of the 41st International Conference on Machine Learning, pages 38885–38904. find
  19. Josef Pichlmeier, Philipp Ross, and Andre Luckow. 2024. Performance characterization of expert router for scalable llm inference. In 2024 IEEE International Conference on Big Data (BigData), pages 1686–1693. find
  20. William M. Rand. 1971. Objective criteria for the evaluation of clustering methods. Journal of the American Statistical Association, 66(336):846–850. find
  21. S. Lloyd. 1982. Least squares quantization in pcm. IEEE Transactions on Information Theory, 28(2):129–137. find
  22. Nestor Maslej, Loredana Fattorini, Raymond Perrault, Yolanda Gil, Vanessa Parli, Njenga Kariuki, Emily Capstick, Anka Reuel, Erik Brynjolfsson, John Etchemendy, Katrina Ligett, Terah Lyons, James Manyika, Juan Carlos Niebles, Yoav Shoham, Russell Wald, Tobi Walsh, Armin Hamrah, Lapo Santarlasci, Jul… find
  23. Evgenia Rusak, Patrik Reizinger, Attila Juhos, Oliver Bringmann, Roland S. Zimmermann, and Wieland Brendel. 2025. Infonce: Identifying the gap between theory and practice. Preprint, arXiv:2407.00143. — [arXiv:2407.00143](https://arxiv.org/abs/2407.00143) find
  24. Tal Shnitzer, Anthony Ou, Mirian Silva, Kate Soule, Yuekai Sun, Justin Solomon, Neil Thompson, and Mikhail Yurochkin. 2023. Large language model routing with benchmark datasets. In Annual Conference on Neural Information Processing Systems.
  25. Oscar Skean, Md Rifat Arefin, Dan Zhao, Niket Patel, Jalal Naghiyev, Yann LeCun, and Ravid Shwartz-Ziv. 2025. Layer by layer: Uncovering hidden representations in language models.
  26. Leland McInnes, John Healy, and James Melville. 2020. Umap: Uniform manifold approximation and projection for dimension reduction. Preprint, arXiv:1802.03426.
  27. Yixuan Tang and Yi Yang. 2024. Pooling and attention: What are effective designs for llm-based embedding models? Preprint, arXiv:2409.02727. — [arXiv:2409.02727](https://arxiv.org/abs/2409.02727) find
  28. Arindam Mitra, Luciano Del Corro, Guoqing Zheng, et al. 2024. Agentinstruct: Toward generative teaching with agentic flows. Preprint, arXiv:2407.03502. find
  29. Gemma Team, Aishwarya Kamath, Johan Ferret, Shreya Pathak, Nino Vieillard, Ramona Merhej, et al. 2025. Gemma 3 technical report. Preprint, arXiv:2503.19786. find
  30. Daniel Morales-Brotons, Thijs Vogels, and Hadrien Hendrikx. 2024. Exponential moving average of weights in deep learning: Dynamics and benefits. Preprint, arXiv:2411.18704. — [arXiv:2411.18704](https://arxiv.org/abs/2411.18704) find
  31. Mohammed Muqeeth, Haokun Liu, Yufan Liu, and Colin Raffel. 2024a. Learning to route among specialized experts for zero-shot generalization. Preprint, arXiv:2402.05859. find
  32. Xinyuan Wang, Yanchi Liu, Wei Cheng, Xujiang Zhao, Zhengzhang Chen, Wenchao Yu, Yanjie Fu, and Haifeng Chen. 2025. Mixllm: Dynamic routing in mixed large language models. In Proceedings of the 2025 Conference of the Nations of the Americas Chapter of the Association for Computational Linguistics: Hu… find
  33. Prateek Yadav, Colin Raffel, Mohammed Muqeeth, Lucas Caccia, Haokun Liu, Tianlong Chen, Mohit Bansal, Leshem Choshen, and Alessandro Sordoni. 2025. A survey on model moerging: Recycling and routing among specialized experts for collaborative learning. Preprint, arXiv:2408.07057. find
  34. Yanzhao Zhang, Mingxin Li, Dingkun Long, Xin Zhang, Huan Lin, Baosong Yang, Pengjun Xie, An Yang, Dayiheng Liu, Junyang Lin, Fei Huang, and Jingren Zhou. 2025. Qwen3 embedding: Advancing text embedding and reranking through foundation models. Preprint, arXiv:2506.05176.
  35. Zonghui. 2025. CNMLEQA: Chinese national medical licensing examinations dataset. https://huggingface.co/datasets/zonghui/CNMLEQA. Hugging Face; accessed 2025-09-21. ↗

Updated

↑ back to top

This page in Markdown → https://md.research.elamur.ai/free/en/papers/pichlmeier-2026-what-router-sees-matters-funnel/