Elamur Research⌕
☰
home / papers / 2026 / Distilling Large Embeddings via Hyperspherical Householder Quantization

Distilling Large Embeddings via Hyperspherical Householder Quantization

Paper · 2026 · Yihang Wang, Bin Wu, Yueyang Su et al.

Summary

Generative retrieval trains a sequence model to output a document identifier for a query, so no dense vector index is needed. Identifiers are usually built by quantizing document embeddings with Euclidean methods such as PQ or RQ. The authors argue that these methods fit contrastively trained embeddings poorly, because those embeddings encode meaning mainly through direction (cosine similarity). They also note that Euclidean schemes tend to need long codes or very large codebooks.

The paper proposes Hyperspherical Householder Quantization (HHQ). It works on the unit sphere:

  • The first token is the codeword closest in angle to the normalized embedding.
  • Each later layer selects a learned direction and applies a Householder reflection. The reflection keeps the vector's length and moves it closer in angle to the target.
  • Codebooks are trained end to end with a cosine-based loss.

To make the generator produce valid identifiers, the authors add two components. Constrained supervised fine-tuning (cSFT) aligns training with tree-constrained decoding. Tree-aware dynamic masking prunes invalid branches.

On NQ and MS MARCO, HHQ matches or beats prior methods with only five tokens per document. This reduces decoding cost and shows that identifier length is an important lever for the trade-off between accuracy and efficiency.

Key points

  • HHQ represents each document as a short sequence of codebook indices. The indices are produced by norm-preserving, involutive Householder reflections, and the codebooks are optimized for cosine similarity rather than Euclidean reconstruction.
  • The motivating example: PQ with k = 256 and m = 24 yields 256^24 possible IDs for a corpus of only about 300k documents. Such an oversized ID space leads to many invalid generations.
  • Without cSFT, the generator did not learn useful identifier mappings even after 20 epochs. With cSFT, it converged within a few epochs and scored higher on R@1, R@10 and MRR@10.
  • Pseudo-query pretraining with DocT5Query raised R@1 by 4.4–6.0 points. The gain held across codebook depths 4–6 and embedding sizes 256 and 512.
  • The authors list these limitations:
    • Deeper layers can suffer codeword collapse.
    • The identifiers are tied to a specific corpus, and incremental indexing was not evaluated.
    • Short IDs struggle to separate documents in dense regions such as MS MARCO Relevant 300K.

Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.

 Institutions

🏢ByteDanceBeijing · China41 works · 4,180 citations

 Fields, topics and concepts

📚 Shelves 5The field of science this work belongs to (OpenAlex fields).Computer Science · 16,667 worksArtificial Intelligence · 16,194 worksArts and Humanities · 16,039 worksLanguage and Linguistics · 15,891 worksComputer Vision and Pattern Recognition · 2,869 works

Cite

Cite as Yihang Wang, Bin Wu, Yueyang Su, Tianfu Zhang, Yiqi Du, Lei Yu, Jiafeng Guo and Xueqi Cheng. 2026. Distilling Large Embeddings via Hyperspherical Householder Quantization. Proceedings of the 64th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers).

The paper itself, not this page. BibTeX from the library's record of the work. DOI 10.18653/v1/2026.acl-long.482

@inproceedings{wang2026distilling,
  title = {{Distilling Large Embeddings via Hyperspherical Householder Quantization}},
  author = {Yihang Wang, Bin Wu, Yueyang Su, Tianfu Zhang, Yiqi Du, Lei Yu, Jiafeng Guo and Xueqi Cheng},
  booktitle = {Proceedings of the 64th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers)},
  year = {2026},
  doi = {10.18653/v1/2026.acl-long.482},
  url = {https://doi.org/10.18653/v1/2026.acl-long.482}
}

References

32 references, 11 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

  1. 2025. Onerec: Unifying retrieve and rank with generative recommender and iterative preference alignment. arXiv preprint arXiv:2502.18965. — [arXiv:2502.18965](https://arxiv.org/abs/2502.18965) find
  2. Tianyu Gao, Xingcheng Yao, and Danqi Chen. 2021. SimCSE: Simple contrastive learning of sentence embeddings. In Proceedings of the 2021 Conference on Empirical Methods in Natural Language Processing, pages 6894–6910, Online and Punta Cana, Dominican Republic. Association for Computational Linguistic…
  3. Alston S. Householder. 1958. Unitary triangularization of a nonsymmetric matrix. J. ACM, 5(4):339–342. find
  4. Vladimir Karpukhin, Barlas Oguz, Sewon Min, Patrick Lewis, Ledell Wu, Sergey Edunov, Danqi Chen, and Wen-tau Yih. 2020. Dense passage retrieval for open-domain question answering. In Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing (EMNLP), pages 6769–6781, Onli…
  5. Aditya Kusupati, Gantavya Bhatt, Aniket Rege, Matthew Wallingford, Aditya Sinha, Vivek Ramanujan, William Howard-Snyder, Kaifeng Chen, Sham Kakade, Prateek Jain, and Ali Farhadi. 2022. Matryoshka representation learning. In Proceedings of the 36th International Conference on Neural Information Proce… find
  6. Tom Kwiatkowski, Jennimaria Palomaki, Olivia Redfield, Michael Collins, Ankur Parikh, Chris Alberti, Danielle Epstein, Illia Polosukhin, Jacob Devlin, Kenton Lee, Kristina Toutanova, Llion Jones, Matthew Kelcey, Ming-Wei Chang, Andrew M. Dai, Jakob Uszkoreit, Quoc Le, and Slav Petrov. 2019. Natural…
  7. Michele Bevilacqua, Giuseppe Ottaviano, Patrick Lewis, Wen tau Yih, Sebastian Riedel, and Fabio Petroni. 2022a. Autoregressive search engines: Generating substrings as document identifiers. In arXiv pre-print 2204.10628. find
  8. Jinhyuk Lee, Feiyang Chen, Sahil Dua, Daniel Cer, Madhuri Shanbhogue, Iftekhar Naim, Gustavo Hernández Ábrego, Zhe Li, Kaifeng Chen, Henrique Schechter Vera, and 1 others. 2025. Gemini embedding: Generalizable embeddings from gemini. arXiv preprint arXiv:2503.07891.
  9. Michele Bevilacqua, Giuseppe Ottaviano, Patrick Lewis, Wen-tau Yih, Sebastian Riedel, and Fabio Petroni. 2022b. Autoregressive search engines: generating substrings as document identifiers. In Proceedings of the 36th International Conference on Neural Information Processing Systems, NIPS ’22, Red Ho… find
  10. Yongqi Li, Nan Yang, Liang Wang, Furu Wei, and Wenjie Li. 2023. Multiview identifiers enhanced generative retrieval. In Proceedings of the 61st Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 6636–6648, Toronto, Canada. Association for Computational Lin… find
  11. Xinyan Dai, Xiao Yan, Kelvin KW Ng, Jiu Liu, and James Cheng. 2020. Norm-explicit quantization: Improving vector quantization for maximum inner product search. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 34, pages 51–58. find
  12. Yongqi Li, Nan Yang, Liang Wang, Furu Wei, and Wenjie Li. 2024. Learning to rank in generative retrieval. In Proceedings of the Thirty-Eighth AAAI Conference on Artificial Intelligence and Thirty-Sixth Conference on Innovative Applications of Artificial Intelligence and Fourteenth Symposium on Educa… find
  13. Jiaxin Deng, Shiyao Wang, Kuo Cai, Lejian Ren, Qigen Hu, Weifeng Ding, Qiang Luo, and Guorui Zhou. 10571 10 find
  14. Kidist Amde Mekonnen, Yubao Tang, and Maarten de Rijke. 2025. Lightweight and direct document relevance optimization for generative information retrieval. In Proceedings of the 48th International ACM SIGIR Conference on Research and Development in Information Retrieval, SIGIR ’25, page 1327–1338, Ne… find
  15. Tri Nguyen, Mir Rosenberg, Xia Song, Jianfeng Gao, Saurabh Tiwary, Rangan Majumder, and Li Deng. 2016. Ms marco: A human-generated machine reading comprehension dataset.
  16. Jianmo Ni, Gustavo Hernandez Abrego, Noah Constant, Ji Ma, Keith Hall, Daniel Cer, and Yinfei Yang. 2022. Sentence-t5: Scalable sentence encoders from pre-trained text-to-text models. In Findings of the Association for Computational Linguistics: ACL 2022, pages 1864–1874, Dublin, Ireland. Associatio…
  17. Rodrigo Nogueira, Jimmy Lin, and AI Epistemic. 2019. From doc2query to doctttttquery. Online preprint, 6(2). find
  18. Colin Raffel, Noam Shazeer, Adam Roberts, Katherine Lee, Sharan Narang, Michael Matena, Yanqi Zhou, Wei Li, and Peter J. Liu. 2020. Exploring the limits of transfer learning with a unified text-to-text transformer. Journal of Machine Learning Research, 21(140):1–67.
  19. Shashank Rajput, Nikhil Mehta, Anima Singh, Raghu-nandan Keshavan, Trung Vu, Lukasz Heidt, Lichan Hong, Yi Tay, Vinh Q. Tran, Jonah Samost, Maciej Kula, Ed H. Chi, and Maheswaran Sathiamoorthy. 2023. Recommender systems with generative retrieval. In Proceedings of the 37th International Conference o… find
  20. Nils Reimers and Iryna Gurevych. 2019. Sentence-BERT: Sentence embeddings using Siamese BERT-networks. In Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing (EMNLP-IJCNLP), pages 3982–3992,…
  21. Nils Reimers and Iryna Gurevych. 2021. The curse of dense low-dimensional information retrieval for large index sizes. In Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Conference on Natural Language Processing (Volume 2: Shor… find
  22. Stephen Robertson, S. Walker, S. Jones, M. M. Hancock-Beaulieu, and M. Gatford. 1995. Okapi at trec-3. In Overview of the Third Text REtrieval Conference (TREC-3), pages 109–126. Gaithersburg, MD: NIST.
  23. Weiwei Sun, Lingyong Yan, Zheng Chen, Shuaiqiang Wang, Haichao Zhu, Pengjie Ren, Zhumin Chen, Dawei Yin, Maarten de Rijke, and Zhaochun Ren. 2023. Learning to tokenize for generative retrieval. In Proceedings of the 37th International Conference on Neural Information Processing Systems, NIPS ’23, Re… find
  24. Yi Tay, Vinh Q. Tran, Mostafa Dehghani, Jianmo Ni, Dara Bahri, Harsh Mehta, Zhen Qin, Kai Hui, Zhe Zhao, Jai Gupta, Tal Schuster, William W. Cohen, and Donald Metzler. 2022. Transformer memory as a differentiable search index. In Proceedings of the 36th International Conference on Neural Information…
  25. Yujing Wang, Yingyan Hou, Haonan Wang, Ziming Miao, Shibin Wu, Hao Sun, Qi Chen, Yuqing Xia, Chengmin Chi, Guoshuai Zhao, Zheng Liu, Xing Xie, Hao Allen Sun, Weiwei Deng, Qi Zhang, and Mao Yang. 2022. A neural corpus indexer for document retrieval. In Proceedings of the 36th International Conference… find
  26. Jingtao Zhan, Jiaxin Mao, Yiqun Liu, Min Zhang, and Shaoping Ma. 2020. Repbert: Contextualized text embeddings for first-stage retrieval. arXiv preprint arXiv:2006.15498. — [arXiv:2006.15498](https://arxiv.org/abs/2006.15498) find
  27. Yanzhao Zhang, Mingxin Li, Dingkun Long, Xin Zhang, Huan Lin, Baosong Yang, Pengjun Xie, An Yang, Dayiheng Liu, Junyang Lin, and 1 others. 2025. Qwen3 embedding: Advancing text embedding and reranking through foundation models. arXiv preprint arXiv:2506.05176.
  28. Xuefei Zhe, Shifeng Chen, and Hong Yan. 2019. Directional statistics-based deep metric learning for image classification and retrieval. Pattern Recognition, 93:113–123. find
  29. Yujia Zhou, Jing Yao, Zhicheng Dou, Yiteng Tu, Ledell Wu, Tat-Seng Chua, and Ji-Rong Wen. 2024a. Roger: Ranking-oriented generative retrieval. ACM Trans. Inf. Syst., 42(6). find
  30. Yujia Zhou, Jing Yao, Zhicheng Dou, Ledell Wu, and Ji-Rong Wen. 2022. Dynamicretriever: A pre-training model-based ir system with neither sparse nor dense index. arXiv preprint arXiv:2203.00537. — [arXiv:2203.00537](https://arxiv.org/abs/2203.00537) find
  31. Yujia Zhou, Jing Yao, Ledell Wu, Zhicheng Dou, and Ji-Rong Wen. 2024b. Webultron: An ultimate retriever on webpages under the model-centric paradigm. IEEE Trans. on Knowl. and Data Eng., 36(9):4996–5006. find
  32. Shengyao Zhuang, Houxing Ren, Linjun Shou, Jian Pei, Ming Gong, Guido Zuccon, and Daxin Jiang. 2022. Bridging the gap between indexing and retrieval for differentiable search index with query generation. arXiv preprint arXiv:2206.10128. 10572 11 — [arXiv:2206.10128](https://arxiv.org/abs/2206.10128) find

Updated

↑ back to top

This page in Markdown → https://md.research.elamur.ai/free/en/papers/wang-2026-distilling-large-embeddings-hyperspherical-householder/