Elamur Research⌕
☰
home / papers / 2026 / LLMs Meet Isolation Kernel: Lightweight, Learning-free Binary Embeddings for Fast Retrieval

LLMs Meet Isolation Kernel: Lightweight, Learning-free Binary Embeddings for Fast Retrieval

Paper · 2026 · Zhibo Zhang, Yang Xu, Kai Ming Ting et al.

Summary

Embeddings from LLM-based encoders such as LLM2Vec or Qwen3 often have thousands of dimensions, which makes storage and search expensive. Existing compression methods have drawbacks. MRL needs full fine-tuning of the model. CSR trains an adapter per code length and adds transformation overhead. Both lose some retrieval accuracy. The authors propose IKE (Isolation Kernel Embedding), which requires no training. It builds an Isolation Forest of random iTrees on a sample of corpus embeddings. Each document or query is then encoded by the leaf it reaches in every tree, giving a compact binary code that is compared with bitwise matching. IKE generalizes the earlier VDeH hashing method, which performs poorly on LLM embeddings. The authors argue that good binary hashing needs four properties: full space coverage, maximal entropy, bit independence, and a newly identified fourth one, high diversity of partitions. On text retrieval benchmarks, IKE nearly matches the accuracy of the original embeddings while being far faster and smaller. It also works well with HNSW indexes. This makes it attractive for low-cost, large-scale retrieval.

Key points

  • Exhaustive search: IKE gives a 2.5–16.7× speedup and 8–16× storage reduction. Its MRR@10 is retained at 98% to 101% of what the uncompressed LLM embeddings achieve. Each point needs only t·⌈log2 ψ⌉ bits, instead of 32 bits per dimension.
  • ANN search: Paired with HNSW, IKE reaches up to 5× the throughput of the raw LLM space. It delivers 2.9–10× higher throughput than rpLSH, ScaNN and PQfs at comparable accuracy, and it outperforms the learning-based CSR.
  • Diversity matters: Plain VDeH collapses on FiQA2018 (LLM2Vec), scoring 25.84 nDCG@10 versus 52.58 for IKE. The variant IKE_VD, which samples a small random subset of m dimensions per partition, closes this gap.
  • Adjustable code length without retraining: Because trees are independent, they can be dropped to shorten codes. On Istella22, going from 4K to 3K trees cut latency from 0.42 s to 0.34 s and memory by 25%, while nDCG@10 fell by only 0.36 points. The parameter ψ has little effect on accuracy.
  • Evaluation and limitations: Experiments used datasets including HotpotQA, Istella22 and FiQA2018 with LLM2Vec and Qwen3 embeddings. The method has not been shown to work for cross-modal (image–text) retrieval.

Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.

 Institutions

🎓Nanjing UniversityNanjing · China99 works · 8,512 citations

 Fields, topics and concepts

📚 Shelves 5The field of science this work belongs to (OpenAlex fields).Computer Science · 18,614 worksArtificial Intelligence · 17,965 worksArts and Humanities · 17,450 worksLanguage and Linguistics · 17,296 worksComputer Vision and Pattern Recognition · 3,486 works

🧭 Topics 2Research subjects and languages the work is about.Language models & pretraining · 3,894 worksInformation retrieval & search · 552 works

Cite

Cite as Zhibo Zhang, Yang Xu, Kai Ming Ting and Cam-Tu Nguyen. 2026. LLMs Meet Isolation Kernel: Lightweight, Learning-free Binary Embeddings for Fast Retrieval. Findings of the Association for Computational Linguistics: ACL 2026.

The paper itself, not this page. BibTeX from the library's record of the work. DOI 10.18653/v1/2026.findings-acl.666

@inproceedings{zhang2026llms,
  title = {{LLMs Meet Isolation Kernel: Lightweight, Learning-free Binary Embeddings for Fast Retrieval}},
  author = {Zhibo Zhang, Yang Xu, Kai Ming Ting and Cam-Tu Nguyen},
  booktitle = {Findings of the Association for Computational Linguistics: ACL 2026},
  year = {2026},
  doi = {10.18653/v1/2026.findings-acl.666},
  url = {https://doi.org/10.18653/v1/2026.findings-acl.666}
}

References

61 references, 16 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

  1. Fabien André, Anne-Marie Kermarrec, and Nicolas Le Scouarnec. 2015. Cache locality is not enough: high-performance nearest neighbor search with product quantization fast scan. Proc. VLDB Endow., 9(4):288–299. find
  2. Ilias Azizi, Karima Echihabi, and Themis Palpanas. 2025. Graph-based vector search: An experimental evaluation of the state-of-the-art. Proc. ACM Manag. Data, 3(1). find
  3. Parishad BehnamGhader, Vaibhav Adlakha, Marius Mosbach, Dzmitry Bahdanau, Nicolas Chapados, and Siva Reddy. 2024. LLM2Vec: Large language models are secretly powerful text encoders. In COLM.
  4. Leo Breiman. 2001. Random forests. Machine learning, 45(1):5–32. find
  5. Deng Cai. 2016. A revisit of hashing algorithms for approximate nearest neighbor search. IEEE TKDE, PP. find
  6. Deng Cai. 2019. A revisit of hashing algorithms for approximate nearest neighbor search. IEEE TKDE, 33(6):2337–2348. find
  7. Moses S. Charikar. 2002. Similarity estimation techniques from rounding algorithms. In STOC, page 380–388, New York, NY, USA.
  8. Jeff Johnson, Matthijs Douze, and Hervé Jégou. 2019. Billion-scale similarity search with GPUs. IEEE TBD, 7(3):535–547.
  9. Nick Craswell. 2009. Mean reciprocal rank. In Encyclopedia of Database Systems. find
  10. Herve Jégou, Matthijs Douze, and Cordelia Schmid. 2011. Product quantization for nearest neighbor search. IEEE Trans. Pattern Anal. Mach. Intell., 33(1):117–128. find
  11. Nick Craswell, Bhaskar Mitra, Emine Yilmaz, Hossein A. Rahmani, Daniel Campos, Jimmy Lin, Ellen M. Voorhees, and Ian Soboroff. 2023. Overview of the trec 2023 deep learning track. In TREC. find
  12. Domenico Dato, Sean MacAvaney, Franco Maria Nardini, Raffaele Perego, and Nicola Tonellotto. 2022. The istella22 dataset: Bridging traditional and neural learning to rank evaluation. In SIGIR, SIGIR ’22, page 3099–3107, New York, NY, USA. find
  13. Jacob Devlin, Ming-Wei Chang, Kenton Lee, and Kristina Toutanova. 2019. BERT: Pre-training of deep bidirectional transformers for language understanding. In Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies,…
  14. Matthijs Douze, Alexandr Guzhva, Chengqi Deng, Jeff Johnson, Gergely Szilvasy, Pierre-Emmanuel Mazaré, Maria Lomeli, Lucas Hosseini, and Hervé Jégou. 2025. The faiss library. IEEE TBD, pages 1–17.
  15. Kenneth Enevoldsen, Isaac Chung, Imene Kerboua, Márton Kardos, Ashwin Mathur, David Stap, Jay Gala, Wissam Siblini, Dominik Krzemiński, Genta Indra Winata, Saba Sturua, Saiteja Utpala, Mathieu Ciancone, Marion Schaeffer, Gabriel Sequeira, Diganta Misra, Shreeya Dhakal, Jonathan Rystrøm, Roman Solom… — MMTEB: Massive Multilingual Text Embedding Benchmark
  16. Yunchao Gong, Sanjiv Kumar, Henry A Rowley, and Svetlana Lazebnik. 2013a. Learning binary codes for high-dimensional data using bilinear projections. In CVPR, pages 484–491. find
  17. Yunchao Gong, Svetlana Lazebnik, Albert Gordo, and Florent Perronnin. 2013b. Iterative quantization: A procrustean approach to learning binary codes for large-scale image retrieval. IEEE Trans. Pattern Anal. Mach. Intell., 35:2916–29. find
  18. Ruiqi Guo, Philip Sun, Erik Lindgren, Quan Gu, David Simcha, Felix Chern, and Sanjiv Kumar. 2020. Accelerating large-scale inference with anisotropic vector quantization. In ICML, pages 3887–3896. find
  19. Kalervo Järvelin and Jaana Kekäläinen. 2002. Cumulated gain-based evaluation of IR techniques. ACM Trans. Inf. Syst., 20(4):422–446. find
  20. Zhongming Jin, Cheng Li, Yue Lin, and Deng Cai. 2013. Density sensitive hashing. IEEE Trans. Cybern., 44(8):1362–1371. find
  21. Aditya Kusupati, Gantavya Bhatt, Aniket Rege, Matthew Wallingford, Aditya Sinha, Vivek Ramanujan, William Howard-Snyder, Kaifeng Chen, Sham Kakade, Prateek Jain, and 1 others. 2022. Matryoshka representation learning. NeurIPS, 35:30233–30249. find
  22. Yinhe Lan, Zhenyu Weng, and Yuesheng Zhu. 2017. A new spherical hashing method in a low-dimensional isotropic space. In VCIP, pages 1–4. find
  23. Jinhyuk Lee, Zhuyun Dai, Xiaoqi Ren, Blair Chen, Daniel Cer, Jeremy R Cole, Kai Hui, Michael Boratko, Rajvi Kapadia, Wen Ding, and 1 others. 2024. Gecko: Versatile text embeddings distilled from large language models. arXiv preprint arXiv:2403.20327.
  24. Jinfeng Li, Xiao Yan, Jian Zhang, An Xu, James Cheng, Jie Liu, Kelvin K. W. Ng, and Ti-chung Cheng. 2018. A general and efficient querying method for learning to hash. In SIGMOD, page 1333–1347, New York, NY, USA. find
  25. Zehan Li, Xin Zhang, Yanzhao Zhang, Dingkun Long, Pengjun Xie, and Meishan Zhang. 2023. Towards general text embeddings with multi-stage contrastive learning. arXiv preprint arXiv:2308.03281.
  26. Victor Weixin Liang, Yuhui Zhang, Yongchan Kwon, Serena Yeung, and James Y Zou. 2022. Mind the gap: Understanding the modality gap in multi-modal contrastive representation learning. NeurIPS, 35:17612– 17625. find
  27. Jimmy Lin. 2025. Operational advice for dense and sparse retrievers: HNSW, flat, or inverted indexes? In Proceedings of the 63rd Annual Meeting of the Association for Computational Linguistics (Volume 6: Industry Track), pages 865–872, Vienna, Austria. Association for Computational Linguistics. find
  28. Fei Tony Liu, Kai Ming Ting, and Zhi-Hua Zhou. 2008. Isolation forest. In IEEE ICDM, pages 413–422. find
  29. Huawen Liu, Wenhua Zhou, Zongda Wu, Shichao Zhang, Gang Li, and Xuelong Li. 2024a. Refining codes for locality sensitive hashing. IEEE TKDE, 36(3):1274–1284. find
  30. Yinhan Liu, Myle Ott, Naman Goyal, Jingfei Du, Mandar Joshi, Danqi Chen, Omer Levy, Mike Lewis, Luke Zettlemoyer, and Veselin Stoyanov. 2019. Roberta: A robustly optimized bert pretraining approach. Preprint, arXiv:1907.11692.
  31. Zheng Liu, Chaofan Li, Shitao Xiao, Yingxia Shao, and Defu Lian. 2024b. Llama2Vec: Unsupervised adaptation of large language models for dense retrieval. In Proceedings of the 62nd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 3490–3500, Bangkok, Thail… find
  32. Yu A Malkov and Dmitry A Yashunin. 2018. Efficient and robust approximate nearest neighbor search using hierarchical navigable small world graphs. IEEE Trans. Pattern Anal. Mach. Intell., 42(4):824–836.
  33. Niklas Muennighoff, Nouamane Tazi, Loic Magne, and Nils Reimers. 2023. MTEB: Massive text embedding benchmark. In Proceedings of the 17th Conference of the European Chapter of the Association for Computational Linguistics, pages 2014–2037, Dubrovnik, Croatia. Association for Computational Linguistic…
  34. Zhijie Nie, Zhangchi Feng, Mingxin Li, Cunwang Zhang, Yanzhao Zhang, Dingkun Long, and Richong Zhang. 2024. When text embedding meets large language model: A comprehensive survey. CoRR, abs/2412.09165. — [arXiv:2412.09165](https://arxiv.org/abs/2412.09165) find
  35. Zach Nussbaum, John X. Morris, Brandon Duderstadt, and Andriy Mulyar. 2025. Nomic embed: Training a reproducible long context text embedder. TMLR.
  36. OpenAI. 2024a. New embedding models and api updates. find
  37. OpenAI. 2024b. New embedding models and api updates. https://openai.com/index/ new-embedding-models-and-api-updates. ↗
  38. Yun Peng, Byron Choi, Tsz Nam Chan, Jianye Yang, and Jianliang Xu. 2023. Efficient approximate nearest neighbor search in multi-dimensional databases. Proc. ACM Manag. Data, 1(1). find
  39. Zexuan Qiu, Qinliang Su, Jianxing Yu, and Shijing Si. 2022. Efficient document retrieval by end-to-end refining and quantizing BERT embedding with contrastive product quantization. In Proceedings of the 2022 Conference on Empirical Methods in Natural Language Processing, pages 853–863, Abu Dhabi, Un… find
  40. Colin Raffel, Noam Shazeer, Adam Roberts, Katherine Lee, Sharan Narang, Michael Matena, Yanqi Zhou, Wei Li, and Peter J. Liu. 2020. Exploring the limits of transfer learning with a unified text-to-text transformer. JMLR, 21(140):1–67.
  41. Suhas Jayaram Subramanya, Devvrit, Rohan Kadekodi, Ravishankar Krishaswamy, and Harsha Vardhan Simhadri. 2019. DiskANN: fast accurate billion-point nearest neighbor search on a single node. Curran Associates Inc., Red Hook, NY, USA. find
  42. Nandan Thakur, Luiz Bonifacio, Maik Fröbe, Alexander Bondarenko, Ehsan Kamalloo, Martin Potthast, Matthias Hagen, and Jimmy Lin. 2024. Systematic evaluation of neural retrieval models on the Touché 2020 argument retrieval subset of BEIR. In SIGIR. find
  43. Nandan Thakur, Nils Reimers, Andreas Rücklé, Abhishek Srivastava, and Iryna Gurevych. 2021. BEIR: A heterogeneous benchmark for zero-shot evaluation of information retrieval models. In NeurIPS. find
  44. James Thorne, Andreas Vlachos, Christos Christodoulopoulos, and Arpit Mittal. 2018. FEVER: a large-scale dataset for fact extraction and VERification. In Proceedings of the 2018 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Vo…
  45. Kai Ming Ting, Takashi Washio, Ye Zhu, Yang Xu, and Kaifeng Zhang. 2024. Is it possible to find the single nearest neighbor of a query in high dimensions? AIJ, 336:104206. find
  46. Kai Ming Ting, Yue Zhu, and Zhi-Hua Zhou. 2018. Isolation kernel and its effect on svm. In SIGKDD, pages 2329–2337. find
  47. L. Van Der Maaten, E. Postma, J. Vanden Herik, and et al. 2009. Dimensionality reduction: a comparative. JMLR, 10:66–71. find
  48. Mengzhao Wang, Weizhi Xu, Xiaomeng Yi, Songlin Wu, Zhangyang Peng, Xiangyu Ke, Yunjun Gao, Xiaoliang Xu, Rentong Guo, and Charles Xie. 2024. Starling: An i/o-efficient disk-resident graph index framework for high-dimensional vector similarity search on data segment. Proc. ACM Manag. Data, 2(1). find
  49. Jiuqi Wei, Xiaodong Lee, Zhenyu Liao, Themis Palpanas, and Botao Peng. 2025. Subspace collision: an efficient and accurate framework for high-dimensional approximate nearest neighbor search. Proc. ACM Manag. Data, 3(1):1–29. find
  50. Yair Weiss, Antonio Torralba, and Rob Fergus. 2008. Spectral hashing. In NeurIPS, volume 21. find
  51. Tiansheng Wen, Yifei Wang, Zequn Zeng, Zhong Peng, Yudi Su, Xinyang Liu, Bo Chen, Hongwei Liu, Stefanie Jegelka, and Chenyu You. 2025. Beyond matryoshka: Revisiting sparse coding for adaptive representation. In ICML. find
  52. Yan Xia, Kaiming He, Pushmeet Kohli, and Jian Sun. 2015. Sparse projections for high-dimensional binary codes. In CVPR, pages 3332–3339. find
  53. Yang Xu and Kai Ming Ting. 2025. Voronoi diagram encoded hashing. In ECML PKDD, page 87–103, Berlin, Heidelberg. find
  54. Zhilin Yang, Peng Qi, Saizheng Zhang, Yoshua Bengio, William Cohen, Ruslan Salakhutdinov, and Christopher D. Manning. 2018. HotpotQA: A dataset for diverse, explainable multi-hop question answering.
  55. In Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing, pages 2369–2380, Brussels, Belgium. Association for Computational Linguistics. find
  56. Felix Yu, Sanjiv Kumar, Yunchao Gong, and Shih-Fu Chang. 2014. Circulant binary embedding. In ICML, volume 32 of Proceedings of Machine Learning Research, pages 946–954, Bejing, China. find
  57. Peng Yu, Luke Merrick, Giovanni Nuti, and Daniel Campos. 2024. Arctic-embed 2.0: Multilingual retrieval without compromise. arXiv preprint arXiv:2412.04506. — [arXiv:2412.04506](https://arxiv.org/abs/2412.04506) find
  58. Hailin Zhang, Xiaodong Ji, Yilin Chen, Fangcheng Fu, Xupeng Miao, Xiaonan Nie, Weipeng Chen, and Bin Cui. 2025a. Pqcache: Product quantization-based kvcache for long context llm inference. Proc. ACM Manag. Data, 3(3). find
  59. Yanzhao Zhang, Mingxin Li, Dingkun Long, Xin Zhang, Huan Lin, Baosong Yang, Pengjun Xie, An Yang, Dayiheng Liu, Junyang Lin, Fei Huang, and Jingren Zhou. 2025b. Qwen3 embedding: Advancing text embedding and reranking through foundation models. arXiv preprint arXiv:2506.05176.
  60. Guang-Tong Zhou, Kai Ming Ting, Fei Tony Liu, and Yilong Yin. 2012. Relevance feature mapping for content-based multimedia information retrieval. Pattern Recogn., 45(4):1707–1720. find
  61. Xiaofeng Zhu, Lei Zhang, and Zi Huang. 2014. A sparse embedding and least variance encoding approach to hashing. IEEE Trans. Image Process., 23(9):3737–3750. find

Updated

↑ back to top

This page in Markdown → https://md.research.elamur.ai/free/en/papers/zhang-2026-llms-meet-isolation-kernel-lightweight/