Elamur Research⌕
☰
home / papers / 2023 / Text Embeddings Reveal (Almost) As Much As Text

Text Embeddings Reveal (Almost) As Much As Text

Paper · 2023 · John Morris, Volodymyr Kuleshov, Vitaly Shmatikov et al.

Summary

The paper asks whether dense text embeddings, such as those stored in hosted vector databases, can be turned back into the original text. The authors treat inversion as controlled generation: an attacker who can only query a black-box encoder searches for text whose embedding matches a target vector. Their method, Vec2Text, starts from an initial guess produced by a conditional T5-style encoder-decoder. It then repeatedly re-embeds that guess and feeds the target embedding, the guess's embedding and their difference back into a correction model, using sequence-level beam search over successive edits. The models invert GTR-base, trained on 5M Wikipedia passages from Natural Questions, and OpenAI text-embeddings-ada-002, trained on MSMARCO. Evaluation covers in-domain data, BEIR and MIMIC-III clinical notes. Iterative correction far outperforms one-shot decoding and recovers most short passages verbatim. Because personal details such as patient names can be extracted, the authors argue that embeddings deserve the same privacy safeguards as the raw text they encode.

Key points

  • On 32-token Natural Questions passages embedded with GTR-base, 50 correction steps plus sequence beam search reach BLEU 97.3 and 92% exact recovery. The uncorrected base model reaches BLEU 31.9 and 0% exact matches.
  • For text-embeddings-ada-002 on MSMARCO, exact recovery is 60.9% for 32-token inputs. For 128-token inputs it falls to 8.0%, with BLEU 55.0.
  • On MIMIC-III clinical notes, the method recovers 89% of full names.
  • Earlier baselines perform far worse, including the bag-of-words inversion of Song and Raghunathan (2020) at BLEU 0.3. Results keep improving with more search rounds and wider beams (tested up to 50 rounds and beam width 8).
  • Code is released as vec2text on GitHub. Embedding 5M documents through the OpenAI API cost about $35 at 2023 prices, which makes such attacks inexpensive.

Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.

 Institutions

🎓Cornell UniversityIthaca · United States89 works · 58,422 citations

📚 Shelves 4 Computer Science · 9,086 works Artificial Intelligence · 8,889 works Arts and Humanities · 9,296 works Language and Linguistics · 9,206 works

Cite

Cite as John Morris, Volodymyr Kuleshov, Vitaly Shmatikov and Alexander Rush. 2023. Text Embeddings Reveal (Almost) As Much As Text. Proceedings of the 2023 Conference on Empirical Methods in Natural Language Processing.

The paper itself, not this page. BibTeX from the library's record of the work. DOI 10.18653/v1/2023.emnlp-main.765

@inproceedings{morris2023text,
  title = {{Text Embeddings Reveal (Almost) As Much As Text}},
  author = {John Morris, Volodymyr Kuleshov, Vitaly Shmatikov and Alexander Rush},
  booktitle = {Proceedings of the 2023 Conference on Empirical Methods in Natural Language Processing},
  year = {2023},
  doi = {10.18653/v1/2023.emnlp-main.765},
  url = {https://doi.org/10.18653/v1/2023.emnlp-main.765}
}

References

56 references, 24 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

  1. Mohamed Abdalla, Moustafa Abdalla, Graeme Hirst, and Frank Rudzicz. 2020. Exploring the privacy-preserving properties of word embeddings: Algorithmic validation study. J Med Internet Res, 22(7):e18055. find
  2. Leonard Adolphs, Michelle Chen Huebscher, Christian Buck, Sertan Girgin, Olivier Bachem, Massimiliano Ciaramita, and Thomas Hofmann. 2022. Decoding a neural retriever’s latent space for query suggestion. find
  3. Charu C. Aggarwal and ChengXiang Zhai. 2012. A Survey of Text Clustering Algorithms, pages 77–128. Springer US, Boston, MA. find
  4. Payal Bajaj, Daniel Campos, Nick Craswell, Li Deng, Jianfeng Gao, Xiaodong Liu, Rangan Majumder, Andrew McNamara, Bhaskar Mitra, Tri Nguyen, Mir Rosenberg, Xia Song, Alina Stoica, Saurabh Tiwary, and Tong Wang. 2018. Ms marco: A human generated machine reading comprehension dataset. our card
  5. Florian Bordes, Randall Balestriero, and Pascal Vincent. 2021. High fidelity visualization of what your self-supervised representation knows about. find
  6. Trans. Mach. Learn. Res., 2022. find
  7. Sebastian Borgeaud, Arthur Mensch, Jordan Hoffmann, Trevor Cai, Eliza Rutherford, Katie Millican, George van den Driessche, Jean-Baptiste Lespiau, Bogdan Damoc, Aidan Clark, Diego de Las Casas, Aurelia Guy, Jacob Menick, Roman Ring, Tom Hennigan, Saffron Huang, Loren Maggiore, Chris Jones, Albin Cas… find
  8. Samuel R. Bowman, Luke Vilnis, Oriol Vinyals, Andrew M. Dai, Rafal Jozefowicz, and Samy Bengio. 2016. Generating sentences from a continuous space. our card
  9. Sumanth Dathathri, Andrea Madotto, Janice Lan, Jane Hung, Eric Frank, Piero Molino, Jason Yosinski, and Rosanne Liu. 2020. Plug and play language models: A simple approach to controlled text generation. In International Conference on Learning Representations. our card
  10. Alexey Dosovitskiy and Thomas Brox. 2016. Inverting visual representations with convolutional networks. find
  11. Chi Nhan Duong, Thanh-Dat Truong, Kha Gia Quach, Hung Bui, Kaushik Roy, and Khoa Luu. 2020. Vec2face: Unveil human faces from their blackbox features in face recognition. find
  12. Adam Dziedzic, Franziska Boenisch, Mingjian Jiang, Haonan Duan, and Nicolas Papernot. 2023. Sentence embedding encoders are easy to steal but hard to defend. In ICLR 2023 Workshop on Pitfalls of limited data and computation for Trustworthy ML. a freely accessible critical care database. Scientific D… find
  13. Vladimir Karpukhin, Barlas Oğuz, Sewon Min, Patrick Lewis, Ledell Wu, Sergey Edunov, Danqi Chen, and Wen tau Yih. 2020. Dense passage retrieval for open-domain question answering. our card
  14. Donggyu Kim, Garam Lee, and Sungwoo Oh. 2022. Toward privacy-preserving text embedding similarity with homomorphic encryption. In Proceedings of the Fourth Workshop on Financial Technology and Natural Language Processing (FinNLP), pages 25–36, Abu Dhabi, United Arab Emirates (Hybrid). Association fo… find
  15. Ryan Kiros, Yukun Zhu, Ruslan Salakhutdinov, Richard S. Zemel, Antonio Torralba, Raquel Urtasun, and Sanja Fidler. 2015. Skip-thought vectors. find
  16. Tom Kwiatkowski, Jennimaria Palomaki, Olivia Redfield, Michael Collins, Ankur Parikh, Chris Alberti, Danielle Epstein, Illia Polosukhin, Jacob Devlin, Kenton Lee, Kristina Toutanova, Llion Jones, Matthew Kelcey, Ming-Wei Chang, Andrew M. Dai, Jakob Uszkoreit, Quoc Le, and Slav Petrov. 2019. Natural… our card
  17. LangChain. 2023. Hwchase17/langchain: building applications with llms through composability. find
  18. Quoc V. Le and Tomas Mikolov. 2014. Distributed representations of sentences and documents. our card
  19. Jason Lee, Elman Mansimov, and Kyunghyun Cho. 2018. Deterministic non-autoregressive neural sequence modeling by iterative refinement. our card
  20. Eric Lehman, Sarthak Jain, Karl Pichotta, Yoav Goldberg, and Byron C. Wallace. 2021. Does bert pre-trained on clinical notes reveal sensitive data? find
  21. Jonas Geiping, Hartmut Bauermeister, Hannah Dröge, and Michael Moeller. 2020. Inverting gradients – how easy is it to break privacy in federated learning? find
  22. Haoran Li, Mingshi Xu, and Yangqiu Song. 2023. Sentence embedding leaks more information than you expect: Generative embedding inversion attack to recover the whole sentence. find
  23. Marjan Ghazvininejad, Omer Levy, Yinhan Liu, and Luke Zettlemoyer. 2019. Mask-predict: Parallel decoding of conditional masked language models. our card
  24. Xiang Lisa Li, John Thickstun, Ishaan Gulrajani, Percy Liang, and Tatsunori B. Hashimoto. 2022. Diffusion-lm improves controllable text generation. our card
  25. Zhiting Hu, Zichao Yang, Xiaodan Liang, Ruslan Salakhutdinov, and Eric P. Xing. 2018. Toward controlled generation of text. our card
  26. Aravindh Mahendran and Andrea Vedaldi. 2014. Understanding deep image representations by inverting them. 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pages 5188–5196. find
  27. Vineet John, Lili Mou, Hareesh Bahuleyan, and Olga Vechtomova. 2018. Disentangled representation learning for non-parallel text style transfer. our card
  28. Alistair E.W. Johnson, Tom J. Pollard, Lu Shen, Liwei H. Lehman, Mengling Feng, Mohammad Ghassemi, Benjamin Moody, Peter Szolovits, Leo Anthony Celi, and Roger G. Mark. 2016. find
  29. Mimic-iii, Luca Melis, Congzheng Song, Emiliano De Cristofaro, and Vitaly Shmatikov. 2018. Exploiting unintended feature leakage in collaborative learning. find
  30. Tomas Mikolov, Kai Chen, Greg Corrado, and Jeffrey Dean. 2013. Efficient estimation of word representations in vector space. our card
  31. Ron Mokady, Amir Hertz, and Amit H. Bermano. 2021. Clipcap: Clip prefix for image captioning. find
  32. John X. Morris. 2020. Second-order nlp adversarial examples. find
  33. Niklas Muennighoff, Nouamane Tazi, Loïc Magne, and Nils Reimers. 2023. Mteb: Massive text embedding benchmark. our card
  34. Arvind Neelakantan, Tao Xu, Raul Puri, Alec Radford, Jesse Michael Han, Jerry Tworek, Qiming Yuan, Nikolas Tezak, Jong Wook Kim, Chris Hallacy, Johannes Heidecke, Pranav Shyam, Boris Power, Tyna Eloundou Nekoul, Girish Sastry, Gretchen Krueger, David Schnurr, Felipe Petroski Such, Kenny Hsu, Madelei… our card
  35. Jianmo Ni, Chen Qu, Jing Lu, Zhuyun Dai, Gustavo Hernández Ábrego, Ji Ma, Vincent Y. Zhao, Yi Luan, Keith B. Hall, Ming-Wei Chang, and Yinfei Yang. 2021. Large dual encoders are generalizable retrievers. our card
  36. Kishore Papineni, Salim Roukos, Todd Ward, and Wei-Jing Zhu. 2002. Bleu: a method for automatic evaluation of machine translation. In Proceedings of the 40th Annual Meeting of the Association for Computational Linguistics, pages 311–318, Philadelphia, Pennsylvania, USA. Association for Computational… our card
  37. Pinecone. 2023. Pinecone. find
  38. Qdrant. 2023. Qdrant - vector database. find
  39. Colin Raffel, Noam Shazeer, Adam Roberts, Katherine Lee, Sharan Narang, Michael Matena, Yanqi Zhou, Wei Li, and Peter J. Liu. 2020. Exploring the limits of transfer learning with a unified text-to-text transformer. our card
  40. Ori Ram, Liat Bezalel, Adi Zicher, Yonatan Belinkov, Jonathan Berant, and Amir Globerson. 2023. What are you token about? dense retrieval as distributions over the vocabulary. find
  41. Shaina Raza, Deepak John Reji, Femi Shajan, and Syed Raza Bashir. 2022. Large-scale application of named entity recognition to biomedicine and epidemiology. PLOS Digital Health, 1(12):e0000152. find
  42. Tao Shen, Xiubo Geng, Chongyang Tao, Can Xu, Xiaolong Huang, Binxing Jiao, Linjun Yang, and Daxin Jiang. 2023. Lexmae: Lexicon-bottlenecked pretraining for large-scale retrieval. find
  43. Congzheng Song and Ananth Raghunathan. 2020. Information leakage in embedding models. Proceedings of the 2020 ACM SIGSAC Conference on Computer and Communications Security. find
  44. Congzheng Song, Alexander M. Rush, and Vitaly Shmatikov. 2020. Adversarial semantic collisions. find
  45. Piotr Teterwak, Chiyuan Zhang, Dilip Krishnan, and Michael C. Mozer. 2021. Understanding invariance via feedforward inversion of discriminatively trained classifiers. find
  46. Nandan Thakur, Nils Reimers, Andreas Rücklé, Abhishek Srivastava, and Iryna Gurevych. 2021. Beir: A heterogenous benchmark for zero-shot evaluation of information retrieval models. our card
  47. Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N. Gomez, Lukasz Kaiser, and Illia Polosukhin. 2017. Attention is all you need. our card
  48. Vdaas. 2023. Vdaas/vald: Vald. a highly scalable distributed vector search engine. find
  49. Liang Wang, Nan Yang, Xiaolong Huang, Binxing Jiao, Linjun Yang, Daxin Jiang, Rangan Majumder, and Furu Wei. 2023. Simlm: Pre-training with representation bottleneck for dense passage retrieval. our card
  50. Weaviate. 2023. Weaviate - vector database. find
  51. Sean Welleck, Ximing Lu, Peter West, Faeze Brahman, Tianxiao Shen, Daniel Khashabi, and Yejin Choi. 2022. Generating sequences by learning to self-correct. our card
  52. Shitao Xiao, Zheng Liu, Yingxia Shao, and Zhao Cao. 2022. Retromae: Pre-training retrieval-oriented language models via masked auto-encoder. our card
  53. Kevin Yang and Dan Klein. 2021. FUDGE: Controlled text generation with future discriminators. In Proceedings of the 2021 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies. Association for Computational Linguistics. our card
  54. Shunyu Yao, Jeffrey Zhao, Dian Yu, Nan Du, Izhak Shafran, Karthik Narasimhan, and Yuan Cao. 2023. React: Synergizing reasoning and acting in language models. our card
  55. Bo Zhao, Konda Reddy Mopuri, and Hakan Bilen. 2020. idlg: Improved deep leakage from gradients. find
  56. Ligeng Zhu, Zhijian Liu, and Song Han. 2019. Deep leakage from gradients. find

Updated

↑ back to top

This page in Markdown → https://md.research.elamur.ai/free/en/papers/morris-2023-text-embeddings-reveal-almost-as/