Elamur Research⌕
☰
home / papers / 2026 / Do LLMs Really Memorize Personally Identifiable Information? Revisiting PII Leakage with a Cue-Controlled Memorization Framework

Do LLMs Really Memorize Personally Identifiable Information? Revisiting PII Leakage with a Cue-Controlled Memorization Framework

Paper · 2026 · Xiaoyu Luo, Yiyi Chen, Qiongxiu Li et al.

Summary

Earlier studies often take an LLM's ability to reproduce personally identifiable information (PII) as proof that the model memorized it. This paper challenges that assumption. Many apparent "leaks" may come from the prompt itself: names, formatting conventions and character overlap can let a model guess an email address without having stored it. The authors propose Cue-Resistant Memorization (CRM), an evaluation framework that conditions memorization metrics on prompt–target overlap. Under CRM, success in low-cue conditions is a necessary requirement before memorization can be claimed. They apply it in a multilingual study across 32 languages and several mGPT models. The study covers verbatim prefix–suffix completion, associative reconstruction from an owner's name, cue-free generation and membership inference. Once overlap cues are controlled, reconstruction success falls to near zero. Log-likelihood follows cue strength equally for training and unseen samples. Free generation produces mostly generic addresses, and membership inference stays near chance. The findings suggest PII privacy risk in LLMs has been overestimated, and CRM offers a stricter protocol for privacy audits and for mitigation work such as model editing.

Key points

  • CRM is a cue-controlled framework that separates genuine memorization from pattern completion driven by surface cues. It is evaluated on mGPT3-13B, mGPT3-1.3B and mGPT2-560M across 32 languages.
  • The study distinguishes three paradigms: verbatim, associative (e.g., name → email) and extractable (generic requests with no target context). Reconstruction successes occur almost only under high-cue conditions, and the effect is strongest for Latin-script languages.
  • In cue-free generation, true positive rates for emails and phones range from roughly 0.23% to 0.77%. None of these hits overlap with PII recovered in the verbatim or associative settings.
  • Eight membership inference attacks (including loss/perplexity, Zlib, reference-model, neighborhood and Min-K%/Min-K%++) reach AUROC of only 0.50–0.60. English averages 0.525, and differences between languages are small.
  • The code is published on GitHub (xiaoyuluoit97/mllm_pii_mia). The paper appears in the ACL 2026 proceedings and comes from Aalborg University.

Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.

 Institutions

🎓Aalborg UniversityAalborg · Denmark12 works · 743 citations

 Fields, topics and concepts

📚 Shelves 4The field of science this work belongs to (OpenAlex fields).Computer Science · 16,667 worksArtificial Intelligence · 16,194 worksSocial Sciences · 1,772 worksSociology and Political Science · 577 works

🧭 Topics 1Research subjects and languages the work is about.Language models & pretraining · 3,320 works

Cite

Cite as Xiaoyu Luo, Yiyi Chen, Qiongxiu Li and Johannes Bjerva. 2026. Do LLMs Really Memorize Personally Identifiable Information? Revisiting PII Leakage with a Cue-Controlled Memorization Framework. Proceedings of the 64th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers).

The paper itself, not this page. BibTeX from the library's record of the work. DOI 10.18653/v1/2026.acl-long.1560

@inproceedings{luo2026do,
  title = {{Do LLMs Really Memorize Personally Identifiable Information? Revisiting PII Leakage with a Cue-Controlled Memorization Framework}},
  author = {Xiaoyu Luo, Yiyi Chen, Qiongxiu Li and Johannes Bjerva},
  booktitle = {Proceedings of the 64th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers)},
  year = {2026},
  doi = {10.18653/v1/2026.acl-long.1560},
  url = {https://doi.org/10.18653/v1/2026.acl-long.1560}
}

References

39 references, 13 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

  1. Nicholas Carlini, Steve Chien, Milad Nasr, Shuang Song, Andreas Terzis, and Florian Tramer. 2022a. Membership inference attacks from first principles. In 2022 IEEE symposium on security and privacy (SP), pages 1897–1914. IEEE. find
  2. Nicholas Carlini, Daphne Ippolito, Matthew Jagielski, Katherine Lee, Florian Tramer, and Chiyuan Zhang. 2022b. Quantifying memorization across neural language models. In The Eleventh International Conference on Learning Representations. find
  3. Nicholas Carlini, Chang Liu, Úlfar Erlingsson, Jernej Kos, and Dawn Song. 2019. The secret sharer: Evaluating and testing unintended memorization in neural networks. In 28th USENIX security symposium (USENIX security 19), pages 267–284.
  4. Nicholas Carlini, Florian Tramer, Eric Wallace, Matthew Jagielski, Ariel Herbert-Voss, Katherine Lee, Adam Roberts, Tom Brown, Dawn Song, Ulfar Erlingsson, and 1 others. 2021. Extracting training data from large language models. In 30th USENIX security symposium (USENIX Security 21), pages 2633–2650…
  5. Kent Chang, Mackenzie Cramer, Sandeep Soni, and David Bamman. 2023. Speak, memory: An archaeology of books known to chatgpt/gpt-4. In Proceedings of the 2023 Conference on Empirical Methods in Natural Language Processing, pages 7312–7327.
  6. Yiyi Chen, Russa Biswas, Heather Lent, and Johannes Bjerva. 2025. Against all odds: Overcoming typology, script, and language confusion in multilingual embedding inversion attacks. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 39, pages 23632–23641. find
  7. Michael Duan, Anshuman Suri, Niloofar Mireshghallah, Sewon Min, Weijia Shi, Luke Zettlemoyer, Yulia Tsvetkov, Yejin Choi, David Evans, and Hannaneh Hajishirzi. 2024. Do membership inference attacks work on large language models? In Conference on Language Modeling (COLM).
  8. Leo Gao, Stella Biderman, Sid Black, Laurence Golding, Travis Hoppe, Charles Foster, Jason Phang, Horace He, Anish Thite, Noa Nabeshima, and 1 others. 2020. The pile: An 800gb dataset of diverse text for language modeling. arXiv preprint arXiv:2101.00027. find
  9. Xiaoyu Luo, Yiyi Chen, Johannes Bjerva, and Qiongxiu Li. 2025. Shared path: Unraveling memorization in multilingual llms through language similarities. In Proceedings of the 2025 Conference on Empirical Methods in Natural Language Processing (EMNLP), Suzhou, China. find
  10. Jamie Hayes, Ilia Shumailov, Christopher A Choquette-Choo, Matthew Jagielski, George Kaissis, Katherine Lee, Milad Nasr, Sahra Ghalebikesabi, Niloofar Mireshghallah, Meenatchi Sundaram Mutu Selva Annamalai, and 1 others. 2025. Strong membership inference attacks on massive datasets and (moderately)… find
  11. Justus Mattern, Fatemehsadat Mireshghallah, Zhijing Jin, Bernhard Schoelkopf, Mrinmaya Sachan, and Taylor Berg-Kirkpatrick. 2023. Membership inference attacks against language models via neighbourhood comparison. In Findings of the Association for Computational Linguistics: ACL 2023, Toronto, Canada…
  12. Jie Huang, Hanyin Shao, and Kevin Chen-Chuan Chang. 2022. Are large pre-trained language models leaking your personal information? arXiv preprint arXiv:2205.12628. find
  13. Abhyuday Jagannatha, Bhanu Pratap Singh Rawat, and Hong Yu. 2021. Membership inference attack susceptibility of clinical language models. arXiv preprint arXiv:2104.08305. — [arXiv:2104.08305](https://arxiv.org/abs/2104.08305) find
  14. Antonia Karamolegkou, Jiaang Li, Li Zhou, and Anders Søgaard. 2023. Copyright violations and large language models. In Proceedings of the 2023 Conference on Empirical Methods in Natural Language Processing, pages 7403–7412.
  15. Siwon Kim, Sangdoo Yun, Hwaran Lee, Martin Gubri, Sungroh Yoon, and Seong Joon Oh. 2023. Propile: Probing privacy leakage in large language models. Advances in Neural Information Processing Systems, 36:20750–20762. find
  16. Bryan Klimt and Yiming Yang. 2004. The enron corpus: A new dataset for email classification research. In European conference on machine learning, pages 217–226. Springer. find
  17. Katherine Lee, Daphne Ippolito, Andrew Nystrom, Chiyuan Zhang, Douglas Eck, Chris Callison-Burch, and Nicholas Carlini. 2021. Deduplicating training data makes language models better. arXiv preprint arXiv:2107.06499. find
  18. Bo Li, Qinghua Zhao, and Lijie Wen. 2024. Rome: Memorization insights from text, logits and representation. arXiv preprint arXiv:2403.00510. — [arXiv:2403.00510](https://arxiv.org/abs/2403.00510) find
  19. Qiongxiu Li, Xiaoyu Luo, Yiyi Chen, and Johannes Bjerva. 2025. Trustworthy machine learning via memorization and the granular long-tail: A survey on interactions, tradeoffs, and beyond. arXiv preprint arXiv:2503.07501. — [arXiv:2503.07501](https://arxiv.org/abs/2503.07501) find
  20. Nils Lukas, Ahmed Salem, Robert Sim, Shruti Tople, Lukas Wutschitz, and Santiago Zanella-Béguelin. 2023. Analyzing leakage of personally identifiable information in language models. In 2023 IEEE Symposium on Security and Privacy (SP), pages 346–363. IEEE. find
  21. Milad Nasr, Nicholas Carlini, Jonathan Hayase, Matthew Jagielski, A Feder Cooper, Daphne Ippolito, Christopher A Choquette-Choo, Eric Wallace, Florian Tramèr, and Katherine Lee. 2023. Scalable extraction of training data from (production) language models. arXiv preprint arXiv:2311.17035.
  22. Esther Ploeger, Wessel Poelman, Miryam de Lhoneux, and Johannes Bjerva. 2024. What is “typological diversity” in NLP? In Proceedings of the 2024 Conference on Empirical Methods in Natural Language Processing, pages 5681–5700, Miami, Florida, USA. Association for Computational Linguistics. find
  23. Esther Ploeger, Wessel Poelman, Andreas Holck Høeg-Petersen, Anders Schlichtkrull, Miryam de Lhoneux, and Johannes Bjerva. 2025. A principled framework for evaluating on typologically diverse languages. Computational Linguistics, pages 1–36.
  24. QwenTeam. 2025. Qwen3 technical report. Preprint, arXiv:2505.09388.
  25. Colin Raffel, Noam Shazeer, Adam Roberts, Katherine Lee, Sharan Narang, Michael Matena, Yanqi Zhou, Wei Li, and Peter J Liu. 2020. Exploring the limits of transfer learning with a unified text-to-text transformer. Journal of machine learning research, 21(140):1–67.
  26. Elena Sofia Ruzzetti, Giancarlo A Xompero, Davide Venditti, and Fabio Massimo Zanzotto. 2025. Private memorization editing: Turning memorization into a defense to strengthen data privacy in large language models. arXiv preprint arXiv:2506.10024. find
  27. Tom Sander, Bargav Jayaraman, Mark Ibrahim, Kamalika Chaudhuri, and Chuan Guo. 2025. Rethinking the role of verbatim memorization in llm privacy. In The Thirty-ninth Annual Conference on Neural Information Processing Systems. find
  28. Ali Satvaty, Anna Visman, Dan Seidel, Suzan Verberne, and Fatih Turkmen. 2025. Memorization is language-sensitive: Analyzing memorization and inference risks of llms in a multilingual setting. In Proceedings of the First Workshop on Large Language Model Memorization (L2M2), pages 106–126. find
  29. Weijia Shi, Anirudh Ajith, Mengzhou Xia, Yangsibo Huang, Daogao Liu, Terra Blevins, Danqi Chen, and Luke Zettlemoyer. 2023. Detecting pretraining data from large language models. Preprint, arXiv:2310.16789. find
  30. Oleh Shliazhko, Alena Fenogenova, Maria Tikhonova, Anastasia Kozlova, Vladislav Mikhailov, and Tatiana Shavrina. 2024. mgpt: Few-shot learners go multilingual. Transactions of the Association for Computational Linguistics, 12:58–79.
  31. Reza Shokri, Marco Stronati, Congzheng Song, and Vitaly Shmatikov. 2017. Membership inference attacks against machine learning models. In 2017 IEEE symposium on security and privacy (SP), pages 3–18. IEEE.
  32. Hedvig Skirgård, Hannah J Haynie, Damián E Blasi, Harald Hammarström, Jeremy Collins, Jay J Latarche, Jakob Lesage, Tobias Weber, Alena Witzlack-Makarevich, Sam Passmore, and 1 others. 2023. Grambank reveals the importance of genealogical constraints on linguistic diversity and highlights the impact… find
  33. Alisha Srivastava, Emir Korukluoglu, Minh Nhat Le, Duyen Tran, Chau Minh Pham, Marzena Karpinska, and Mohit Iyyer. 2025. Owl: Probing cross-lingual recall of memorized texts via world literature. arXiv preprint arXiv:2505.22945. find
  34. Zhixing Tan, Xiangwen Zhang, Shuo Wang, and Yang Liu. 2021. Msp: Multi-stage prompting for making pre-trained language models better translators. arXiv preprint arXiv:2110.06609. find
  35. Davide Venditti, Elena Sofia Ruzzetti, Giancarlo A Xompero, Cristina Giannone, Andrea Favalli, Raniero Romagnoli, and Fabio Massimo Zanzotto. 2024. Enhancing data privacy in large language models through private association editing. arXiv preprint arXiv:2406.18221. — [arXiv:2406.18221](https://arxiv.org/abs/2406.18221) find
  36. Ben Wang and Aran Komatsuzaki. 2021. GPT-J-6B: A 6 Billion Parameter Autoregressive Language Model. https://github.com/kingoflolz/ mesh-transformer-jax. ↗
  37. Chengkun Wei, Minghu Zhao, Zhikun Zhang, Min Chen, Wenlong Meng, Bo Liu, Yuan Fan, and Wenzhi Chen. 2023. Dpmlbench: Holistic evaluation of differentially private machine learning. In Proceedings of the 2023 ACM SIGSAC Conference on Computer and Communications Security, pages 2621–2635. find
  38. BigScience Workshop, Teven Le Scao, Angela Fan, Christopher Akiki, Ellie Pavlick, Suzana Ilić, Daniel Hesslow, Roman Castagné, Alexandra Sasha Luccioni, François Yvon, and 1 others. 2022. Bloom: A 176b-parameter open-access multilingual language model. arXiv preprint arXiv:2211.05100. find
  39. Linting Xue, Noah Constant, Adam Roberts, Mihir Kale, Rami Al-Rfou, Aditya Siddhant, Aditya Barua, and Colin Raffel. 2020. mt5: A massively multilingual pre-trained text-to-text transformer. arXiv preprint arXiv:2010.11934.

Updated

↑ back to top

This page in Markdown → https://md.research.elamur.ai/free/en/papers/luo-2026-do-llms-really-memorize-personally/