Elamur Research⌕
☰
home / papers / 2026 / Adaptive Backtracking for Privacy Protection in Large Language Models

Adaptive Backtracking for Privacy Protection in Large Language Models

Paper · 2026 · Zhihao Yao, Yuxuan Gu, Xiachong Feng et al.

Summary

In retrieval-augmented generation, attackers can write prompts that make an LLM reveal private details from the documents it retrieves. Existing defenses either mask the data before generation, which leaves the model with less useful context, or cut out leaked tokens during decoding. Cutting tokens does not change the model's underlying drive to disclose, so it often leaks again later in the response. The authors propose ABack, a training-free decoding method with three steps:

  1. An LLM extracts personally identifiable entities from the retrieved text.
  2. Generation is monitored against that entity list.
  3. When a leak is caught, the method treats the urge to disclose as a hidden mental state and reasons backward to find where that intention first appeared. It then rewinds to that point and regenerates with explicit awareness of which entities are private.

Because formal privacy datasets are scarce, the authors also release PriGenQA. It is a synthetic question-answering benchmark in which each query carries personal attributes. Under adversarial attacks on both open and closed models, ABack lifts the combined privacy-utility score as much as 14% above the best-performing baseline, with no loss in answer quality. This suggests the usual trade-off between privacy and usefulness can be avoided.

Key points

  • Two leak detectors:
    • Hard string matching uses look-ahead: if a token matches the start of a private entity, the next m tokens are generated and checked.
    • Soft semantic matching uses bge-large-en-v1.5 cosine similarity with a threshold δ.
  • Backtracking: The procedure relies on four mental states (from neutral description to disclosed), grounded in Dual-Pathway Behavioral Development Theory. For efficiency, it rewinds a fixed d tokens and processes text in units of l tokens.
  • Evaluated models: Qwen2.5, Gemma-2 and Gemini-2.0. Latency stays comparable to the strongest baseline, and results remain stable across hyperparameter settings.
  • Paraphrase attacks: In tests on Qwen2.5-7B-Instruct, with GPT-5 acting as the inference judge, post-processing defenses collapse. The soft-matching variant of ABack largely contains this kind of leakage.
  • PriGenQA construction: The benchmark uses LLM-generated, randomized personal data (names, ages, occupations, financial and medical details). It was verified by automated and manual checks to contain no real individuals.

Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.

 Institutions

🎓Harbin Institute of TechnologyHarbin · China185 works · 29,140 citations🎓Harbin Engineering UniversityHarbin · China3 works · 9 citations

 Fields, topics and concepts

📚 Shelves 4The field of science this work belongs to (OpenAlex fields).Computer Science · 18,614 worksArtificial Intelligence · 17,965 worksArts and Humanities · 17,450 worksLanguage and Linguistics · 17,296 works

🧭 Topics 1Research subjects and languages the work is about.Language models & pretraining · 3,894 works

Cite

Cite as Zhihao Yao, Yuxuan Gu, Xiachong Feng, Weitao Ma, Bo Li, Xiaocheng Feng and Bing Qin. 2026. Adaptive Backtracking for Privacy Protection in Large Language Models. Findings of the Association for Computational Linguistics: ACL 2026.

The paper itself, not this page. BibTeX from the library's record of the work. DOI 10.18653/v1/2026.findings-acl.1857

@inproceedings{yao2026adaptive,
  title = {{Adaptive Backtracking for Privacy Protection in Large Language Models}},
  author = {Zhihao Yao, Yuxuan Gu, Xiachong Feng, Weitao Ma, Bo Li, Xiaocheng Feng and Bing Qin},
  booktitle = {Findings of the Association for Computational Linguistics: ACL 2026},
  year = {2026},
  doi = {10.18653/v1/2026.findings-acl.1857},
  url = {https://doi.org/10.18653/v1/2026.findings-acl.1857}
}

References

42 references, 12 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

  1. Christopher M Bishop and Nasser M Nasrabadi. 2006. Pattern recognition and machine learning, volume 4. Springer. find
  2. Jaydeep Borkar. 2023. What can we learn from data leakage and unlearning for law? arXiv preprint arXiv:2307.10476. find
  3. Jingxue Chen, Hang Yan, Zhiyuan Liu, Min Zhang, Hu Xiong, and Shui Yu. 2024. When federated learning meets privacy-preserving computation. ACM Computing Surveys, 56(12):1–36. find
  4. Gheorghe Comanici, Eric Bieber, Mike Schaekermann, Ice Pasupat, Noveen Sachdeva, Inderjit Dhillon, Marcel Blistein, Ori Ram, Dan Zhang, Evan Rosen, and 1 others. 2025. Gemini 2.5: Pushing the frontier with advanced reasoning, multimodality, long context, and next generation agentic capabilities. arX…
  5. Lucas BV De Amorim, George DC Cavalcanti, and Rafael MO Cruz. 2023. The choice of scaling technique matters for classification performance. Applied Soft Computing, 133:109924. find
  6. Yunfan Gao, Yun Xiong, Xinyu Gao, Kangxiang Jia, Jinliu Pan, Yuxi Bi, Yixin Dai, Jiawei Sun, Haofen Wang, and Haofen Wang. 2023. Retrieval-augmented generation for large language models: A survey. arXiv preprint arXiv:2312.10997, 2(1).
  7. Lei Huang, Weijiang Yu, Weitao Ma, Weihong Zhong, Zhangyin Feng, Haotian Wang, Qianglong Chen, Weihua Peng, Xiaocheng Feng, Bing Qin, and 1 others. 2025. A survey on hallucination in large language models: Principles, taxonomy, challenges, and open questions. ACM Transactions on Information Systems,…
  8. Yangsibo Huang, Samyak Gupta, Zexuan Zhong, Kai Li, and Danqi Chen. 2023. Privacy implications of retrieval-based language models. arXiv preprint arXiv:2305.14888. find
  9. Qiao Jin, Bhuwan Dhingra, Zhengping Liu, William W Cohen, and Xinghua Lu. 2019. Pubmedqa: A dataset for biomedical research question answering. arXiv preprint arXiv:1909.06146.
  10. Alan E Kazdin and Jerome Kagan. 1994. Models of dysfunction in developmental psychopathology. Clinical Psychology: Science and Practice, 1(1):35. find
  11. Siwon Kim, Sangdoo Yun, Hwaran Lee, Martin Gubri, Sungroh Yoon, and Seong Joon Oh. 2023. Propile: Probing privacy leakage in large language models. Advances in Neural Information Processing Systems, 36:20750–20762. find
  12. Guangwei Li, Yuansen Zhang, Yinggui Wang, Shoumeng Yan, Lei Wang, and Tao Wei. 2025. Priv-qa: Privacy-preserving question answering for cloud large language models. arXiv preprint arXiv:2502.13564. — [arXiv:2502.13564](https://arxiv.org/abs/2502.13564) find
  13. Qinbin Li, Junyuan Hong, Chulin Xie, Jeffrey Tan, Rachel Xin, Junyi Hou, Xavier Yin, Zhun Wang, Dan Hendrycks, Zhangyang Wang, and 1 others. 2024. Llm-pbe: Assessing data privacy in large language models. arXiv preprint arXiv:2408.12787. find
  14. Qinbin Li, Zeyi Wen, Zhaomin Wu, Sixu Hu, Naibo Wang, Yuan Li, Xu Liu, and Bingsheng He. 2021. A survey on federated learning systems: Vision, hype and reality for data privacy and protection. IEEE Transactions on Knowledge and Data Engineering, 35(4):3347–3366. find
  15. Yi Liu, Gelei Deng, Yuekang Li, Kailong Wang, Zihao Wang, Xiaofeng Wang, Tianwei Zhang, Yepang Liu, Haoyu Wang, Yan Zheng, and 1 others. 2023a. Prompt injection attack against llm-integrated applications. arXiv preprint arXiv:2306.05499.
  16. Yisu Liu, Jinyang An, Wanqian Zhang, Dayan Wu, Jingzi Gu, Zheng Lin, and Weiping Wang. 2024. Disrupting diffusion: Token-level attention erasure attack against diffusion-based customization. In Proceedings of the 32nd ACM International Conference on Multimedia, pages 3587–3596. find
  17. Zhenzhen Liu, Chao Wan, Varsha Kishore, Jin Peng Zhou, Minmin Chen, and Kilian Q Weinberger. 2023b. Correction with backtracking reduces hallucination in summarization. arXiv preprint arXiv:2310.16176. — [arXiv:2310.16176](https://arxiv.org/abs/2310.16176) find
  18. Nils Lukas, Ahmed Salem, Robert Sim, Shruti Tople, Lukas Wutschitz, and Santiago Zanella-Béguelin. 2023. Analyzing leakage of personally identifiable information in language models. In 2023 IEEE Symposium on Security and Privacy (SP), pages 346–363. IEEE.
  19. Stephen Meisenbacher, Alexandra Klymenko, and Florian Matthes. 2025. Llm-as-a-judge for privacy evaluation? exploring the alignment of human and llm perceptions of privacy in textual data. In Proceedings of the 2025 Workshop on Human-Centered AI Privacy and Security, pages 126–138. find
  20. Fábio Perez and Ian Ribeiro. 2022. Ignore previous prompt: Attack techniques for language models. arXiv preprint arXiv:2211.09527.
  21. Zhenting Qi, Hanlin Zhang, Eric Xing, Sham Kakade, and Himabindu Lakkaraju. 2024. Follow my instruction and spill the beans: Scalable data extraction from retrieval-augmented generation systems. arXiv preprint arXiv:2402.17840. — [arXiv:2402.17840](https://arxiv.org/abs/2402.17840) find
  22. Qwen, :, An Yang, Baosong Yang, Beichen Zhang, Binyuan Hui, Bo Zheng, Bowen Yu, Chengyuan Li, Dayiheng Liu, Fei Huang, Haoran Wei, Huan Lin, Jian Yang, Jianhong Tu, Jianwei Zhang, Jianxin Yang, Jiaxi Yang, Jingren Zhou, and 25 others. 2025. Qwen2.5 technical report. Preprint, arXiv:2412.15115. find
  23. Bilgehan Sel, Dingcheng Li, Phillip Wallis, Vaishakh Keshava, Ming Jin, and Siddhartha Reddy Jonnalagadda. 2025. Backtracking for safety. arXiv preprint arXiv:2503.08919. — [arXiv:2503.08919](https://arxiv.org/abs/2503.08919) find
  24. Zhihong Shao, Peiyi Wang, Qihao Zhu, Runxin Xu, Junxiao Song, Xiao Bi, Haowei Zhang, Mingchuan Zhang, YK Li, Yang Wu, and 1 others. 2024. Deepseekmath: Pushing the limits of mathematical reasoning in open language models. arXiv preprint arXiv:2402.03300.
  25. Li Siyan, Vethavikashini Chithrra Raghuram, Omar Khattab, Julia Hirschberg, and Zhou Yu. 2024. Papillon: Privacy preservation from internet-based and local language model ensembles. arXiv preprint arXiv:2410.17127. find
  26. Panjun Sun, Shigen Shen, Yi Wan, Zongda Wu, Zhaoxi Fang, and Xiao-zhi Gao. 2024. A survey of iot privacy security: Architecture, technology, challenges, and trends. IEEE internet of things journal, 11(21):34567–34591. find
  27. Gemma Team, Morgane Riviere, Shreya Pathak, Pier Giuseppe Sessa, Cassidy Hardin, Surya Bhupatiraju, Léonard Hussenot, Thomas Mesnard, Bobak Shahriari, Alexandre Ramé, and 1 others. 2024. Gemma 2: Improving open language models at a practical size. arXiv preprint arXiv:2408.00118. find
  28. Rongsheng Wang. 2023. Healthcaremagic-100ken. https://huggingface.co/datasets/ wangrongsheng/HealthCareMagic-100k-en. ↗
  29. Yujing Wang, Hainan Zhang, Liang Pang, Yongxin Tong, Binghui Guo, Hongwei Zheng, and Zhiming Zheng. 2025. Learning to erase private knowledge from multi-documents for retrieval-augmented large language models. arXiv preprint arXiv:2504.09910. — [arXiv:2504.09910](https://arxiv.org/abs/2504.09910) find
  30. Kaiyan Zhang, Jianyu Wang, Ermo Hua, Biqing Qi, Ning Ding, and Bowen Zhou. 2024a. Cogenesis: A framework collaborating large and small language models for secure context-aware instruction following. arXiv preprint arXiv:2403.03129. find
  31. Jason Wei, Xuezhi Wang, Dale Schuurmans, Maarten Bosma, Fei Xia, Ed Chi, Quoc V Le, Denny Zhou, and 1 others. 2022. Chain-of-thought prompting elicits reasoning in large language models. Advances in neural information processing systems, 35:24824– 24837.
  32. Yiming Zhang, Jianfeng Chi, Hailey Nguyen, Kartikeya Upasani, Daniel M Bikel, Jason Weston, and Eric Michael Smith. 2024b. Backtracking improves generation safety. arXiv preprint arXiv:2409.14586. — [arXiv:2409.14586](https://arxiv.org/abs/2409.14586) find
  33. Junchao Wu, Shu Yang, Runzhe Zhan, Yulin Yuan, Lidia Sam Chao, and Derek Fai Wong. 2025a. A survey on llm-generated text detection: Necessity, methods, and future directions. Computational Linguistics, 51(1):275–338.
  34. Tsung-Han Wu, Heekyung Lee, Jiaxin Ge, Joseph E Gonzalez, Trevor Darrell, and David M Chan. 2025b. Generate, but verify: Reducing hallucination in vision-language models with retrospective resampling. arXiv preprint arXiv:2504.13169. find
  35. Ying Zhao, Jia Tina Du, and Jinjun Chen. 2024. Scenario-based adaptations of differential privacy: a technical survey. ACM Computing Surveys, 56(8):1– 39. find
  36. Yuan Zhou, Zhuo Zhang, and Xiangyu Zhang. 2025. Exploiting the shadows: Unveiling privacy leaks through lower-ranked tokens in large language models. In Proceedings of the 63rd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 8376–8386. find
  37. Yangzhen Wu, Zhiqing Sun, Shanda Li, Sean Welleck, and Yiming Yang. 2024. Inference scaling laws: An empirical analysis of compute-optimal inference for problem-solving with language models. arXiv preprint arXiv:2408.00724.
  38. Biwei Yan, Kun Li, Minghui Xu, Yueyan Dong, Yue Zhang, Zhaochun Ren, and Xiuzhen Cheng. 2025. On protecting the data privacy of large language models (llms) and llm agents: A literature review. High-Confidence Computing, page 100300. find
  39. Xiao-Wen Yang, Xuan-Yi Zhu, Wen-Da Wei, Ding-Chu Zhang, Jie-Jing Shao, Zhi Zhou, Lan-Zhe Guo, and Yu-Feng Li. 2025. Step back to leap forward: Self-backtracking for boosting reasoning of language models. arXiv preprint arXiv:2502.04404. — [arXiv:2502.04404](https://arxiv.org/abs/2502.04404) find
  40. Shenglai Zeng, Jiankun Zhang, Pengfei He, Jie Ren, Tianqi Zheng, Hanqing Lu, Han Xu, Hui Liu, Yue Xing, and Jiliang Tang. 2024a. Mitigating the privacy issues in retrieval-augmented generation (rag) via pure synthetic data. arXiv preprint arXiv:2406.14773. find
  41. Shenglai Zeng, Jiankun Zhang, Pengfei He, Yue Xing, Yiding Liu, Han Xu, Jie Ren, Shuaiqiang Wang, Dawei Yin, Yi Chang, and 1 others. 2024b. The good and the bad: Exploring privacy issues in retrieval-augmented generation (rag). arXiv preprint arXiv:2402.16893. find
  42. Yi Zeng, Hongpeng Lin, Jingwen Zhang, Diyi Yang, Ruoxi Jia, and Weiyan Shi. 2024c. How johnny can persuade llms to jailbreak them: Rethinking persuasion to challenge ai safety by humanizing llms. In Proceedings of the 62nd Annual Meeting of the Association for Computational Linguistics (Volume 1: Lo…

Updated

↑ back to top

This page in Markdown → https://md.research.elamur.ai/free/en/papers/yao-2026-adaptive-backtracking-privacy-protection-large/