Elamur Research⌕
☰
home / papers / 2026 / CheckMIABench: Firm Foundations For Membership Inference Attacks on Language Models

CheckMIABench: Firm Foundations For Membership Inference Attacks on Language Models

Paper · 2026 · Jeffrey George Wang, Jason Wang, Marvin Li et al.

Summary

Evaluating membership inference attacks (MIAs) on LLMs is hard because member and non-member sets often differ in distribution. Earlier benchmarks built from temporal cutoffs (WikiMIA) or from deduplicated train/validation splits (MIMIR on The Pile) can be separated by classifiers that never see the model. As a result, reported attack success may reflect dataset artifacts rather than real privacy leakage.

The authors address this with checkpoint-based splits. Training data is shuffled before training, so examples seen before an intermediate checkpoint and examples scheduled after it come from the same distribution. The checkpoint's model is then attacked using these "before" and "after" sets as members and non-members. They confirm the splits are clean with model-free ("blind") classifiers. They then test six published attacks on Pythia models from 70m to 2.8b, plus OLMo 7B.

Once distribution shift is removed, no attack beats chance with statistical significance. The authors release the modular pandora_llm library and argue that future model releases should deduplicate the full corpus jointly before splitting it.

Key points

  • Pythia setup: Models trained on deduplicated The Pile, evaluated at step 97,000 of 143,000, i.e. more than 95% through the first epoch. OLMo 7B (trained on Dolma) is evaluated at step 400,000, about 88% through its first epoch.
  • Blind baselines: BoW, TF-IDF, Word2Vec and BERT-embedding classifiers were trained on 4,000 points per class and tested on 1,000 per class. They reach AUCs of only 0.490–0.505. For comparison, a bag-of-words classifier reached 94.7% TPR at 5% FPR on WikiMIA and AUC up to 0.86 on MIMIR splits.
  • Attacks tested: LOSS, Min-K, Min-K%++, Zlib, ReCaLL and MoPe. AUCs fall roughly between 0.487 and 0.513, with standard errors around 0.013, so none differs significantly from 0.5.
  • Best low-FPR result: MoPe reaches TPR 0.025 at 1% FPR on Pythia-70m, about 2.5× random. This advantage fades on larger models.
  • Limitations: The method needs open training data, intermediate checkpoints and the exact training order, which Llama and Qwen do not provide. Only English-heavy models up to 7B were tested.

Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.

 Institutions

🎓Harvard UniversityCambridge · United States126 works · 45,130 citations

 Fields, topics and concepts

📚 Shelves 4The field of science this work belongs to (OpenAlex fields).Computer Science · 18,614 worksArtificial Intelligence · 17,965 worksSocial Sciences · 1,954 worksSociology and Political Science · 626 works

🧭 Topics 1Research subjects and languages the work is about.Language models & pretraining · 3,894 works

Cite

Cite as Jeffrey George Wang, Jason Wang, Marvin Li and Seth Neel. 2026. CheckMIABench: Firm Foundations For Membership Inference Attacks on Language Models. Proceedings of the 64th Annual Meeting of the Association for Computational Linguistics (Volume 2: Short Papers).

The paper itself, not this page. BibTeX from the library's record of the work. DOI 10.18653/v1/2026.acl-short.30

@inproceedings{wang2026checkmiabench,
  title = {{CheckMIABench: Firm Foundations For Membership Inference Attacks on Language Models}},
  author = {Jeffrey George Wang, Jason Wang, Marvin Li and Seth Neel},
  booktitle = {Proceedings of the 64th Annual Meeting of the Association for Computational Linguistics (Volume 2: Short Papers)},
  year = {2026},
  doi = {10.18653/v1/2026.acl-short.30},
  url = {https://doi.org/10.18653/v1/2026.acl-short.30}
}

References

30 references, 12 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

  1. Stella Biderman, Hailey Schoelkopf, Quentin Anthony, Herbie Bradley, Kyle O’Brien, Eric Hallahan, Mohammad Aflah Khan, Shivanshu Purohit, USVSN Sai Prashanth, Edward Raff, Aviya Skowron, Lintang Sutawika, and Oskar van der Wal. 2023. Pythia: A suite for analyzing large language models across trainin… find
  2. Nicholas Carlini, Steve Chien, Milad Nasr, Shuang Song, Andreas Terzis, and Florian Tramer. 2022. Membership inference attacks from first principles. Preprint, arXiv:2112.03570.
  3. Nicholas Carlini, Florian Tramèr, Eric Wallace, Matthew Jagielski, Ariel Herbert-Voss, Katherine Lee, Adam Roberts, Tom B. Brown, Dawn Xiaodong Song, Úlfar Erlingsson, Alina Oprea, and Colin Raffel. 2020. Extracting training data from large language models. Preprint, arXiv:2012.07805.
  4. Debeshee Das, Jie Zhang, and Florian Tramèr. 2025. Blind baselines beat membership inference attacks for foundation models. Preprint, arXiv:2406.16201. — [arXiv:2406.16201](https://arxiv.org/abs/2406.16201) find
  5. DeepSeek-AI, Aixin Liu, Bei Feng, Bing Xue, Bingxuan Wang, Bochao Wu, Chengda Lu, Chenggang Zhao, Chengqi Deng, Chenyu Zhang, Chong Ruan, Damai Dai, Daya Guo, Dejian Yang, Deli Chen, Dongjie Ji, Erhang Li, Fangyun Lin, Fucong Dai, and 181 others. 2025. DeepSeek-V3 technical report. Preprint, arXiv:2… find
  6. Jacob Devlin, Ming-Wei Chang, Kenton Lee, and Kristina Toutanova. 2019. BERT: Pre-training of deep bidirectional transformers for language understanding. Preprint, arXiv:1810.04805.
  7. Michael Duan, Anshuman Suri, Niloofar Mireshghallah, Sewon Min, Weijia Shi, Luke Zettlemoyer, Yulia Tsvetkov, Yejin Choi, David Evans, and Hannaneh Hajishirzi. 2024. Do membership inference attacks work on large language models? Preprint, arXiv:2402.07841.
  8. Leo Gao, Stella Biderman, Sid Black, Laurence Golding, Travis Hoppe, Charles Foster, Jason Phang, Horace He, Anish Thite, Noa Nabeshima, Shawn Presser, and Connor Leahy. 2020. The Pile: An 800GB dataset of diverse text for language modeling. Preprint, arXiv:2101.00027. find
  9. Dirk Groeneveld, Iz Beltagy, Pete Walsh, Akshita Bhagia, Rodney Kinney, Oyvind Tafjord, Ananya Harsh Jha, Hamish Ivison, Ian Magnusson, Yizhong Wang, Shane Arora, David Atkinson, Russell Authur, Khyathi Raghavi Chandu, Arman Cohan, Jennifer Dumas, Yanai Elazar, Yuling Gu, Jack Hessel, and 24 others.… find
  10. Jamie Hayes, Ilia Shumailov, Christopher A. Choquette-Choo, Matthew Jagielski, George Kaissis, Milad Nasr, Sahra Ghalebikesabi, Meenatchi Sundaram Mutu Selva Annamalai, Niloofar Mireshghallah, Igor Shilov, Matthieu Meeus, Yves-Alexandre de Montjoye, Katherine Lee, Franziska Boenisch, Adam Dziedzic,… — [arXiv:2505.18773](https://arxiv.org/abs/2505.18773) find
  11. Jamie Hayes, Ilia Shumailov, Eleni Triantafillou, Amr Khalifa, and Nicolas Papernot. 2024. Inexact unlearning needs more careful evaluations to avoid a false sense of privacy. Preprint, arXiv:2403.01218. — [arXiv:2403.01218](https://arxiv.org/abs/2403.01218) find
  12. Gyuwan Kim, Yang Li, Evangelia Spiliopoulou, Jie Ma, Miguel Ballesteros, and William Yang Wang. 2025. Detecting training data of large language models via expectation maximization. Preprint, arXiv:2410.07582.
  13. Meghdad Kurmanji, Peter Triantafillou, Jamie Hayes, and Eleni Triantafillou. 2023. Towards unbounded machine unlearning. Preprint, arXiv:2302.09880. — [arXiv:2302.09880](https://arxiv.org/abs/2302.09880) find
  14. Marvin Li, Jason Wang, Jeffrey Wang, and Seth Neel. 2023. MoPe: Model perturbation based privacy attacks on language models. In Proceedings of the 2023 Conference on Empirical Methods in Natural Language Processing, pages 13647–13660, Singapore. Association for Computational Linguistics. find
  15. Bo Liu, Ming Ding, Sina Shaham, Wenny Rahayu, Farhad Farokhi, and Zihuai Lin. 2021. When machine learning meets privacy: A survey and outlook. ACM Comput. Surv., 54(2). find
  16. Zhenhua Liu, Tong Zhu, Chuanyuan Tan, Haonan Lu, Bing Liu, and Wenliang Chen. 2024. Probing language models for pre-training data detection. Preprint, arXiv:2406.01333. find
  17. Pratyush Maini and Anshuman Suri. 2024. Reassessing EMNLP 2024’s best paper: Does divergence-based calibration for membership inference attacks hold up? Accessed May 1, 2025. find
  18. Justus Mattern, Fatemehsadat Mireshghallah, Zhijing Jin, Bernhard Schoelkopf, Mrinmaya Sachan, and Taylor Berg-Kirkpatrick. 2023. Membership inference attacks against language models via neighbourhood comparison. In Findings of the Association for Computational Linguistics: ACL 2023, pages 11330– 11…
  19. Matthieu Meeus, Igor Shilov, Shubham Jain, Manuel Faysse, Marek Rei, and Yves-Alexandre de Montjoye. 2025. SoK: Membership inference attacks on LLMs are rushing nowhere (and how to fix it). Preprint, arXiv:2406.17975. — [arXiv:2406.17975](https://arxiv.org/abs/2406.17975) find
  20. Tomas Mikolov, Kai Chen, Greg Corrado, and Jeffrey Dean. 2013. Efficient estimation of word representations in vector space. Preprint, arXiv:1301.3781.
  21. Martin Pawelczyk, Seth Neel, and Himabindu Lakkaraju. 2023. In-context unlearning: Language models as few shot unlearners. Preprint, arXiv:2310.07579.
  22. Weijia Shi, Anirudh Ajith, Mengzhou Xia, Yangsibo Huang, Daogao Liu, Terra Blevins, Danqi Chen, and Luke Zettlemoyer. 2023. Detecting pretraining data from large language models. Preprint, arXiv:2310.16789. find
  23. Reza Shokri, Marco Stronati, Congzheng Song, and Vitaly Shmatikov. 2017. Membership inference attacks against machine learning models. Preprint, arXiv:1610.05820.
  24. Luca Soldaini, Rodney Kinney, Akshita Bhagia, Dustin Schwenk, David Atkinson, Russell Authur, Ben Bogin, Khyathi Chandu, Jennifer Dumas, Yanai Elazar, Valentin Hofmann, Ananya Harsh Jha, Sachin Kumar, Li Lucy, Xinxi Lyu, Nathan Lambert, Ian Magnusson, Jacob Morrison, Niklas Muennighoff, and 17 other… find
  25. Hugo Touvron, Thibaut Lavril, Gautier Izacard, Xavier Martinet, Marie-Anne Lachaux, Timothée Lacroix, Baptiste Rozière, Naman Goyal, Eric Hambro, Faisal Azhar, Aurelien Rodriguez, Armand Joulin, Edouard Grave, and Guillaume Lample. 2023. LLaMA: Open and efficient foundation language models. Preprint…
  26. Roy Xie, Junlin Wang, Ruomin Huang, Minxing Zhang, Rong Ge, Jian Pei, Neil Zhenqiang Gong, and Bhuwan Dhingra. 2024. ReCaLL: Membership inference via relative conditional log-likelihoods. Preprint, arXiv:2406.15968. find
  27. An Yang, Anfeng Li, Baosong Yang, Beichen Zhang, Binyuan Hui, Bo Zheng, Bowen Yu, Chang Gao, Chengen Huang, Chenxu Lv, Chujie Zheng, Dayiheng Liu, Fan Zhou, Fei Huang, Feng Hu, Hao Ge, Haoran Wei, Huan Lin, Jialong Tang, and 41 others. 2025. Qwen3 technical report. Preprint, arXiv:2505.09388.
  28. Samuel Yeom, Irene Giacomelli, Matt Fredrikson, and Somesh Jha. 2018. Privacy risk in machine learning: Analyzing the connection to overfitting. Preprint, arXiv:1709.01604.
  29. Jingyang Zhang, Jingwei Sun, Eric Yeats, Yang Ouyang, Martin Kuo, Jianyi Zhang, Hao Frank Yang, and Hai Li. 2025. Min-k%++: Improved baseline for detecting pre-training data from large language models. Preprint, arXiv:2404.02936. — [arXiv:2404.02936](https://arxiv.org/abs/2404.02936) find
  30. Zhenhong Zhou, Jiuyang Xiang, Chaomeng Chen, and Sen Su. 2023. Quantifying and analyzing entity-level memorization in large language models. Preprint, arXiv:2308.15727. — [arXiv:2308.15727](https://arxiv.org/abs/2308.15727) find

Updated

↑ back to top

This page in Markdown → https://md.research.elamur.ai/free/en/papers/wang-2026-checkmiabench-firm-foundations-membership-inference/