Elamur Research⌕
☰
home / papers / 2026 / Calibration vs Decision Making: Revisiting the Reliability Paradox in Unlearned Language Models

Calibration vs Decision Making: Revisiting the Reliability Paradox in Unlearned Language Models

Paper · 2026 · Divyaksh Shukla and Ashutosh Modi

Summary

Machine unlearning aims to erase the effect of chosen training data while keeping a model trustworthy on the rest. Calibration is often treated as evidence of that trustworthiness. This paper tests whether the assumption holds for generative LLMs after unlearning.

The authors build several versions of Llama-3.1-8B: pretrained, fully fine-tuned on TOFU, retained (trained only on the retain split, the ideal reference) and unlearned. They evaluate them with the RELU protocol, which turns TOFU into four-option multiple-choice QA so that answer-level confidence can be computed. Calibration is scored with ECE, MCE and Brier. Decision rules are checked with the method of Bihani and Rayz (2024): a prediction counts as shortcut-cued when a highly attributed token (Integrated Gradients) is also strongly linked to the predicted label (Local Mutual Information).

Unlearned models remain well calibrated even as accuracy on the forget split drops, yet they rely more on correlation-based tokens. This extends the "reliability paradox" to decoder-only models and unlearning. It shows that calibration alone cannot certify an unlearned model as reliable.

Key points

  • Setup: Llama-3.1-8B on the TOFU benchmark, using RELU's MCQA format. The authors compare pretrained, full fine-tuned, retained and unlearned models across forget ratios.
  • Fine-tuning drops ECE from above 0.5 (pretrained) to roughly 0.04. Unlearned models keep similarly low calibration error.
  • Shortcut detection: tokens in the top 5% by LMI per label are treated as candidate shortcuts. A prediction is flagged when one of its top-k IG-attributed tokens falls in that set.
  • Shortcut reliance often grows after unlearning, particularly at higher forget ratios. Qualitative examples come from a DPO-unlearned model on the 90% retain split, where function words like "does" and "about" act as cues.
  • Limitations: only one model, MCQA-only calibration, approximate attribution and no robustness or OOD tests. Code is available at github.com/Exploration-Lab/Unlearning-Reliability-Paradox.

Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.

 Institutions

🎓Indian Institute of Technology KanpurKanpur · India15 works · 798 citations

 Fields, topics and concepts

📚 Shelves 2The field of science this work belongs to (OpenAlex fields).Computer Science · 18,614 worksArtificial Intelligence · 17,965 works

🧭 Topics 1Research subjects and languages the work is about.Language models & pretraining · 3,894 works

Cite

Cite as Divyaksh Shukla and Ashutosh Modi. 2026. Calibration vs Decision Making: Revisiting the Reliability Paradox in Unlearned Language Models. Proceedings of the 64th Annual Meeting of the Association for Computational Linguistics (Volume 4: Student Research Workshop).

The paper itself, not this page. BibTeX from the library's record of the work. DOI 10.18653/v1/2026.acl-srw.100

@inproceedings{shukla2026calibration,
  title = {{Calibration vs Decision Making: Revisiting the Reliability Paradox in Unlearned Language Models}},
  author = {Divyaksh Shukla and Ashutosh Modi},
  booktitle = {Proceedings of the 64th Annual Meeting of the Association for Computational Linguistics (Volume 4: Student Research Workshop)},
  year = {2026},
  doi = {10.18653/v1/2026.acl-srw.100},
  url = {https://doi.org/10.18653/v1/2026.acl-srw.100}
}

References

31 references, 8 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

  1. Basit Ali, Sachin Pawar, Girish Palshikar, Anindita Sinha Banerjee, and Dhirendra Singh. 2023. Legal argument extraction from court judgements using integer linear programming. In Proceedings of the 10th Workshop on Argument Mining, pages 52–63, Singapore. Association for Computational Linguistics. find
  2. Geetanjali Bihani and Julia Rayz. 2024. The reliability paradox: Exploring how shortcut learning undermines language model calibration. find
  3. Yinzhi Cao and Junfeng Yang. 2015. Towards making systems forget with machine unlearning. In 2015 IEEE Symposium on Security and Privacy, pages 463–480. find
  4. Mengnan Du, Fengxiang He, Na Zou, Dacheng Tao, and Xia Hu. 2023. Shortcut learning of large language models in natural language understanding. Commun. ACM, 67(1):110–120. find
  5. Mengnan Du, Varun Manjunatha, Rajiv Jain, Ruchi Deshpande, Franck Dernoncourt, Jiuxiang Gu, Tong Sun, and Xia Hu. 2021. Towards interpreting and mitigating shortcut learning behavior of NLU models. find
  6. Ronen Eldan and Mark Russinovich. 2023. Who’s harry potter? approximate unlearning in llms. Preprint, arXiv:2310.02238.
  7. Leon Eshuijs, Shihan Wang, and Antske Fokkens. 2025. Short-circuiting shortcuts: Mechanistic investigation of shortcuts in text classification. In Proceedings of the 29th Conference on Computational Natural Language Learning, pages 105–125, Vienna, Austria. Association for Computational Linguistics. find
  8. Antonio Ginart, Melody Guan, Gregory Valiant, and James Y Zou. 2019. Making ai forget you: Data deletion in machine learning. In Advances in Neural Information Processing Systems, volume 32. Curran Associates, Inc. find
  9. Aaron Grattafiori, Abhimanyu Dubey, Abhinav Jauhri, Abhinav Pandey, Abhishek Kadian, Ahmad Al-Dahle, Aiesha Letman, Akhil Mathur, Alan Schelten, Alex Vaughan, Amy Yang, Angela Fan, Anirudh Goyal, Anthony Hartshorn, Aobo Yang, Archi Mitra, Archie Sravankumar, Artem Korenev, Arthur Hinsvark, and 542 o… — The Llama 3 Herd of Models
  10. Chuan Guo, Geoff Pleiss, Yu Sun, and Kilian Q. Weinberger. 2017. On calibration of modern neural networks. In Proceedings of the 34th International Conference on Machine Learning - Volume 70, ICML’17, page 1321–1330. JMLR.org. find
  11. Pavel Izmailov, Polina Kirichenko, Nate Gruver, and Andrew Gordon Wilson. 2022. On feature learning in the presence of spurious correlations. In Proceedings of the 36th International Conference on Neural Information Processing Systems, NIPS ’22, Red Hook, NY, USA. Curran Associates Inc. find
  12. Abhinav Joshi, Shaswati Saha, Divyaksh Shukla, Sriram Vema, Harsh Jhamtani, Manas Gaur, and Ashutosh Modi. 2024. Towards robust evaluation of unlearning in LLMs via data transformations. In Findings of the Association for Computational Linguistics: EMNLP 2024, pages 12100–12119, Miami, Florida, USA.… find
  13. Nathaniel Li, Alexander Pan, Anjali Gopal, Summer Yue, Daniel Berrios, Alice Gatti, Justin D. Li, Ann-Kathrin Dombrowski, Shashwat Goel, Long Phan, Gabriel Mukobi, Nathan Helm-Burger, Rassin Lababidi, Lennart Justen, Andrew B. Liu, Michael Chen, Isabelle Barrass, Oliver Zhang, Xiaoyuan Zhu, and 38 o… — The WMDP Benchmark: Measuring and Reducing Malicious Use With Unlearning
  14. Bo Liu, Qiang Liu, and Peter Stone. 2022. Continual learning and private unlearning. Preprint, arXiv:2203.12817. — [arXiv:2203.12817](https://arxiv.org/abs/2203.12817) find
  15. Yezi Liu, Hanning Chen, Wenjun Huang, Yang Ni, and Mohsen Imani. 2025a. LUNE: Efficient LLM unlearning via loRA fine-tuning with negative examples. In Socially Responsible and Trustworthy Foundation Models at NeurIPS 2025. find
  16. Mukund Sundararajan, Ankur Taly, and Qiqi Yan. 2017. Axiomatic attribution for deep networks. In Proceedings of the 34th International Conference on Machine Learning - Volume 70, ICML’17, page 3319–3328. JMLR.org. find
  17. Yezi Liu, Hanning Chen, Wenjun Huang, Yang Ni, and Mohsen Imani. 2025b. Recover-to-forget: Gradient reconstruction from loRA for efficient LLM unlearning. In Socially Responsible and Trustworthy Foundation Models at NeurIPS 2025. find
  18. Tianlu Wang, Rohit Sridhar, Diyi Yang, and Xuezhi Wang. 2022. Identifying and mitigating spurious correlations for improving robustness in NLP models. In Findings of the Association for Computational Linguistics: NAACL 2022, pages 1719–1729, Seattle, United States. Association for Computational Ling… find
  19. Pratyush Maini, Zhili Feng, Avi Schwarzschild, Zachary C. Lipton, and J. Zico Kolter. 2024. Tofu: A task of fictitious unlearning for llms. Preprint, arXiv:2401.06121.
  20. Subhankar Maity and Manob Jyoti Saikia. 2025. Large language models in healthcare and medical applications: A review. Bioengineering, 12(6). find
  21. Jishnu Mukhoti, Viveka Kulharia, Amartya Sanyal, Stuart Golodetz, Philip H. S. Torr, and Puneet K. Dokania. 2020. Calibrating deep neural networks using focal loss. In Proceedings of the 34th International Conference on Neural Information Processing Systems, NIPS ’20, Red Hook, NY, USA. Curran Assoc… find
  22. Maja Pavlovic. 2025. Understanding model calibration - a gentle introduction and visual exploration of calibration and the expected calibration error (ece). In ICLR Blogposts 2025. Https://iclr-blogposts.github.io/2025/blog/calibration/. find
  23. Danish Pruthi, Mansi Gupta, Bhuwan Dhingra, Graham Neubig, and Zachary C. Lipton. 2020. Learning to deceive with attention-based explanations. In Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics, pages 4782–4793, Online. Association for Computational Linguistic…
  24. Gabriele Sarti, Nils Feldhus, Ludwig Sickert, Oskar van der Wal, Malvina Nissim, and Arianna Bisazza. 2023. Inseq: An interpretability toolkit for sequence generation models. In Proceedings of the 61st Annual Meeting of the Association for Computational Linguistics (Volume 3: System Demonstrations),…
  25. Zezheng Wu, Rui Wang, Xinghe Cheng, Yang Shao, Qing Yang, Jiapu Wang, and Jingwei Zhang. 2026. Ucan: Utility-aware contrastive attenuation for efficient unlearning in generative recommendation. Preprint, arXiv:2602.23400. — [arXiv:2602.23400](https://arxiv.org/abs/2602.23400) find
  26. Yahan Yang, Soham Dan, Dan Roth, and Insup Lee. 2026. On calibration of multilingual question answering LLMs. Transactions on Machine Learning Research. find
  27. Yu Yuan, Lili Zhao, Kai Zhang, Guangting Zheng, and Qi Liu. 2024. Do LLMs overcome shortcut learning? an evaluation of shortcut challenges in large language models. In Proceedings of the 2024 Conference on Empirical Methods in Natural Language Processing, pages 12188–12200, Miami, Florida, USA. Asso… find
  28. Ruiqi Zhang, Licong Lin, Yu Bai, and Song Mei. 2024. Negative preference optimization: From catastrophic collapse to effective unlearning. Preprint, arXiv:2404.05868.
  29. Haoxi Zhong, Chaojun Xiao, Cunchao Tu, Tianyang Zhang, Zhiyuan Liu, and Maosong Sun. 2020. How does NLP benefit legal system: A summary of legal artificial intelligence. In Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics, pages 5218–5230, Online. Association f…
  30. Weijia Shi, Jaechan Lee, Yangsibo Huang, Sadhika Malladi, Jieyu Zhao, Ari Holtzman, Daogao Liu, Luke Zettlemoyer, Noah A. Smith, and Chiyuan Zhang. 2024. Muse: Machine unlearning six-way evaluation for language models. Preprint, arXiv:2407.06460. find
  31. Karan Singhal, Tao Tu, Juraj Gottweis, Rory Sayres, Ellery Wulczyn, Mohamed Amin, Le Hou, Kevin Clark, Stephen R. Pfohl, Heather Cole-Lewis, Darlene Neal, Qazi Mamunur Rashid, Mike Schaekermann, Amy Wang, Dev Dash, Jonathan H. Chen, Nigam H. Shah, Sami Lachgar, Philip Andrew Mansfield, and 16 others… find

Updated

↑ back to top

This page in Markdown → https://md.research.elamur.ai/free/en/papers/shukla-2026-calibration-vs-decision-making-revisiting/