Elamur Research⌕
☰
home / papers / 2025 / What Did I Do Wrong? Quantifying LLMs' Sensitivity and Consistency to Prompt Engineering

What Did I Do Wrong? Quantifying LLMs' Sensitivity and Consistency to Prompt Engineering

Paper · 2025 · Federico Errica, Davide Sanvito, Giuseppe Siracusano et al.

Summary

Developers who put LLMs inside software pipelines often see predictions change after tiny, meaning-preserving edits to a prompt, such as adding an article to a label description or reordering labels in an Instructor/Function Calling setup. Accuracy alone does not reveal this fragility, and measuring it requires labeled data. The authors propose two diagnostic metrics for black-box LLM classifiers that complement task performance. Sensitivity measures how much a sample's predicted class distribution shifts across semantically equivalent prompt rephrasings, and it needs no ground-truth labels. Consistency measures how similar those distributions are for different samples of the same class. Both are estimated by drawing Q rephrasings from a sampler S. The authors test the metrics on text classification datasets with GPT3.5 and Llama3 under several prompting strategies. They find that the metrics capture information beyond accuracy. They also use the metrics for qualitative debugging, such as spotting groups of samples that are misclassified in similar ways. The work gives practitioners a practical way to judge whether prompt tuning is worthwhile and whether a model is reliable enough for production.

Key points

  • Two metrics are introduced: sensitivity, which is label-free and computed per sample across prompt rephrasings, and consistency, which compares pairs of same-class samples through C_y(x, x′).
  • Experiments use the TREC, DBPedia, WoS and CB classification datasets, with GPT3.5 and Llama3 as classifiers. The prompting strategies compared are Simple, Detail and 1-shot.
  • A motivating example shows GPT3.5 changing its TREC prediction when the class "ENTY" is redefined as "An Entity" instead of "Entity".
  • On TREC, the 1-shot strategy does not uniformly raise consistency and even increases the number of zero-consistency pairs. On CB, the Detail strategy makes predictions more sensitive for some classes. The authors conclude that per-sample analysis is more informative than averages.
  • The main limitations are that the metrics apply only to classification and that results depend on the rephrasing sampler, the number of rephrasings, and the cost of the extra queries. Proposed future work includes extending the metrics to code generation and integrating them into automatic prompt optimization.

Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.

 Institutions

🏢NECTokyo · Japan6 works · 546 citations

 Fields, topics and concepts

📚 Shelves 4The field of science this work belongs to (OpenAlex fields).Computer Science · 16,667 worksArtificial Intelligence · 16,194 worksArts and Humanities · 16,039 worksLanguage and Linguistics · 15,891 works

🧭 Topics 1Research subjects and languages the work is about.Language models & pretraining · 3,320 works

Cite

Cite as Federico Errica, Davide Sanvito, Giuseppe Siracusano and Roberto Bifulco. 2025. What Did I Do Wrong? Quantifying LLMs' Sensitivity and Consistency to Prompt Engineering. Proceedings of the 2025 Conference of the Nations of the Americas Chapter of the Association for Computational Linguistics: Human Language Technologies (Volume 1: Long Papers).

The paper itself, not this page. BibTeX from the library's record of the work. DOI 10.18653/v1/2025.naacl-long.73

@inproceedings{errica2025what,
  title = {{What Did I Do Wrong? Quantifying LLMs' Sensitivity and Consistency to Prompt Engineering}},
  author = {Federico Errica, Davide Sanvito, Giuseppe Siracusano and Roberto Bifulco},
  booktitle = {Proceedings of the 2025 Conference of the Nations of the Americas Chapter of the Association for Computational Linguistics: Human Language Technologies (Volume 1: Long Papers)},
  year = {2025},
  doi = {10.18653/v1/2025.naacl-long.73},
  url = {https://doi.org/10.18653/v1/2025.naacl-long.73}
}

References

40 references, 11 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

  1. Melissa Ailem, Katerina Marazopoulou, Charlotte Siska, and James Bono. 2024. Examining the robustness of llm evaluation to the distributional assumptions of benchmarks. arXiv preprint arXiv:2404.16966. find
  2. David Barber. 2012. Bayesian reasoning and machine learning. Cambridge University Press. Chapter 27.1. find
  3. Tom Brown, Benjamin Mann, Nick Ryder, Melanie Subbiah, Jared D Kaplan, Prafulla Dhariwal, Arvind Neelakantan, Pranav Shyam, Girish Sastry, Amanda Askell, et al. 2020. Language models are few-shot learners. In Proceedings of the 34th Conference on Neural Information Processing Systems (NeurIPS).
  4. Lingjiao Chen, Jared Quincy Davis, Boris Hanin, Peter Bailis, Ion Stoica, Matei Zaharia, and James Zou. 2024. Are more llm calls all you need? towards scaling laws of compound inference systems. arXiv preprint arXiv:2403.02419. find
  5. Ido Dagan, Oren Glickman, and Bernardo Magnini. 2005. The pascal recognising textual entailment challenge. In Machine learning challenges workshop. Springer. find
  6. Arghavan Moradi Dakhel, Vahid Majdinasab, Amin Nikanjam, Foutse Khomh, Michel C Desmarais, and Zhen Ming Jack Jiang. 2023. Github copilot ai pair programmer: Asset or liability? Journal of Systems and Software, 203. find
  7. Marie-Catherine De Marneffe, Mandy Simons, and Judith Tonhauser. 2019. The commitmentbank: Investigating projection in naturally occurring discourse. In proceedings of Sinn und Bedeutung.
  8. Jacob Devlin, Ming-Wei Chang, Kenton Lee, and Kristina Toutanova. 2019. BERT: pre-training of deep bidirectional transformers for language understanding. In Proceedings of the 2019 Annual Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Techno…
  9. Jiahui Geng, Fengyu Cai, Yuxia Wang, Heinz Koeppl, Preslav Nakov, and Iryna Gurevych. 2024. A survey of confidence estimation and calibration in large language models. In Proceedings of the 2024 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language… find
  10. Luca Gioacchini, Giuseppe Siracusano, Davide Sanvito, Kiril Gashteovski, David Friede, Roberto Bifulco, and Carolin Lawrence. 2024. Agentquest: A modular benchmark framework to measure progress and improve llm agents. In Proceedings of the 2024 Annual Conference of the North American Chapter of the… find
  11. Xinmeng Huang, Shuo Li, Mengxin Yu, Matteo Sesia, Hamed Hassani, Insup Lee, Osbert Bastani, and Edgar Dobriban. 2024. Uncertainty in language models: Assessment through rank-calibration. arXiv preprint arXiv:2404.03163. find
  12. Yuheng Huang, Jiayang Song, Zhijie Wang, Huaming Chen, and Lei Ma. 2023. Look before you leap: An exploratory study of uncertainty measurement for large language models. arXiv preprint arXiv:2307.10236. — [arXiv:2307.10236](https://arxiv.org/abs/2307.10236) find
  13. Albert Q Jiang, Alexandre Sablayrolles, Antoine Roux, Arthur Mensch, Blanche Savary, Chris Bamford, Devendra Singh Chaplot, Diego de las Casas, Emma Bou Hanna, Florian Bressand, et al. 2024. Mixtral of experts. arXiv preprint arXiv:2401.04088. find
  14. Saurav Kadavath, Tom Conerly, Amanda Askell, Tom Henighan, Dawn Drain, Ethan Perez, Nicholas Schiefer, Zac Hatfield-Dodds, Nova DasSarma, Eli Tran-Johnson, et al. 2022. Language models (mostly) know what they know. arXiv preprint arXiv:2207.05221. find
  15. Sungmin Kang, Gabin An, and Shin Yoo. 2023. A preliminary evaluation of llm-based fault localization. arXiv preprint arXiv:2308.05487. — [arXiv:2308.05487](https://arxiv.org/abs/2308.05487) find
  16. Sunnie SY Kim, Q Vera Liao, Mihaela Vorvoreanu, Stephanie Ballard, and Jennifer Wortman Vaughan. 2024. "i’m not sure, but...": Examining the impact of large language models’ uncertainty expression on user reliance and trust. arXiv preprint arXiv:2405.00623. — [arXiv:2405.00623](https://arxiv.org/abs/2405.00623) find
  17. Kamran Kowsari, Donald Brown, Mojtaba Heidarysafa, Kiana Jafari Meimandi, Matthew Gerber, and Laura Barnes. 2018. Web of science dataset v6. Mendeley Data. find
  18. Patrick Lewis, Ethan Perez, Aleksandra Piktus, Fabio Petroni, Vladimir Karpukhin, Naman Goyal, Heinrich Küttler, Mike Lewis, Wen-tau Yih, Tim Rocktäschel, et al. 2020. Retrieval-augmented generation for knowledge-intensive nlp tasks. In Proceesings of the 34th Conference on Neural Information Proces…
  19. Percy Liang, Rishi Bommasani, Tony Lee, Dimitris Tsipras, Dilara Soylu, Michihiro Yasunaga, Yian Zhang, Deepak Narayanan, Yuhuai Wu, Ananya Kumar, Benjamin Newman, Binhang Yuan, Bobby Yan, Ce Zhang, Christian Alexander Cosgrove, Christopher D Manning, Christopher Re, Diana Acosta-Navas, Drew Arad Hu… find
  20. Jason Liu. 2024. Instructor. https://github.com/ jxnl/instructor. ↗
  21. Jiawei Liu, Chunqiu Steven Xia, Yuyao Wang, and Lingming Zhang. 2023. Is your code generated by chatgpt really correct? rigorous evaluation of large language models for code generation. In Proceedings of the 37th Conference on Neural Information Processing Systems (NeurIPS).
  22. Linhao Luo, Yuan-Fang Li, Reza Haf, and Shirui Pan. 2024. Reasoning on graphs: Faithful and interpretable large language model reasoning. In The 12th International Conference on Learning Representations (ICLR). find
  23. Ruotian Ma, Xiaolei Wang, Xin Zhou, Jian Li, Nan Du, Tao Gui, Qi Zhang, and Xuanjing Huang. 2024. Are large language models good prompt optimizers? arXiv preprint arXiv:2402.02101. — [arXiv:2402.02101](https://arxiv.org/abs/2402.02101) find
  24. Steve McConnell. 2004. Code complete. Pearson Education. find
  25. R Thomas McCoy, Shunyu Yao, Dan Friedman, Matthew Hardy, and Thomas L Griffiths. 2023. Embers of autoregression: Understanding large language models through the problem they are trained to solve. arXiv preprint arXiv:2309.13638. find
  26. Harsha Nori, Yin Tat Lee, Sheng Zhang, Dean Carignan, Richard Edgar, Nicolo Fusi, Nicholas King, Jonathan Larson, Yuanzhi Li, Weishung Liu, et al. 2023. Can generalist foundation models outcompete special-purpose tuning? case study in medicine. Medicine, 84. find
  27. Ori Press, Ravid Shwartz-Ziv, Yann LeCun, and Matthias Bethge. 2024. The entropy enigma: Success and failure of entropy minimization. arXiv preprint arXiv:2405.05012. — [arXiv:2405.05012](https://arxiv.org/abs/2405.05012) find
  28. Pranab Sahoo, Ayush Kumar Singh, Sriparna Saha, Vinija Jain, Samrat Mondal, and Aman Chadha. 2024. A systematic survey of prompt engineering in large language models: Techniques and applications. arXiv preprint arXiv:2402.07927.
  29. Melanie Sclar, Yejin Choi, Yulia Tsvetkov, and Alane Suhr. 2024. Quantifying language models’ sensitivity to spurious features in prompt design or: How i learned to start worrying about prompt formatting. In The 12th International Conference on Learning Representations (ICLR). find
  30. Hugo Touvron, Thibaut Lavril, Gautier Izacard, Xavier Martinet, Marie-Anne Lachaux, Timothée Lacroix, Baptiste Rozière, Naman Goyal, Eric Hambro, Faisal Azhar, et al. 2023. Llama: Open and efficient foundation language models. arXiv preprint arXiv:2302.13971.
  31. Saad Ullah, Mingji Han, Saurabh Pujar, Hammond Pearce, Ayse Coskun, and Gianluca Stringhini. 2024. Llms cannot reliably identify and reason about security vulnerabilities (yet?): A comprehensive evaluation, framework, and benchmarks. In IEEE Symposium on Security and Privacy (SP). find
  32. Ellen M Voorhees and Dawn M Tice. 2000. Building a question answering test collection. In Proceedings of the 23rd annual international ACM SIGIR conference on Research and development in information retrieval (SIGIR). find
  33. Jason Wei, Xuezhi Wang, Dale Schuurmans, Maarten Bosma, Fei Xia, Ed Chi, Quoc V Le, Denny Zhou, et al. 2022. Chain-of-thought prompting elicits reasoning in large language models. In Proceesings of the 36th Conference on Neural Information Processing Systems (NeurIPS).
  34. Yasin Abbasi Yadkori, Ilja Kuzborskij, András György, and Csaba Szepesvári. 2024. To believe or not to believe your llm. arXiv preprint arXiv:2406.02543. find
  35. Adam Yang, Chen Chen, and Konstantinos Pitas. 2024a. Just rephrase it! uncertainty estimation in closed-source language models via multiple rephrased queries. arXiv preprint arXiv:2405.13907. — [arXiv:2405.13907](https://arxiv.org/abs/2405.13907) find
  36. Zhe Yang, Yichang Zhang, Tianyu Liu, Jian Yang, Junyang Lin, Chang Zhou, and Zhifang Sui. 2024b. Can large language models always solve easy problems if they can solve harder ones? In Proceedings of the 2024 Conference on Empirical Methods in Natural Language Processing, EMNLP. find
  37. Xiang Zhang, Junbo Zhao, and Yann LeCun. 2015. Character-level convolutional networks for text classification. In Proceedings of the 29th Conference on Neural Information Processing Systems (NeurIPS).
  38. Zihao Zhao, Eric Wallace, Shi Feng, Dan Klein, and Sameer Singh. 2021. Calibrate before use: Improving few-shot performance of language models. In Proceedings of the 38th International Conference on Machine Learning (ICML).
  39. Andy Zhou, Bo Li, and Haohan Wang. 2024. Robust prompt optimization for defending language models against jailbreaking attacks. arXiv preprint arXiv:2401.17263. — [arXiv:2401.17263](https://arxiv.org/abs/2401.17263) find
  40. Yongchao Zhou, Andrei Ioan Muresanu, Ziwen Han, Keiran Paster, Silviu Pitis, Harris Chan, and Jimmy Ba. 2023. Large language models are human-level prompt engineers. In The 11th International Conference on Learning Representations (ICLR).

Updated

↑ back to top

This page in Markdown → https://md.research.elamur.ai/free/en/papers/errica-2025-what-did-i-do-wrong/