Elamur Research⌕
☰
home / papers / 2026 / Prompting the Unknown: Understanding Response Uncertainty in Large Language Models

Prompting the Unknown: Understanding Response Uncertainty in Large Language Models

Paper · 2026 · Ze Yu Zhang, Arun Verma, Finale Doshi-Velez et al.

Summary

LLM applications in sensitive fields such as medicine or finance rely on carefully designed prompts, but it is unclear how the content of a prompt affects how stable and trustworthy the model's answers are. The authors propose the prompt-response concept (PRC) model. It treats generation as mapping a latent "prompt concept", made of semantic attributes extracted from the prompt, to a latent "response concept". Response uncertainty is measured with entropy while the LLM and its decoding hyperparameters stay fixed. The framework splits uncertainty into four sources. Prompt underspecification is the reducible, epistemic part. Model quality, task variability and semantic redundancy act as aleatoric sources. The authors assume an idealized, perfectly trained LLM and prove that uncertainty falls as the prompt carries more task-relevant information or as model quality rises. Experiments on real-world datasets and a simulated physical-therapy mHealth scenario support this. The result gives a principled reason to reduce uncertainty by enriching prompts rather than by suppressing randomness through temperature or top-p settings.

Key points

  • Four sources of response uncertainty: prompt underspecification (epistemic) plus model quality, task variability and semantic redundancy (aleatoric under a fixed model). Uncertainty is measured as Shannon entropy of the response given the prompt.
  • Theorem 1: response uncertainty decreases monotonically with prompt informativeness and with model quality. This mirrors how epistemic uncertainty behaves in probabilistic models.
  • Empirical validation: GPT-3.5-Turbo, GPT-4, Llama 2 and Llama 3 largely follow the predictions, while Qwen2_1.5b shows deviations. With enough relevant information, a smaller model can sometimes beat a larger model given less.
  • Healthcare use case: an LLM app agent supports post-surgery physical therapy in a simulation where the patient is modelled as an MDP. Prompts with more information about patient intentions and available strategies improved intervention outcomes more consistently across patient types.
  • Limitations: the PRC model assumes an idealized LLM. It does not capture effects such as exemplar ordering in ICL or adversarial and jailbreak behaviour, though such issues tend to weaken as models get stronger.

Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.

 Fields, topics and concepts

📚 Shelves 4The field of science this work belongs to (OpenAlex fields).Computer Science · 16,667 worksArtificial Intelligence · 16,194 worksArts and Humanities · 16,039 worksLanguage and Linguistics · 15,891 works

🧭 Topics 1Research subjects and languages the work is about.Language models & pretraining · 3,320 works

Cite

Cite as Ze Yu Zhang, Arun Verma, Finale Doshi-Velez and Bryan Kian Hsiang Low. 2026. Prompting the Unknown: Understanding Response Uncertainty in Large Language Models. Findings of the Association for Computational Linguistics: ACL 2026.

The paper itself, not this page. BibTeX from the library's record of the work. DOI 10.18653/v1/2026.findings-acl.1548

@inproceedings{zhang2026prompting,
  title = {{Prompting the Unknown: Understanding Response Uncertainty in Large Language Models}},
  author = {Ze Yu Zhang, Arun Verma, Finale Doshi-Velez and Bryan Kian Hsiang Low},
  booktitle = {Findings of the Association for Computational Linguistics: ACL 2026},
  year = {2026},
  doi = {10.18653/v1/2026.findings-acl.1548},
  url = {https://doi.org/10.18653/v1/2026.findings-acl.1548}
}

References

6 references, 3 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

  1. Gati V Aher, Rosa I Arriaga, and Adam Tauman Kalai. 2023. Using large language models to simulate multiple humans and replicate human subject studies. In Proc. ICML, pages 337–371. find
  2. Lisa P Argyle, Ethan C Busby, Nancy Fulda, Joshua R Gubler, Christopher Rytting, and David Wingate. 2023. Out of one, many: Using language models to simulate human samples. Political Analysis, pages 337–351.
  3. Konstantine Arkoudas. 2023. arXiv:2308.03762.
  4. Marcel Binz and Eric Schulz. 2023. Using cognitive psychology to understand gpt-3. Proceedings of the National Academy of Sciences, page e2218523120. find
  5. Omar Boursalie, Reza Samavi, and Thomas E Doyle. 2018. Machine learning and mobile health monitoring platforms: a case study on research and implementation challenges. Journal of Healthcare Informatics Research, pages 179–203. find
  6. Kyunghyun Cho, Bart Van Merriënboer, Dzmitry Bahdanau, and Yoshua Bengio. 2014. On the properties of neural machine translation: Encoder-decoder approaches. arXiv:1409.1259.

Updated

This page in Markdown → https://md.research.elamur.ai/free/en/papers/zhang-2026-prompting-unknown-understanding-response-uncertainty/