Elamur Research
☰
home / papers / 2023 / A Prompt Pattern Catalog to Enhance Prompt Engineering with ChatGPT

A Prompt Pattern Catalog to Enhance Prompt Engineering with ChatGPT

Written for the Research library (claude-opus-5-5) from the paper text, in our own words: the license "arXiv — non-exclusive license to distribute" does not allow redistributing the text itself. Original — arxiv.org/pdf/2302.11382

Summary

Getting dependable, well-structured output from conversational LLMs such as ChatGPT often relies on informal prompting tricks that are hard to share or reuse. The Vanderbilt authors argue that prompting is a form of programming. They borrow the documentation tradition of software design patterns (Gamma et al.) to record prompting know-how in a reusable, largely domain-independent way. They adapt the classic pattern template so that each entry contains:

  • a name and category
  • intent and context
  • motivation
  • a structure described as a set of "key ideas" rather than UML diagrams or a formal grammar
  • an example wording
  • consequences

With this template they build a catalog of 16 prompt patterns and show how several patterns can be combined within one prompt. The study is qualitative. It is illustrated with ChatGPT examples, mostly from software engineering, and reports no benchmark scores. Its value is a shared vocabulary and design checklist for prompts, which can also be applied to tasks such as data annotation or NLP tooling.

Key points

  • The catalog contains 16 prompt patterns aimed at output generation and interaction with conversational LLMs, demonstrated on ChatGPT (paper dated February 2023).
  • The patterns are sorted into several categories, among them Context Control, Interaction, Prompt Improvement, Error Identification, and Output Customization.
  • Each pattern's structure is expressed as "key ideas" that can be worded in many ways. The authors rejected UML and formal grammars as too rigid for natural-language prompts and too unfriendly for non-programmers.
  • An illustrative example has the model interview the user before generating a script to deploy a Python app to AWS. This shows prompts that reshape the interaction itself, not just the output format.
  • There is no quantitative evaluation. The contribution is the documentation framework, the catalog, and guidance on combining patterns.

Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.

Fields: Other

Cited by

131 papers of our library cite it (of the 140673 whose bibliographies we read); 6 of them have a page here, 125 not yet.

Cite

The paper itself, not this page. BibTeX from the library's record of the work. DOI 10.48550/arxiv.2302.11382 · arXiv 2302.11382

@misc{white2023prompt,
  title = {{A Prompt Pattern Catalog to Enhance Prompt Engineering with ChatGPT}},
  author = {White, Jules and Fu, Quchen and Hays, Sam and Sandborn, Michael and Olea, Carlos and Gilbert, Henry and Elnashar, Ashraf and Spencer-Smith, Jesse and Schmidt, Douglas C.},
  year = {2023},
  doi = {10.48550/arxiv.2302.11382},
  eprint = {2302.11382},
  archivePrefix = {arXiv},
  url = {https://doi.org/10.48550/arxiv.2302.11382}
}

References

35 references, 12 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

  1. R. Bommasani, D. A. Hudson, E. Adeli, R. Altman, S. Arora, S. von Arx, M. S. Bernstein, J. Bohg, A. Bosselut, E. Brunskill et al., “On the opportunities and risks of foundation models,” arXiv preprint arXiv:2108.07258, 2021. find our card
  2. Y. Bang, S. Cahyawijaya, N. Lee, W. Dai, D. Su, B. Wilie, H. Lovenia, Z. Ji, T. Yu, W. Chung et al., “A multitask, multilingual, multimodal evaluation of chatgpt on reasoning, hallucination, and interactivity,” arXiv preprint arXiv:2302.04023, 2023. find our card
  3. A. Gilson, C. Safranek, T. Huang, V. Socrates, L. Chi, R. A. Taylor, and D. Chartash, “How well does chatgpt do when taking the medical licensing exams?” medRxiv, pp. 2022–12, 2022. find
  4. A. Carleton, M. H. Klein, J. E. Robert, E. Harper, R. K. Cunningham, D. de Niz, J. T. Foreman, J. B. Goodenough, J. D. Herbsleb, I. Ozkaya, and D. C. Schmidt, “Architecting the future of software engineering,” Computer, vol. 55, no. 9, pp. 89–93, 2022. find
  5. “Github copilot · your ai pair programmer.” [Online]. Available: https://github.com/features/copilot ↗
  6. O. Asare, M. Nagappan, and N. Asokan, “Is github’s copilot as bad as humans at introducing vulnerabilities in code?” arXiv preprint arXiv:2204.04741, 2022. — [arXiv:2204.04741](https://arxiv.org/abs/2204.04741) find
  7. H. Pearce, B. Ahmad, B. Tan, B. Dolan-Gavitt, and R. Karri, “Asleep at the keyboard? assessing the security of github copilot’s code contributions,” in 2022 IEEE Symposium on Security and Privacy (SP). IEEE, 2022, pp. 754–768. find
  8. J. Krochmalski, IntelliJ IDEA Essentials. Packt Publishing Ltd, 2014. find
  9. P. Liu, W. Yuan, J. Fu, Z. Jiang, H. Hayashi, and G. Neubig, “Pre-train, prompt, and predict: A systematic survey of prompting methods in natural language processing,” ACM Computing Surveys, vol. 55, no. 9, pp. 1–35, 2023. find our card
  10. E. Gamma, R. Johnson, R. Helm, R. E. Johnson, and J. Vlissides, Design patterns: elements of reusable object-oriented software. Pearson Deutschland GmbH, 1995. find
  11. D. C. Schmidt, M. Stal, H. Rohnert, and F. Buschmann, Pattern-oriented software architecture, patterns for concurrent and networked objects. John Wiley & Sons, 2013. find
  12. OpenAI, “ChatGPT: Large-Scale Generative Language Models for Automated Content Creation,” https://openai.com/blog/chatgpt/, 2023, [Online; accessed 19-Feb-2023]. ↗
  13. ——, “DALL·E 2: Creating Images from Text,” https://openai.com/dall-e-2/, 2023, [Online; accessed 19-Feb-2023]. ↗
  14. D. Zhou, N. Schärli, L. Hou, J. Wei, N. Scales, X. Wang, D. Schuurmans, O. Bousquet, Q. Le, and E. Chi, “Least-to-most prompting enables complex reasoning in large language models,” arXiv preprint arXiv:2205.10625, 2022. find our card
  15. J. Ellson, E. R. Gansner, E. Koutsofios, S. C. North, and G. Woodhull, “Graphviz and dynagraph—static and dynamic graph drawing tools,” Graph drawing software, pp. 127–148, 2004. find
  16. S. Owen, “Building a virtual machine inside a javascript library,” https://www.engraved.blog/building-a-virtual-machine-inside/, 2022, accessed: 2023-02-20. ↗
  17. P. Zhang, J. White, D. C. Schmidt, and G. Lenz, “Applying software patterns to address interoperability in blockchain-based healthcare apps,” CoRR, vol. abs/1706.03700, 2017. [Online]. Available: http://arxiv.org/abs/1706.03700 — [arXiv:1706.03700](https://arxiv.org/abs/1706.03700) ↗
  18. X. Xu, C. Pautasso, L. Zhu, Q. Lu, and I. Weber, “A pattern collection for blockchain-based applications,” in Proceedings of the 23rd European Conference on Pattern Languages of Programs, 2018, pp. 1–20. find
  19. E. A. van Dis, J. Bollen, W. Zuidema, R. van Rooij, and C. L. Bockting, “Chatgpt: five priorities for research,” Nature, vol. 614, no. 7947, pp. 224–226, 2023. find
  20. L. Reynolds and K. McDonell, “Prompt programming for large language models: Beyond the few-shot paradigm,” CoRR, vol. abs/2102.07350, 2021. [Online]. Available: https://arxiv.org/abs/2102.07350 ↗ our card
  21. J. Wei, X. Wang, D. Schuurmans, M. Bosma, E. H. Chi, Q. Le, and D. Zhou, “Chain of thought prompting elicits reasoning in large language models,” CoRR, vol. abs/2201.11903, 2022. [Online]. Available: https://arxiv.org/abs/2201.11903 ↗ our card
  22. J. Wei, Y. Tay, R. Bommasani, C. Raffel, B. Zoph, S. Borgeaud, D. Yogatama, M. Bosma, D. Zhou, D. Metzler, E. H. Chi, T. Hashimoto, O. Vinyals, P. Liang, J. Dean, and W. Fedus, “Emergent abilities of large language models,” 2022. [Online]. Available: https://arxiv.org/abs/2206.07682 ↗ our card
  23. Y. Zhou, A. I. Muresanu, Z. Han, K. Paster, S. Pitis, H. Chan, and J. Ba, “Large language models are human-level prompt engineers,” 2022. [Online]. Available: https://arxiv.org/abs/2211.01910 ↗ our card
  24. T. Shin, Y. Razeghi, R. L. L. IV, E. Wallace, and S. Singh, “Autoprompt: Eliciting knowledge from language models with automatically generated prompts,” CoRR, vol. abs/2010.15980, 2020. [Online]. Available: https://arxiv.org/abs/2010.15980 ↗ our card
  25. A. Radford, J. Wu, R. Child, D. Luan, D. Amodei, and I. Sutskever, “Language models are unsupervised multitask learners,” 2019. find
  26. D. Zhou, N. Schärli, L. Hou, J. Wei, N. Scales, X. Wang, D. Schuurmans, C. Cui, O. Bousquet, Q. Le, and E. Chi, “Least-to-most prompting enables complex reasoning in large language models,” 2022. [Online]. Available: https://arxiv.org/abs/2205.10625 ↗ our card
  27. J. Jung, L. Qin, S. Welleck, F. Brahman, C. Bhagavatula, R. L. Bras, and Y. Choi, “Maieutic prompting: Logically consistent reasoning with recursive explanations,” 2022. [Online]. Available: https://arxiv.org/abs/2205.11822 ↗ our card
  28. S. Arora, A. Narayan, M. F. Chen, L. Orr, N. Guha, K. Bhatia, I. Chami, and C. Re, “Ask me anything: A simple strategy for prompting language models,” in International Conference on Learning Representations, 2023. [Online]. Available: https://openreview.net/forum?id=bhUPJnS2g0X ↗
  29. V. Liu and L. B. Chilton, “Design guidelines for prompt engineering text-to-image generative models,” in Proceedings of the 2022 CHI Conference on Human Factors in Computing Systems, 2022, pp. 1–23. find
  30. P. Maddigan and T. Susnjak, “Chat2vis: Generating data visualisations via natural language using chatgpt, codex and gpt-3 large language models,” arXiv preprint arXiv:2302.02094, 2023. — [arXiv:2302.02094](https://arxiv.org/abs/2302.02094) find
  31. X. Han, W. Zhao, N. Ding, Z. Liu, and M. Sun, “Ptr: Prompt tuning with rules for text classification,” AI Open, vol. 3, pp. 182–192, 2022. find
  32. S. Wang, H. Scells, B. Koopman, and G. Zuccon, “Can chatgpt write a good boolean query for systematic review literature search?” arXiv preprint arXiv:2302.03495, 2023. — [arXiv:2302.03495](https://arxiv.org/abs/2302.03495) find
  33. C. S. Xia and L. Zhang, “Conversational automated program repair,” arXiv preprint arXiv:2301.13246, 2023. — [arXiv:2301.13246](https://arxiv.org/abs/2301.13246) find
  34. J. H. Choi, K. E. Hickman, A. Monahan, and D. Schwarcz, “Chatgpt goes to law school,” Available at SSRN, 2023. find
  35. S. Frieder, L. Pinchetti, R.-R. Griffiths, T. Salvatori, T. Lukasiewicz, P. C. Petersen, A. Chevalier, and J. Berner, “Mathematical capabilities of chatgpt,” arXiv preprint arXiv:2301.13867, 2023. find our card