Elamur Research⌕
☰
home / papers / 2026 / Feedback-Aware Prompt Optimization Framework for Generating Job Postings

Feedback-Aware Prompt Optimization Framework for Generating Job Postings

Paper · 2026 · Suraj Maharjan, Ainur Yessenalina and Srinivasan H. Sengamedu

Summary

Large employers have difficulty producing consistent, policy-compliant job postings, and the work weighs heavily on hiring managers and recruiters. The authors, from Amazon, describe a system that generates postings with an LLM. Its key feature is a prompt that improves over time from human input. Subject matter experts review the generated postings, and an LLM then rewrites the generation instructions to reflect their comments. The updated prompt is tested before reuse.

The prompt draws on several sources:

  • job metadata
  • compliance and role guidelines
  • competencies specific to each job family and level
  • the organization's brand statement
  • recent example postings

The authors compare two designs. One makes a single LLM call. The other pairs a Writer agent with a Reviewer agent that checks and revises each section of the posting. Evaluation used LLM-as-a-judge on 1,056 postings, plus human review, along three axes: Compliance, Standardization and User Perception.

The multi-agent design scored only about 1% higher but ran roughly seven times slower. The authors therefore chose the single-prompt version with streaming output for production. The work shows that structured expert feedback can make prompt-based generation work in an enterprise setting.

Key points

  • Dataset: 1,056 proprietary postings covering nine job families and Levels 4–7. The target was 30 per cell, except Support Engineering at Level 7, which has only 6. About 82.1% of the roles are individual-contributor positions.
  • Pipeline: Claude 3.7 Sonnet extracted hiring-manager-style inputs from existing postings. Sampling settings were temperature 0.7, top_p 0.7 and top_k 100. The refined prompt averages about 4,866 words.
  • Phase 1 human review: 15 internal reviewers took part, with an average compliance score of 86.69%. The review also exposed an ambiguous survey question about internal team names, which produced false positives.
  • Multi-agent vs. single prompt: The multi-agent design improved Compliance by about 1.0%, Standardization by 0.6%, User Perception by 0.2% and Overall by 0.4%. It was about 7.3× slower.
  • Limitations: Only closed-source models were tested (Claude family, Nova Pro). Turning feedback into prompt changes still needs human expertise. The data cannot be released.

Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.

 Fields, topics and concepts

📚 Shelves 6The field of science this work belongs to (OpenAlex fields).Computer Science · 16,667 worksArtificial Intelligence · 16,194 worksArts and Humanities · 16,039 worksLanguage and Linguistics · 15,891 worksBusiness, Management and Accounting · 32 worksOrganizational Behavior and Human Resource Management · 12 works

Cite

Cite as Suraj Maharjan, Ainur Yessenalina and Srinivasan H. Sengamedu. 2026. Feedback-Aware Prompt Optimization Framework for Generating Job Postings. Proceedings of the 19th Conference of the European Chapter of the Association for Computational Linguistics (Volume 5: Industry Track).

The paper itself, not this page. BibTeX from the library's record of the work. DOI 10.18653/v1/2026.eacl-industry.35

@inproceedings{maharjan2026feedback,
  title = {{Feedback-Aware Prompt Optimization Framework for Generating Job Postings}},
  author = {Suraj Maharjan, Ainur Yessenalina and Srinivasan H. Sengamedu},
  booktitle = {Proceedings of the 19th Conference of the European Chapter of the Association for Computational Linguistics (Volume 5: Industry Track)},
  year = {2026},
  doi = {10.18653/v1/2026.eacl-industry.35},
  url = {https://doi.org/10.18653/v1/2026.eacl-industry.35}
}

References

25 references, 10 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

  1. Tom Brown, Benjamin Mann, Nick Ryder, Melanie Subbiah, Jared D Kaplan, Prafulla Dhariwal, Arvind Neelakantan, Pranav Shyam, Girish Sastry, Amanda Askell, Sandhini Agarwal, Ariel Herbert-Voss, Gretchen Krueger, Tom Henighan, Rewon Child, Aditya Ramesh, Daniel Ziegler, Jeffrey Wu, Clemens Winter, and… find
  2. Cheng-Han Chiang and Hung-yi Lee. 2023. Can large language models be an alternative to human evaluations? In Proceedings of the 61st Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 15607–15631, Toronto, Canada. Association for Computational Linguistics. find
  3. Wei-Lin Chiang, Lianmin Zheng, Ying Sheng, Anastasios N. Angelopoulos, Tianle Li, Dacheng Li, Banghua Zhu, Hao Zhang, Michael I. Jordan, Joseph E. Gonzalez, and Ion Stoica. 2024. Chatbot arena: an open platform for evaluating llms by human preference. In Proceedings of the 41st International Confere…
  4. Yilun Du, Shuang Li, Antonio Torralba, Joshua B. Tenenbaum, and Igor Mordatch. 2024. Improving factuality and reasoning in language models through multiagent debate. In Forty-first International Conference on Machine Learning.
  5. Amazon Artificial General Intelligence. 2024. The amazon nova family of models: Technical report and model card. Amazon Technical Reports. find
  6. Md. Ashraful Islam, Mohammed Eunus Ali, and Md Rizwan Parvez. 2024. MapCoder: Multi-agent code generation for competitive problem solving. In Proceedings of the 62nd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 4912–4944, Bangkok, Thailand. Associati… find
  7. Albert Q Jiang, Alexandre Sablayrolles, Arthur Mensch, Chris Bamford, Devendra Singh Chaplot, Diego de las Casas, Florian Bressand, Gianna Lengyel, Guillaume Lample, Lucile Saulnier, and 1 others. 2023. Mistral 7b. arXiv preprint arXiv:2310.06825. find
  8. Albert Q Jiang, Alexandre Sablayrolles, Antoine Roux, Arthur Mensch, Blanche Savary, Chris Bamford, Devendra Singh Chaplot, Diego de las Casas, Emma Bou Hanna, Florian Bressand, and 1 others. 2024. Mixtral of experts. arXiv preprint arXiv:2401.04088. find
  9. Omar Khattab, Arnav Singhvi, Paridhi Maheshwari, Zhiyuan Zhang, Keshav Santhanam, Sri Vardhamanan, Saiful Haq, Ashutosh Sharma, Thomas T. Joshi, Hanna Moazam, Heather Miller, Matei Zaharia, and Christopher Potts. 2024. Dspy: Compiling declarative language model calls into self-improving pipelines.
  10. Bingxuan Li, Yiwei Wang, Jiuxiang Gu, Kai-Wei Chang, and Nanyun Peng. 2025. METAL: A multi-agent framework for chart generation with test-time scaling. In Proceedings of the 63rd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 30054–30069, Vienna, Austr…
  11. Jiahui Li and Roman Klinger. 2025. iPrOp: Interactive prompt optimization for large language models with a human in the loop. In Proceedings of the 63rd Annual Meeting of the Association for Computational Linguistics (Volume 4: Student Research Workshop), pages 276–285, Vienna, Austria. Association… find
  12. Xiaoqiang Lin, Zhongxiang Dai, Arun Verma, See-Kiong Ng, Patrick Jaillet, and Bryan Kian Hsiang Low. 2024. Prompt optimization with human feedback. arXiv preprint arXiv:2405.17346. — [arXiv:2405.17346](https://arxiv.org/abs/2405.17346) find
  13. Yao Lu, Max Bartolo, Alastair Moore, Sebastian Riedel, and Pontus Stenetorp. 2022. Fantastically ordered prompts and where to find them: Overcoming few-shot prompt order sensitivity. In Proceedings of the 60th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), p…
  14. Mahmoud Mohammadi, Yipeng Li, Jane Lo, and Wendy Yip. 2025. Evaluation and benchmarking of llm agents: A survey. In Proceedings of the 31st ACM SIGKDD Conference on Knowledge Discovery and Data Mining V.2, KDD ’25, page 6129–6139, New York, NY, USA. Association for Computing Machinery. find
  15. Yijia Shao, Yucheng Jiang, Theodore Kanell, Peter Xu, Omar Khattab, and Monica Lam. 2024. Assisting in writing Wikipedia-like articles from scratch with large language models. In Proceedings of the 2024 Conference of the North American Chapter of the Association for Computational Linguistics: Human…
  16. Noah Shinn, Federico Cassano, Ashwin Gopinath, Karthik Narasimhan, and Shunyu Yao. 2023. Reflexion: language agents with verbal reinforcement learning. In Proceedings of the 37th International Conference on Neural Information Processing Systems, NIPS ’23, Red Hook, NY, USA. Curran Associates Inc.
  17. Haoyang Su, Renqi Chen, Shixiang Tang, Zhenfei Yin, Xinzhe Zheng, Jinzhe Li, Biqing Qi, Qi Wu, Hui Li, Wanli Ouyang, Philip Torr, Bowen Zhou, and Nanqing Dong. 2025. Many heads are better than one: Improved scientific idea generation by a LLM-based multi-agent system. In Proceedings of the 63rd Annu…
  18. Lianmin Zheng, Wei-Lin Chiang, Ying Sheng, Siyuan Zhuang, Zhanghao Wu, Yonghao Zhuang, Zi Lin, Zhuohan Li, Dacheng Li, Eric P. Xing, Hao Zhang, Joseph E. Gonzalez, and Ion Stoica. 2023. Judging llm-as-a-judge with mt-bench and chatbot arena. In Proceedings of the 37th International Conference on Neu…
  19. Aman Singh Thakur, Kartik Choudhary, Venkat Srinik Ramayapally, Sankaran Vaidyanathan, and Dieuwke Hupkes. 2025. Judging the judges: Evaluating alignment and vulnerabilities in LLMs-as-judges. In Proceedings of the Fourth Workshop on Generation, Evaluation and Metrics (GEM²), pages 404–430, Vienna,… find
  20. Hugo Touvron, Louis Martin, Kevin Stone, Peter Albert, Amjad Almahairi, Yasmine Babaei, Nikolay Bashlykov, Soumya Batra, Prajjwal Bhargava, Shruti Bhosale, and 1 others. 2023. Llama 2: Open foundation and fine-tuned chat models. arXiv preprint arXiv:2307.09288.
  21. Yangkun Wang, Zihan Wang, and Jingbo Shang. 2025. Direct prompt optimization with continuous representations. In Proceedings of the 63rd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 2642–2652, Vienna, Austria. Association for Computational Linguistic… find
  22. Yijia Xiao, Edward Sun, Di Luo, and Wei Wang. 2025. Tradingagents: Multi-agents llm financial trading framework. Preprint, arXiv:2412.20138. — [arXiv:2412.20138](https://arxiv.org/abs/2412.20138) find
  23. Cilin Yan, Jingyun Wang, Lin Zhang, Ruihui Zhao, Xiaopu Wu, Kai Xiong, Qingsong Liu, Guoliang Kang, and Yangyang Kang. 2025. Efficient and accurate prompt optimization: the benefit of memory in exemplar-guided reflection. In Proceedings of the 63rd Annual Meeting of the Association for Computational… find
  24. Mert Yuksekgonul, Federico Bianchi, Joseph Boen, Sheng Liu, Pan Lu, Zhi Huang, Carlos Guestrin, and James Zou. 2025. Optimizing generative ai by backpropagating language model feedback. Nature, 639:609–616. find
  25. Cheng Zhen, Ervine Zheng, Jilong Kuang, and Geoffrey Jay Tso. 2025. Enhancing LLM-as-a-judge through active-sampling-based prompt optimization. In Proceedings of the 63rd Annual Meeting of the Association for Computational Linguistics (Volume 6: Industry Track), pages 960–970, Vienna, Austria. Assoc… find

Updated

↑ back to top

This page in Markdown → https://md.research.elamur.ai/free/en/papers/maharjan-2026-feedback-aware-prompt-optimization-framework/