Elamur Research⌕
☰
home / papers / 2025 / Can Many-Shot In-Context Learning Help LLMs as Evaluators? A Preliminary Empirical Study

Can Many-Shot In-Context Learning Help LLMs as Evaluators? A Preliminary Empirical Study

Paper · 2025 · Mingyang Song, Mao Zheng and Xuan Luo

Summary

LLM judges are increasingly used to grade other models, but their scores can be unstable and skewed by hidden biases. This paper asks whether many-shot in-context learning makes such judges more reliable. The authors had LLaMA3-70B answer GSM8K math problems and then had GPT-4o rate those answers on a 1–10 scale. GPT-4o was chosen over Gemini1.5-Pro and Claude3.5-Sonnet because it showed less bias in a preliminary test. They compare zero-shot judging with two many-shot prompt designs. MSwR supplies demonstrations that include earlier model-generated evaluations as rationales. MSoR mostly lists example question–answer pairs without those evaluations. As the number of demonstrations grows, GPT-4o's ratings become more consistent across repeated runs and win more often against zero-shot ratings in pairwise comparisons. MSwR usually beats MSoR. The paper also identifies a "symbol bias", a preference for certain option labels, and reduces it by swapping labels and pooling the results. This suggests that long context windows offer a cheap way to stabilise automatic evaluation.

Key points

  • With zero-shot prompts, roughly half of GPT-4o's repeated ratings of the same LLaMA3-70B answers on the full GSM8K test set disagreed with each other.
  • The data are GSM8K, which has 8.5K grade-school problems split into 7.5K training and 1K test items. Answers were generated by LLaMA3-70B at temperature 0.7.
  • The number of in-context examples was scaled up to 512 (2⁹). Consistency stops improving around 256–512 shots, while evaluation quality keeps rising.
  • A random-symbol selection test showed that each LLM favours different labels:
    • GPT-4o is affected more by symbol bias.
    • Claude3.5-Sonnet and Gemini1.5-Pro are affected more by position bias and tend to favour later options.
  • Symbol bias was mitigated by swapping the answers behind labels A and B and combining the results. This mirrors an earlier fix for position bias that pools both answer orders.

Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.

 Fields, topics and concepts

📚 Shelves 4The field of science this work belongs to (OpenAlex fields).Computer Science · 16,667 worksArtificial Intelligence · 16,194 worksArts and Humanities · 16,039 worksLanguage and Linguistics · 15,891 works

🧭 Topics 1Research subjects and languages the work is about.Language models & pretraining · 3,320 works

Cite

Cite as Mingyang Song, Mao Zheng and Xuan Luo. 2025. Can Many-Shot In-Context Learning Help LLMs as Evaluators? A Preliminary Empirical Study. Proceedings of the 31st International Conference on Computational Linguistics.

The paper itself, not this page. BibTeX from the library's record of the work.

@inproceedings{song2025can,
  title = {{Can Many-Shot In-Context Learning Help LLMs as Evaluators? A Preliminary Empirical Study}},
  author = {Mingyang Song, Mao Zheng and Xuan Luo},
  booktitle = {Proceedings of the 31st International Conference on Computational Linguistics},
  year = {2025},
  url = {https://aclanthology.org/2025.coling-main.548/}
}

References

28 references, 19 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

  1. Rishabh Agarwal, Avi Singh, Lei M. Zhang, Bernd Bohnet, Stephanie C. Y. Chan, Ankesh Anand, Zaheer Abbas, Azade Nova, John D. Co-Reyes, Eric Chu, Feryal M. P. Behbahani, Aleksandra Faust, and Hugo Larochelle. 2024. Many-shot in-context learning. CoRR, abs/2404.11018.
  2. Anthropic. 2024. The claude 3 model family: Opus, sonnet, haiku. Technical Report. find
  3. Tom B. Brown, Benjamin Mann, Nick Ryder, Melanie Subbiah, Jared Kaplan, Prafulla Dhariwal, Arvind Neelakantan, Pranav Shyam, Girish Sastry, Amanda Askell, Sandhini Agarwal, Ariel Herbert-Voss, Gretchen Krueger, Tom Henighan, Rewon Child, Aditya Ramesh, Daniel M. Ziegler, Jeffrey Wu, Collin Burns, Pa… — Weak-to-Strong Generalization: Eliciting Strong Capabilities With Weak Supervision
  4. Yupeng Chang, Xu Wang, Jindong Wang, Yuan Wu, Kaijie Zhu, Hao Chen, Linyi Yang, Xiaoyuan Yi, Cunxiang Wang, Yidong Wang, Wei Ye, Yue Zhang, Yi Chang, Philip S. Yu, Qiang Yang, and Xing Xie. 2023. A survey on evaluation of large language models. CoRR, abs/2307.03109.
  5. Guiming Hardy Chen, Shunian Chen, Ziche Liu, Feng Jiang, and Benyou Wang. 2024. Humans or llms as the judge? A study on judgement biases. CoRR, abs/2402.10669.
  6. Aakanksha Chowdhery, Sharan Narang, Jacob Devlin, Maarten Bosma, Gaurav Mishra, Adam Roberts, Paul Barham, Hyung Won Chung, Charles Sutton, Sebastian Gehrmann, Parker Schuh, Kensen Shi, Sasha Tsvyashchenko, Joshua Maynez, Abhishek Rao, Parker Barnes, Yi Tay, Noam Shazeer, Vinodkumar Prabhakaran, Emi… — PaLM: Scaling Language Modeling with Pathways
  7. Karl Cobbe, Vineet Kosaraju, Mohammad Bavarian, Mark Chen, Heewoo Jun, Lukasz Kaiser, Matthias Plappert, Jerry Tworek, Jacob Hilton, Reiichiro Nakano, Christopher Hesse, and John Schulman. 2021. Training verifiers to solve math word problems. CoRR, abs/2110.14168.
  8. Jinlan Fu, See-Kiong Ng, Zhengbao Jiang, and Pengfei Liu. 2023a. Gptscore: Evaluate as you desire. CoRR, abs/2302.04166. find
  9. Xue-Yong Fu, Md. Tahmid Rahman Laskar, Cheng Chen, and Shashi Bhushan TN. 2023b. Are large language models reliable judges? A study on the factuality evaluation capabilities of llms. CoRR, abs/2311.00681. find
  10. Ryan Koo, Minhwa Lee, Vipul Raheja, Jong Inn Park, Zae Myung Kim, and Dongyeop Kang. 2023. Benchmarking cognitive biases in large language models as evaluators. CoRR, abs/2309.17012. find
  11. Mukai Li, Shansan Gong, Jiangtao Feng, Yiheng Xu, Jun Zhang, Zhiyong Wu, and Lingpeng Kong. 2023. In-context learning with many demonstration examples. CoRR, abs/2302.04931. — [arXiv:2302.04931](https://arxiv.org/abs/2302.04931) find
  12. Yiqi Liu, Nafise Sadat Moosavi, and Chenghua Lin. 2023. Llms as narcissistic evaluators: When ego inflates evaluation scores. CoRR, abs/2311.09766. find
  13. Jiaan Wang, Yunlong Liang, Fandong Meng, Haoxiang Shi, Zhixu Li, Jinan Xu, Jianfeng Qu, and Jie Zhou. 2023a. Is chatgpt a good NLG evaluator? A preliminary study. CoRR, abs/2303.04048.
  14. Peiyi Wang, Lei Li, Liang Chen, Dawei Zhu, Binghuai Lin, Yunbo Cao, Qi Liu, Tianyu Liu, and Zhifang Sui. 2023b. Large language models are not fair evaluators. CoRR, abs/2305.17926. find
  15. Yidong Wang, Zhuohao Yu, Zhengran Zeng, Linyi Yang, Cunxiang Wang, Hao Chen, Chaoya Jiang, Rui Xie, Jindong Wang, Xing Xie, Wei Ye, Shikun Zhang, and Yue Zhang. 2023c. Pandalm: An automatic evaluation benchmark for LLM instruction tuning optimization. CoRR, abs/2306.05087.
  16. Adian Liusie, Potsawee Manakul, and Mark J. F. Gales. 2023. Zero-shot NLG evaluation through pairware comparisons with llms. CoRR, abs/2307.07889. — [arXiv:2307.07889](https://arxiv.org/abs/2307.07889) find
  17. Minghao Wu and Alham Fikri Aji. 2023. Style over substance: Evaluation biases for large language models. CoRR, abs/2307.03025.
  18. Qingyu Lu, Baopu Qiu, Liang Ding, Liping Xie, and Dacheng Tao. 2023. Error analysis prompting enables human-like translation evaluation in large language models: A case study on chatgpt. CoRR, abs/2303.13809.
  19. Canwen Xu, Daya Guo, Nan Duan, and Julian J. McAuley. 2023. Baize: An open-source chat model with parameter-efficient tuning on self-chat data. In Proceedings of the 2023 Conference on Empirical Methods in Natural Language Processing, EMNLP 2023, Singapore, December 6-10, 2023, pages 6268– 6278. Ass…
  20. OpenAI. 2023. GPT-4 technical report. abs/2303.08774.
  21. CoRR, Baolin Peng, Chunyuan Li, Pengcheng He, Michel Galley, and Jianfeng Gao. 2023. Instruction tuning with GPT-4. CoRR, abs/2304.03277.
  22. Machel Reid, Nikolay Savinov, Denis Teplyashin, Dmitry Lepikhin, Timothy P. Lillicrap, Jean-Baptiste Alayrac, Radu Soricut, Angeliki Lazaridou, Orhan Firat, Julian Schrittwieser, Ioannis Antonoglou, Rohan Anil, Sebastian Borgeaud, Andrew M. Dai, Katie Millican, Ethan Dyer, Mia Glaese, Thibault Sotti… — Gemini 1.5: Unlocking multimodal understanding across millions of tokens of context
  23. Mingyang Song, Mao Zheng, and Xuan Luo. 2024. Counting-stars: A multi-evidence, position-aware, and scalable benchmark for evaluating long-context large language models. Preprint, arXiv:2403.11802.
  24. Wayne Xin Zhao, Kun Zhou, Junyi Li, Tianyi Tang, Xiaolei Wang, Yupeng Hou, Yingqian Min, Beichen Zhang, Junjie Zhang, Zican Dong, Yifan Du, Chen Yang, Yushuo Chen, Zhipeng Chen, Jinhao Jiang, Ruiyang Ren, Yifan Li, Xinyu Tang, Zikang Liu, Peiyu Liu, Jian-Yun Nie, and Ji-Rong Wen. 2023. A survey of l…
  25. Chujie Zheng, Hao Zhou, Fandong Meng, Jie Zhou, and Minlie Huang. 2024. Large language models are not robust multiple choice selectors. In The Twelfth International Conference on Learning Representations, ICLR 2024, Vienna, Austria, May 7-11, 2024. OpenReview.net. find
  26. Lianmin Zheng, Wei-Lin Chiang, Ying Sheng, Siyuan Zhuang, Zhanghao Wu, Yonghao Zhuang, Zi Lin, Zhuohan Li, Dacheng Li, Eric Xing, Hao Zhang, Joseph E Gonzalez, and Ion Stoica. 2023. Judging llm-as-a-judge with mt-bench and chatbot arena. In Advances in Neural Information Processing Systems, volume 3…
  27. Lianghui Zhu, Xinggang Wang, and Xinlong Wang. 2023. Judgelm: Fine-tuned large language models are scalable judges. CoRR, abs/2310.17631.
  28. Miles Turpin, Julian Michael, Ethan Perez, and Samuel R. Bowman. Language models don’t always say what they think: Unfaithful explanations in chain-of-thought prompting. In Advances in Neural Information Processing Systems 36: Annual Conference on Neural Information Processing Systems 2023. — Language Models Don't Always Say What They Think: Unfaithful Explanations in Chain-of-Thought Prompting

Updated

↑ back to top

This page in Markdown → https://md.research.elamur.ai/free/en/papers/song-2025-can-many-shot-context-learning/