Elamur Research
☰
home / papers / 2023 / Large Language Models Sensitivity to The Order of Options in Multiple-Choice Questions

Large Language Models Sensitivity to The Order of Options in Multiple-Choice Questions

Written for the Research library (claude-opus-5-5) from the paper text, in our own words: the license "arXiv — non-exclusive license to distribute" does not allow redistributing the text itself. Original — arxiv.org/pdf/2308.11483

Summary

This paper asks how much the order of answer options affects large language models on multiple-choice questions (MCQ). The authors treat MCQ as a test case for a broader issue: whether LLMs depend on the arrangement of prompt elements. They test GPT-4 and InstructGPT (text-davinci-003) on five benchmarks with 3, 4 or 5 options per question. For each benchmark, they measure a "sensitivity gap": the spread between the best and worst accuracy reachable by permuting options when the correct answer is known. The gap ranges from roughly 13% to 75%. It stays large in few-shot prompting, even when the demonstrations are similar examples. The authors propose that two things combine to cause this. First, the model is unsure between its two or three leading candidates. Second, a positional bias then decides among them. They also identify option layouts that strengthen or weaken this bias. Finally, they test two calibration methods, which improve accuracy by up to 8 percentage points. The work matters because standard MCQ evaluation may misjudge what LLMs can actually do.

Key points

  • Models and benchmarks. The study covers GPT-4 and InstructGPT (text-davinci-003). The benchmarks are CSQA (5 options), three MMLU subsets with 4 options (Abstract Algebra, High School Chemistry, Professional Law), and Logical Deduction from Big-Bench (3 options).
  • GPT-4 is more robust but still sensitive.
    • On Logical Deduction, GPT-4 scores 92.3% in the original order, yet reordering moves accuracy across about 13 points.
    • InstructGPT's gaps are far larger, for example about 70 points on Abstract Algebra.
  • Few-shot prompting does not fix the problem. Demonstrations were picked by Euclidean distance over Sentence-RoBERTa embeddings. They narrow the gap only slightly, and adding more of them does not reliably help.
  • Option placement shapes the bias. Putting the two strongest candidates in the first and last positions amplifies positional bias. Placing them next to each other reduces it.
  • Calibration has trade-offs. Majority voting over permutations is computationally expensive. Multiple evidence calibration (MEC) gives inconsistent gains, because asking for explanations can increase the model's uncertainty.

Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.

Fields: Other

Cited by

176 papers of our library cite it (of the 140673 whose bibliographies we read); 1 of them have a page here, 175 not yet.

Cite

The paper itself, not this page. BibTeX from the library's record of the work. DOI 10.48550/arxiv.2308.11483 · arXiv 2308.11483

@misc{pezeshkpour2023large,
  title = {{Large Language Models Sensitivity to The Order of Options in Multiple-Choice Questions}},
  author = {Pezeshkpour, Pouya and Hruschka, Estevam},
  year = {2023},
  doi = {10.48550/arxiv.2308.11483},
  eprint = {2308.11483},
  archivePrefix = {arXiv},
  url = {https://doi.org/10.48550/arxiv.2308.11483}
}

References

24 references, 11 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

  1. Rohan Anil, Andrew M Dai, Orhan Firat, Melvin Johnson, Dmitry Lepikhin, Alexandre Passos, Siamak Shakeri, Emanuel Taropa, Paige Bailey, Zhifeng Chen, et al. 2023. Palm 2 technical report. arXiv preprint arXiv:2305.10403. find our card
  2. Aakanksha Chowdhery, Sharan Narang, Jacob Devlin, Maarten Bosma, Gaurav Mishra, Adam Roberts, Paul Barham, Hyung Won Chung, Charles Sutton, Sebastian Gehrmann, et al. 2022. Palm: Scaling language modeling with pathways. arXiv preprint arXiv:2204.02311. find
  3. Peter Clark, Oren Etzioni, Tushar Khot, Daniel Khashabi, Bhavana Mishra, Kyle Richardson, Ashish Sabharwal, Carissa Schoenick, Oyvind Tafjord, Niket Tandon, et al. 2020. From ‘f’to ‘a’on the ny regents science exams: An overview of the aristo project. AI Magazine, 41(4):39–53. find
  4. Dan Hendrycks, Collin Burns, Steven Basart, Andy Zou, Mantas Mazeika, Dawn Song, and Jacob Steinhardt. 2020. Measuring massive multitask language understanding. arXiv preprint arXiv:2009.03300. find our card
  5. Yupeng Hou, Junjie Zhang, Zihan Lin, Hongyu Lu, Ruobing Xie, Julian McAuley, and Wayne Xin Zhao. 2023. Large language models are zero-shot rankers for recommender systems. arXiv preprint arXiv:2305.08845. find
  6. Valentin Liévin, Christoffer Egeberg Hother, and Ole Winther. 2022. Can large language models reason about medical questions? arXiv preprint arXiv:2207.08143. — [arXiv:2207.08143](https://arxiv.org/abs/2207.08143) find
  7. OpenAI. 2023. Gpt-4 technical report. arXiv preprint arXiv:2303.08774. find our card
  8. Long Ouyang, Jeffrey Wu, Xu Jiang, Diogo Almeida, Carroll Wainwright, Pamela Mishkin, Chong Zhang, Sandhini Agarwal, Katarina Slama, Alex Ray, et al. 2022. Training language models to follow instructions with human feedback. Advances in Neural Information Processing Systems, 35:27730–27744. find
  9. Fábio Perez and Ian Ribeiro. 2022. Ignore previous prompt: Attack techniques for language models. arXiv preprint arXiv:2211.09527. find our card
  10. Nils Reimers and Iryna Gurevych. 2019. Sentence-bert: Sentence embeddings using siamese bert-networks. arXiv preprint arXiv:1908.10084. find
  11. Matthew Richardson, Christopher JC Burges, and Erin Renshaw. 2013. Mctest: A challenge dataset for the open-domain machine comprehension of text. In Proceedings of the 2013 conference on empirical methods in natural language processing, pages 193– 203. find our card
  12. Joshua Robinson, Christopher Michael Rytting, and David Wingate. 2022. Leveraging large language models for multiple choice question answering. arXiv preprint arXiv:2210.12353. find our card
  13. Jaromir Savelka, Arav Agarwal, Christopher Bogart, and Majd Sakr. 2023. Large language models (gpt) struggle to answer multiple-choice questions about code. arXiv preprint arXiv:2303.08033. — [arXiv:2303.08033](https://arxiv.org/abs/2303.08033) find
  14. Aarohi Srivastava, Abhinav Rastogi, Abhishek Rao, Abu Awal Md Shoeb, Abubakar Abid, Adam Fisch, Adam R Brown, Adam Santoro, Aditya Gupta, Adrià Garriga-Alonso, et al. 2022. Beyond the imitation game: Quantifying and extrapolating the capabilities of language models. arXiv preprint arXiv:2206.04615. find our card
  15. Asa Cooper Stickland and Iain Murray. 2020. Diverse ensembles improve calibration. arXiv preprint arXiv:2007.04206. — [arXiv:2007.04206](https://arxiv.org/abs/2007.04206) find
  16. Alon Talmor, Jonathan Herzig, Nicholas Lourie, and Jonathan Berant. 2018. Commonsenseqa: A question answering challenge targeting commonsense knowledge. arXiv preprint arXiv:1811.00937. find
  17. Hugo Touvron, Thibaut Lavril, Gautier Izacard, Xavier Martinet, Marie-Anne Lachaux, Timothée Lacroix, Baptiste Rozière, Naman Goyal, Eric Hambro, Faisal Azhar, et al. 2023. Llama: Open and efficient foundation language models. arXiv preprint arXiv:2302.13971. find
  18. Jiongxiao Wang, Zichen Liu, Keun Hee Park, Muhao Chen, and Chaowei Xiao. 2023a. Adversarial demonstration attacks on large language models. arXiv preprint arXiv:2305.14950. — [arXiv:2305.14950](https://arxiv.org/abs/2305.14950) find
  19. Peiyi Wang, Lei Li, Liang Chen, Dawei Zhu, Binghuai Lin, Yunbo Cao, Qi Liu, Tianyu Liu, and Zhifang Sui. 2023b. Large language models are not fair evaluators. arXiv preprint arXiv:2305.17926. find
  20. Alexander Wei, Nika Haghtalab, and Jacob Steinhardt. 2023. Jailbroken: How does llm safety training fail? arXiv preprint arXiv:2307.02483. find our card
  21. Yotam Wolf, Noam Wies, Yoav Levine, and Amnon Shashua. 2023. Fundamental limitations of alignment in large language models. arXiv preprint arXiv:2304.11082. find
  22. Zihao Zhao, Eric Wallace, Shi Feng, Dan Klein, and Sameer Singh. 2021. Calibrate before use: Improving few-shot performance of language models. In International Conference on Machine Learning, pages 12697–12706. PMLR. find our card
  23. Kaijie Zhu, Jindong Wang, Jiaheng Zhou, Zichen Wang, Hao Chen, Yidong Wang, Linyi Yang, Wei Ye, Neil Zhenqiang Gong, Yue Zhang, et al. 2023. Promptbench: Towards evaluating the robustness of large language models on adversarial prompts. arXiv preprint arXiv:2306.04528. find our card
  24. Andy Zou, Zifan Wang, J Zico Kolter, and Matt Fredrikson. 2023. Universal and transferable adversarial attacks on aligned language models. arXiv preprint arXiv:2307.15043. find our card