Elamur Research⌕
☰
home / papers / 2026 / Garbage In, Reasoning Out? Why Benchmark Scores are Unreliable and What to Do About It

Garbage In, Reasoning Out? Why Benchmark Scores are Unreliable and What to Do About It

Paper · 2026 · Seyed Mahed Mousavi, Edoardo Cecchinato, Lucia Horníková et al.

Summary

The paper asks whether high LLM scores on social reasoning benchmarks show real inference or just flawed data and lenient scoring. The authors audit three popular benchmarks: SocialIQa, FauxPas-EAI and ToMi. Five models (GPT-3, ChatGPT/GPT-3.5, GPT-4, GPT-o1, LLaMA 3.1 8B Instruct) are used as probes rather than as subjects of evaluation. When a model's output looked odd or was marked wrong despite sound reasoning, human annotators checked whether the cause lay in the item or in the scoring rule. This revealed many structural, semantic and pragmatic defects, such as repeated items, unclear wording and implausible answer options. It also showed that string-based scoring rewards answer form over reasoning. A sizeable share of each model's errors comes from the benchmark itself. On cleaned data, scores rise, yet small rephrasings or changes in context still make performance swing widely. The authors conclude that benchmark claims about LLM social reasoning are unreliable. They call for process-oriented evaluation and release their audited data, annotation materials and tools.

Key points

  • Data used:
    • The SocialIQa dev set (1,954 questions; reported human accuracy 87%).
    • 100 ToMi stories generated with the official code (600 questions).
    • FauxPas-EAI, with 44 stories (22 faux pas, 22 controls) and four questions per story.
  • Baseline results under standard settings:
    • SocialIQa results go from a low of 56% for GPT-3 up to 81% for GPT-4.
    • On ToMi, accuracy ranges from 47% to 76%.
    • On FauxPas-EAI, GPT-4 scores best: 82% at question level and 39% at story level.
  • Cleaned data: GPT-o1 reaches 92% on the cleaned SocialIQa subset, above human accuracy. It still behaves unstably when the same question is reworded.
  • LLM-as-a-judge: GPT-4o agrees with human annotations on FauxPas-EAI at Cohen's κ = 0.89, compared with κ = 0.77 for Levenshtein distance. The authors see it only as a diagnostic aid, not a replacement for humans.
  • Limitations: The study covers only social reasoning and theory-of-mind tasks. The rephrasing tests rely on a single backbone model.

Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.

 Institutions

🎓University of TrentoTrento · Italy85 works · 15,524 citations🎓Masaryk UniversityBrno · Czechia10 works · 437 citations

 Fields, topics and concepts

📚 Shelves 5The field of science this work belongs to (OpenAlex fields).Computer Science · 16,667 worksArtificial Intelligence · 16,194 worksArts and Humanities · 16,039 worksLanguage and Linguistics · 15,891 worksComputer Vision and Pattern Recognition · 2,869 works

🧭 Topics 2Research subjects and languages the work is about.Reasoning · 1,426 worksEvaluation & benchmarks · 1,220 works

Cite

Cite as Seyed Mahed Mousavi, Edoardo Cecchinato, Lucia Horníková and Giuseppe Riccardi. 2026. Garbage In, Reasoning Out? Why Benchmark Scores are Unreliable and What to Do About It. Findings of the Association for Computational Linguistics: EACL 2026.

The paper itself, not this page. BibTeX from the library's record of the work. DOI 10.18653/v1/2026.findings-eacl.89

@inproceedings{mousavi2026garbage,
  title = {{Garbage In, Reasoning Out? Why Benchmark Scores are Unreliable and What to Do About It}},
  author = {Seyed Mahed Mousavi, Edoardo Cecchinato, Lucia Horníková and Giuseppe Riccardi},
  booktitle = {Findings of the Association for Computational Linguistics: EACL 2026},
  year = {2026},
  doi = {10.18653/v1/2026.findings-eacl.89},
  url = {https://doi.org/10.18653/v1/2026.findings-eacl.89}
}

References

22 references, 7 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

  1. Simon Baron-Cohen, Michelle O’riordan, Valerie Stone, Rosie Jones, and Kate Plaisted. 1999. Recognition of faux pas by normally developing children and children with asperger syndrome or high-functioning autism. Journal of autism and developmental disorders, 29:407–418. find
  2. Joint Conference on Natural Language Processing (EMNLP-IJCNLP), pages 5872–5877, Hong Kong, China. Association for Computational Linguistics. find
  3. Vladimir I. Levenshtein. 1966. Binary codes capable of correcting deletions, insertions and reversals. Soviet Physics Doklady, 10(8):707–710. find
  4. Wei Liu, Ruochen Zhou, Yiyun Deng, Yuzhen Huang, Junteng Liu, Yuntian Deng, Yizhe Zhang, and Junxian He. 2025. Learn to reason efficiently with adaptive length-based reward shaping. arXiv preprint arXiv:2505.15612. — [arXiv:2505.15612](https://arxiv.org/abs/2505.15612) find
  5. Anna Bavaresco, Raffaella Bernardi, Leonardo Bertolazzi, Desmond Elliott, Raquel Fernández, Albert Gatt, Esam Ghaleb, Mario Giulianelli, Michael Hanna, Alexander Koller, et al. 2024. Llms instead of human judges? a large scale empirical study across 20 nlp evaluation tasks. arXiv preprint arXiv:2406…
  6. Ziyi Liu, Soumya Sanyal, Isabelle Lee, Yongkang Du, Rahul Gupta, Yang Liu, and Jieyu Zhao. 2024. Self-contradictory reasoning evaluation and detection. In Findings of the Association for Computational Linguistics: EMNLP 2024, pages 3725–3742, Miami, Florida, USA. Association for Computational Lingui… find
  7. Tom Brown, Benjamin Mann, Nick Ryder, Melanie Subbiah, Jared D Kaplan, Prafulla Dhariwal, Arvind Neelakantan, Pranav Shyam, Girish Sastry, Amanda Askell, et al. 2020. Language models are few-shot learners. Advances in neural information processing systems, 33:1877–1901.
  8. Philipp Mondorf and Barbara Plank. 2024. Beyond accuracy: Evaluating the reasoning behavior of large language models - a survey. In First Conference on Language Modeling.
  9. Xanh Ho, Jiahao Huang, Florian Boudin, and Akiko Aizawa. 2025. Llm-as-a-judge: Reassessing the performance of llms in extractive qa. arXiv preprint arXiv:2504.11972. — [arXiv:2504.11972](https://arxiv.org/abs/2504.11972) find
  10. Wesley H. Holliday, Matthew Mandelkern, and Cedegao E. Zhang. 2024. Conditional and modal reasoning in large language models. In Proceedings of the 2024 Conference on Empirical Methods in Natural Language Processing, pages 3800–3821, Miami, Florida, USA. Association for Computational Linguistics. find
  11. Bowen Jiang, Yangxinyu Xie, Zhuoqun Hao, Xiaomeng Wang, Tanwi Mallick, Weijie J Su, Camillo Jose Taylor, and Dan Roth. 2024. A peek into token bias: Large language models are not yet genuine reasoners. In Proceedings of the 2024 Conference on Empirical Methods in Natural Language Processing, pages 4…
  12. Abhinav Joshi, Areeb Ahmad, Divyaksh Shukla, and Ashutosh Modi. 2025. Towards quantifying commonsense reasoning with mechanistic insights. In Proceedings of the 2025 Conference of the Nations of the Americas Chapter of the Association for Computational Linguistics: Human Language Technologies (Volum… find
  13. Matthew Le, Y-Lan Boureau, and Maximilian Nickel. 2019. Revisiting the evaluation of theory of mind through question answering. In Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Seyed Mahed Mousavi, Simone Alghisi, and Giuseppe Riccar… find
  14. Seyed Mahed Mousavi, Gabriel Roccabruna, Michela Lorandi, Simone Caldarella, and Giuseppe Riccardi. 2022. Evaluation of response generation models: Shouldn‘t it be shareable and replicable? In Proceedings of the 2nd Workshop on Natural Language Generation, Evaluation, and Metrics (GEM), pages 136–14… find
  15. Shramay Palta, Nishant Balepur, Peter Rankel, Sarah Wiegreffe, Marine Carpuat, and Rachel Rudinger. 2024. Plausibly problematic questions in multiple-choice benchmarks for commonsense reasoning. In Findings of the Association for Computational Linguistics: EMNLP 2024, pages 3451–3473, Miami, Florida… find
  16. Maarten Sap, Hannah Rashkin, Derek Chen, Ronan Le Bras, and Yejin Choi. 2019. Social IQa: Commonsense reasoning about social interactions. In Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processi… find
  17. Sneheel Sarangi, Maha Elgarf, and Hanan Salam. 2025. Decompose-ToM: Enhancing theory of mind reasoning in large language models through simulation and task decomposition. In Proceedings of the 31st International Conference on Computational Linguistics, pages 10228–10241, Abu Dhabi, UAE. Association…
  18. Natalie Shapira, Mosh Levy, Seyed Hossein Alavi, Xuhui Zhou, Yejin Choi, Yoav Goldberg, Maarten Sap, and Vered Shwartz. 2024. Clever hans or neural theory of mind? stress testing social reasoning in large language models. In Proceedings of the 18th Conference of the European Chapter of the Associati… find
  19. Natalie Shapira, Guy Zwirn, and Yoav Goldberg. 2023. How well do large language models perform on faux pas tests? In Findings of the Association for Computational Linguistics: ACL 2023, pages 10438–10451, Toronto, Canada. Association for Computational Linguistics. find
  20. Alon Talmor, Jonathan Herzig, Nicholas Lourie, and Jonathan Berant. 2019. CommonsenseQA: A question answering challenge targeting commonsense knowledge. In Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies,…
  21. Taylor Webb, Keith J Holyoak, and Hongjing Lu. 2023. Emergent analogical reasoning in large language models. Nature Human Behaviour, 7(9):1526–1541. find
  22. Zhen Zhang, Xuehai He, Weixiang Yan, Ao Shen, Chenyang Zhao, Shuohang Wang, Yelong Shen, and Xin Eric Wang. 2025. Soft thinking: Unlocking the reasoning potential of llms in continuous concept space. arXiv preprint arXiv:2505.15778.

Updated

↑ back to top

This page in Markdown → https://md.research.elamur.ai/free/en/papers/mousavi-2026-garbage-reasoning-out-why-benchmark/