Elamur Research⌕
☰
home / papers / 2026 / Scaling Test-Time Compute to Achieve IOI Gold Medal with Open-Weight Models

Scaling Test-Time Compute to Achieve IOI Gold Medal with Open-Weight Models

Paper · 2026 · Mehrzad Samadi, Aleksander Ficek, Sean Narenthiran et al.

Summary

OpenAI and Google DeepMind report gold-medal results at olympiad-level programming contests, but they have disclosed little about how they did it. Open-weight models have lagged behind. NVIDIA researchers present GenCluster, a test-time compute pipeline that aims for the same level openly and reproducibly.

For each IOI subtask, the pipeline works in four steps:

  • A model samples thousands of candidate programs in parallel and discards those that do not compile.
  • The model also writes random input generators and several validators. Inputs that pass validator agreement are used to run the candidates, and programs with matching outputs are grouped into behavioral clusters.
  • Pairwise LLM-as-a-judge tournaments rank the clusters.
  • Submissions are spread round-robin across the top clusters, within the 50-submissions-per-problem limit that human contestants face.

The authors built a subtask-level IOI-2025 benchmark and scored it with the official graders. On it, gpt-oss-120b with GenCluster reaches gold-medal performance, which the authors present as a first for an open-weight model. Scores rise steadily as more candidates are generated. The work gives a transparent baseline for studying LLM reasoning on competitive programming.

Key points

  • Benchmark: IOI-2025 covers six problems, each with 2–12 subtasks, for 39 subtasks in total. Its format mirrors the public IOI-2024 benchmark on Hugging Face, and evaluation uses the official grader programs with the original time and memory limits.
  • Models compared: gpt-oss-120b, gpt-oss-20b, DeepSeek-R1-0528 and Qwen3-235B-A22B.
    • gpt-oss-120b performs best and benefits most from extra generations.
    • The gpt-oss models gain from longer reasoning, while Qwen3 plateaus at around 48K tokens.
  • Tournament analysis:
    • Adding more tournament rounds per cluster representative raises the score, but gains level off after about 10 rounds.
    • The top-scoring solution lands within the top 50 clusters in 35 of 39 subtasks.
  • Cost: Generating 5,000 candidates for the benchmark takes about 7.3B tokens, and the ranking tournament takes roughly the same again.
  • Code release: The authors plan to publish the code in the NVIDIA-NeMo/Skills repository under recipes.

Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.

 Fields, topics and concepts

📚 Shelves 4The field of science this work belongs to (OpenAlex fields).Computer Science · 16,667 worksArtificial Intelligence · 16,194 worksArts and Humanities · 16,039 worksLanguage and Linguistics · 15,891 works

Cite

Cite as Mehrzad Samadi, Aleksander Ficek, Sean Narenthiran, Siddhartha Jain, Wasi Uddin Ahmad, Somshubra Majumdar, Vahid Noroozi and Boris Ginsburg. 2026. Scaling Test-Time Compute to Achieve IOI Gold Medal with Open-Weight Models. Proceedings of the 64th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers).

The paper itself, not this page. BibTeX from the library's record of the work. DOI 10.18653/v1/2026.acl-long.1532

@inproceedings{samadi2026scaling,
  title = {{Scaling Test-Time Compute to Achieve IOI Gold Medal with Open-Weight Models}},
  author = {Mehrzad Samadi, Aleksander Ficek, Sean Narenthiran, Siddhartha Jain, Wasi Uddin Ahmad, Somshubra Majumdar, Vahid Noroozi and Boris Ginsburg},
  booktitle = {Proceedings of the 64th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers)},
  year = {2026},
  doi = {10.18653/v1/2026.acl-long.1532},
  url = {https://doi.org/10.18653/v1/2026.acl-long.1532}
}

References

37 references, 17 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

  1. Jacob Austin, Augustus Odena, Maxwell Nye, Maarten Bosma, Henryk Michalewski, David Dohan, Ellen Jiang, Carrie Cai, Michael Terry, Quoc Le, and Charles Sutton. 2021. Program synthesis with large language models. Preprint, arXiv:2108.07732.
  2. Bradley Brown, Jordan Juravsky, Ryan Ehrlich, Ronald Clark, Quoc V. Le, Christopher Ré, and Azalia Mirhoseini. 2024. Large language monkeys: Scaling inference compute with repeated sampling. arXiv preprint. ArXiv:2407.21787 [cs.LG].
  3. Bei Chen, Fengji Zhang, Anh Nguyen, Daoguang Zan, Zeqi Lin, Jian-Guang Lou, and Weizhu Chen. 2023. Codet: Code generation with generated tests. In The Eleventh International Conference on Learning Representations. find
  4. Madanlal Musuvathi, and Jianfeng Gao. 2022. Fault-aware neural code rankers. Advances in Neural Information Processing Systems, 35:13419–13432. find
  5. International Olympiad in Informatics. 2025a. find
  6. Ioi 2025 tasks. https://ioi2025.obi.org.bo/tasks.htm l. Accessed: 2025-10-14. ↗
  7. International Olympiad in Informatics. 2025b. Results of IOI 2025. https://stats.ioinformatics.org/results/2025. Accessed: 2025-10-14. ↗
  8. Naman Jain, King Han, Alex Gu, Wen-Ding Li, Fanjia Yan, Tianjun Zhang, Sida Wang, Armando Solar-Lezama, Koushik Sen, and Ion Stoica. 2025. Livecodebench: Holistic and contamination free evaluation of large language models for code. In The Thirteenth International Conference on Learning Representatio…
  9. Mark Chen, Jerry Tworek, Heewoo Jun, Qiming Yuan, Henrique Ponde De Oliveira Pinto, Jared Kaplan, Harri Edwards, Yuri Burda, Nicholas Joseph, Greg Brockman, and 1 others. 2021. Evaluating large language models trained on code. arXiv preprint arXiv:2107.03374.
  10. Hung Le, Yue Wang, Akhilesh Deepak Gotmare, Silvio Savarese, and Steven Chu Hong Hoi. 2022. Coderl: Mastering code generation through pretrained models and deep reinforcement learning. Advances in Neural Information Processing Systems, 35:21314–21328.
  11. Xiusi Chen, Gaotang Li, Ziqi Wang, Bowen Jin, Cheng Qian, Yu Wang, Hongru Wang, Yu Zhang, Denghui Zhang, Tong Zhang, Hanghang Tong, and Heng Ji. 2025. Rm-r1: Reward modeling as reasoning. Preprint, arXiv:2505.02387.
  12. Rémi Leblond, Felix Gimeno, Florent Altché, Alaa Saade, Anton Ruddock, Corentin Tallec, George Powell, Jean-Bastien Grill, Maciej Mikuła, Matthias Lochbrunner, and 1 others. 2023. AlphaCode 2 technical report. Technical report, Google DeepMind. Accessed: 2025-01-14. find
  13. Karl Cobbe, Vineet Kosaraju, Mohammad Bavarian, Mark Chen, Heewoo Jun, Lukasz Kaiser, Matthias Plappert, Jerry Tworek, Jacob Hilton, Reiichiro Nakano, Christopher Hesse, and John Schulman. 2021. Training verifiers to solve math word problems. Preprint, arXiv:2110.14168.
  14. The Decoder. 2025. Openai’s ai system wins a gold medal-level score at the international olympiad in informatics 2025. https://the-decoder.com/ope nais-ai-system-wins-a-gold-medal-level-s core-at-the-international-olympiad-in-inf ormatics-2025/. Accessed: 2025-10-15. ↗
  15. DeepSeek-AI, Daya Guo, Dejian Yang, Haowei Zhang, Junxiao Song, Ruoyu Zhang, Runxin Xu, Qihao Zhu, Shirong Ma, Peiyi Wang, Xiao Bi, Xiaokang Zhang, Xingkai Yu, Yu Wu, Z. F. Wu, Zhibin Gou, Zhihong Shao, Zhuoshu Li, Ziyi Gao, and 181 others. 2025. Deepseek-r1: Incentivizing reasoning capability in ll…
  16. Yichao Fu, Xuewei Wang, Yuandong Tian, and Jiawei Zhao. 2025. Deep think with confidence. Preprint, arXiv:2508.15260. find
  17. Hugging Face. 2025. IOI Dataset (open-r1/ioi). https://huggingface.co/datasets/open-r1/ioi. Accessed: 2025-10-14. ↗
  18. Jeevana Priya Inala, Chenglong Wang, Mei Yang, Andres Codas, Mark Encarnación, Shuvendu Lahiri, Yujia Li, David Choi, Junyoung Chung, Nate Kushman, Julian Schrittwieser, Rémi Leblond, Tom Eccles, James Keeling, Felix Gimeno, Agustin Dal Lago, Thomas Hubert, Peter Choy, Cyprien de Masson d’Autume, Ig… — Competition-Level Code Generation with AlphaCode
  19. Zhong-Zhi Li, Duzhen Zhang, Ming-Liang Zhang, Jiaxin Zhang, Zengyan Liu, Yuxuan Yao, Haotian Xu, Junhao Zheng, Pei-Jie Wang, Xiuyi Chen, Yingying Zhang, Fei Yin, Jiahua Dong, Zhiwei Li, Bao-Long Bi, Ling-Rui Mei, Junfeng Fang, Zhijiang Guo, Le Song, and Cheng-Lin Liu. 2025. From system 1 to system 2…
  20. Hanzhao (Maggie) Lin and Heng-Tze Cheng. 2025. Gemini achieves gold-level performance at the international collegiate programming contest world finals. https://deepmind.google/discover/blo g/gemini-achieves-gold-level-performance -at-the-international-collegiate-program ming-contest-world-finals/. D… ↗
  21. Yantao Liu, Zijun Yao, Rui Min, Yixin Cao, Lei Hou, and Juanzi Li. 2025. Pairjudge rm: Perform best-of-n sampling with knockout tournament. arXiv preprint arXiv:2501.13007. — [arXiv:2501.13007](https://arxiv.org/abs/2501.13007) find
  22. Dakota Mahan, Duy Van Phung, Rafael Rafailov, Chase Blagden, Nathan Lile, Louis Castricato, Jan-Philipp Fränken, Chelsea Finn, and Alon Albalak. 2024. Generative reward models. Preprint, arXiv:2410.12832. find
  23. OpenAI, :, Ahmed El-Kishky, Alexander Wei, Andre Saraiva, Borys Minaiev, Daniel Selsam, David Dohan, Francis Song, Hunter Lightman, Ignasi Clavera, Jakub Pachocki, Jerry Tworek, Lorenz Kuhn, Lukasz Kaiser, Mark Chen, Max Schwarzer, Mostafa Rohaninejad, Nat McAleese, and 7 others. 2025. Competitive p… find
  24. OpenAI. 2024. Learning to reason with llms. https://openai.com/index/learning-to-reason-wit h-llms/. Accessed: YYYY-MM-DD. ↗
  25. OpenAI. 2025. gpt-oss-120b and gpt-oss-20b model card. Preprint, arXiv:2508.10925.
  26. Guilherme Penedo, Anton Lozhkov, Hynek Kydlíček, Loubna Ben Allal, Edward Beeching, Agustín Piqueres Lajarín, Quentin Gallouédec, Nathan Habib, Lewis Tunstall, and Leandro von Werra. 2025. Code-Forces Dataset (open-r1/codeforces). https://hu ggingface.co/datasets/open-r1/codeforces. Hugging Face re… find
  27. Ren, and Zhenru Zhang. 2024. Qwen2.5-math technical report: Toward mathematical expert model via self-improvement. arXiv preprint arXiv:2409.12122.
  28. Kexun Zhang, Danqing Wang, Jingtao Xia, William Yang Wang, and Lei Li. 2023. Algo: Synthesizing algorithmic programs with llm-generated oracle verifiers. Preprint, arXiv:2305.14591.
  29. Lunjun Zhang, Arian Hosseini, Hritik Bansal, Mehran Kazemi, Aviral Kumar, and Rishabh Agarwal. 2025a. Generative verifiers: Reward modeling as next-token prediction. Preprint, arXiv:2408.15240.
  30. Qiyuan Zhang, Fuyuan Lyu, Zexu Sun, Lei Wang, Weixu Zhang, Wenyue Hua, Haolun Wu, Zhihan Guo, Yufei Wang, Niklas Muennighoff, Irwin King, Xue Liu, and Chen Ma. 2025b. A survey on test-time scaling in large language models: What, how, where, and how well? Preprint, arXiv:2503.24235. find
  31. Zhuorui Zhao, Ruidi Qiu, Chao Lin, Grace Li Zhang, Bing Li, and Ulf Schlichtmann. 2025. Vrank: Enhancing verilog code generation from large language models via self-consistency. In 2025 26th International Symposium on Quality Electronic Design (ISQED), pages 1–7. IEEE. find
  32. Lin Shi, Chiyu Ma, Wenhua Liang, Xingjian Diao, Weicheng Ma, and Soroush Vosoughi. 2024. Judging the judges: A systematic study of position bias in llm-as-a-judge. arXiv preprint arXiv:2406.07791.
  33. Hung To, Minh Nguyen, and Nghi Bui. 2024. Functional overlap reranking for neural code generation. In Findings of the Association for Computational Linguistics: ACL 2024, pages 3686–3704. find
  34. Shubham Toshniwal, Ivan Sorokin, Aleksander Ficek, Ivan Moshkov, and Igor Gitman. 2025. Genselect: A generative approach to best-of-n. Preprint, arXiv:2507.17797. — [arXiv:2507.17797](https://arxiv.org/abs/2507.17797) find
  35. Jason Wei, Xuezhi Wang, Dale Schuurmans, Maarten Bosma, Brian Ichter, Fei Xia, Ed H. Chi, Quoc V. Le, and Denny Zhou. 2022. Chain-of-thought prompting elicits reasoning in large language models. In Advances in Neural Information Processing Systems (NeurIPS) 35, pages 24824–24837. Curran Associates,…
  36. Yangzhen Wu, Zhiqing Sun, Shanda Li, Sean Welleck, and Yiming Yang. 2025. Inference scaling laws: An empirical analysis of compute-optimal inference for problem-solving with language models. Preprint, arXiv:2408.00724.
  37. An Yang, Beichen Zhang, Binyuan Hui, Bofei Gao, Bowen Yu, Chengpeng Li, Dayiheng Liu, Jianhong Tu, Jingren Zhou, Junyang Lin, Keming Lu, Mingfeng Xue, Runji Lin, Tianyu Liu, Xingzhang find

Updated

↑ back to top

This page in Markdown → https://md.research.elamur.ai/free/en/papers/samadi-2026-scaling-test-time-compute-achieve/