Elamur Research⌕
☰
home / papers / 2026 / Tandem Training for Language Models

Tandem Training for Language Models

Paper · 2026 · Robert West, Ashton Anderson, Ece Kamar et al.

Summary

As language models grow more capable, their reasoning may become hard for humans or weaker systems to follow, which complicates oversight and collaboration. The authors avoid defining intelligibility explicitly through prompts, labels or custom rewards. Instead, they measure it by outcome. A strong model's solution counts as intelligible to a weaker one if the weaker model can take over at random points and still reach a correct answer. They call this property handoff robustness.

They adapt tandem training, an idea first tested in chess, to language models. During RL rollouts, a trained "senior" model and a frozen "junior" model take random turns producing tokens. Reward is given only when the jointly written solution succeeds. As a result, the senior is pushed toward reasoning that is both correct and easy for its partner to continue.

On GSM8K with Llama-2-7b variants, seniors quickly dropped notation the junior could not handle and shifted to the junior's language. Their accuracy stayed above the junior's baseline. The approach is lightweight and works with any architecture, which makes it a practical tool for building auditable AI systems.

Key points

  • The method treats intelligibility as continuability. Random handoffs to a frozen weaker model occur during RL rollouts, and success is rewarded only when the joint solution is correct. The approach builds on chess work by Hamade et al. (2024).
  • The experiments cover math reasoning on GSM8K in three settings. They use Llama-2-7b variants trained with plain REINFORCE and binary rewards, running on one or two A100 80 GB GPUs.
  • When senior and junior differed in math skill, senior-specific notation fell from 99% to 0% within 20 gradient updates. Senior accuracy stayed above the junior baseline throughout.
  • In a language-disparity setting, models were prompted in en, de, fr, bg and sr. Seniors adapted to the junior's language without their accuracy dropping below the junior baseline.
  • The study has several stated limitations: it uses one domain and one benchmark, assumes both models share a tokenizer, and relies on fixed i.i.d. handoffs. The authors release code, models and data.

Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.

 Institutions

🏢MicrosoftRedmond · United States378 works · 161,630 citations🎓University of TorontoToronto · Canada110 works · 179,061 citations🎓École Polytechnique Fédérale de LausanneLausanne · Switzerland48 works · 7,352 citations

 Fields, topics and concepts

📚 Shelves 4The field of science this work belongs to (OpenAlex fields).Computer Science · 18,614 worksArtificial Intelligence · 17,965 worksArts and Humanities · 17,450 worksLanguage and Linguistics · 17,296 works

🧭 Topics 1Research subjects and languages the work is about.Language models & pretraining · 3,894 works

Cite

Cite as Robert West, Ashton Anderson, Ece Kamar and Eric Horvitz. 2026. Tandem Training for Language Models. Proceedings of the 19th Conference of the European Chapter of the Association for Computational Linguistics (Volume 1: Long Papers).

The paper itself, not this page. BibTeX from the library's record of the work. DOI 10.18653/v1/2026.eacl-long.386

@inproceedings{west2026tandem,
  title = {{Tandem Training for Language Models}},
  author = {Robert West, Ashton Anderson, Ece Kamar and Eric Horvitz},
  booktitle = {Proceedings of the 19th Conference of the European Chapter of the Association for Computational Linguistics (Volume 1: Long Papers)},
  year = {2026},
  doi = {10.18653/v1/2026.eacl-long.386},
  url = {https://doi.org/10.18653/v1/2026.eacl-long.386}
}

References

25 references, 13 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

  1. Yuntao Bai, Andy Jones, Kamal Ndousse, Amanda Askell, Anna Chen, Nova DasSarma, Dawn Drain, Stanislav Fort, Deep Ganguli, Tom Henighan, and 1 others. 2022. Training a helpful and harmless assistant with reinforcement learning from human feedback. arXiv preprint arXiv:2204.05862. find
  2. Collin Burns, Amanda Askell, Long Chen, and 1 others. 2024. Weak-to-strong generalization: Eliciting strong capabilities with weak supervision. In Proceedings of the 41st International Conference on Machine Learning, volume 235 of Proceedings of Machine Learning Research. PMLR.
  3. Jiahao Cheng, Tiancheng Su, Jia Yuan, Guoxiu He, Jiawei Liu, Xinqi Tao, Jingwen Xie, and Huaxia Li. 2025. Chain-of-thought prompting obscures hallucination cues in large language models: An empirical evaluation. arXiv preprint arXiv:2506.17088. find
  4. Paul Christiano, Buck Shlegeris, and Dario Amodei. 2018. Supervising strong learners by amplifying weak experts. arXiv preprint arXiv:1810.08575. find
  5. Karl Cobbe, Vineet Kosaraju, Mohammad Bavarian, Mark Chen, Heewoo Jun, Lukasz Kaiser, Matthias Plappert, Jerry Tworek, Jacob Hilton, Reiichiro Nakano, and 1 others. 2021. Training verifiers to solve math word problems. arXiv preprint arXiv:2110.14168.
  6. Yihe Deng and Paul Mineiro. 2024. Flow-dpo: Improving llm mathematical reasoning through online multi-agent learning. arXiv preprint arXiv:2410.22304. — [arXiv:2410.22304](https://arxiv.org/abs/2410.22304) find
  7. Victor Sanh, Lysandre Debut, Julien Chaumond, and Thomas Wolf. 2019. DistilBERT, a distilled version of BERT: Smaller, faster, cheaper and lighter. arXiv preprint arXiv:1910.01108.
  8. Daya Guo, Dejian Yang, Haowei Zhang, Junxiao Song, Ruoyu Zhang, Runxin Xu, Qihao Zhu, Shirong Ma, Peiyi Wang, Xiao Bi, and 1 others. 2025. Deepseek-r1: Incentivizing reasoning capability in llms via reinforcement learning. arXiv preprint arXiv:2501.12948.
  9. John Schulman, Filip Wolski, Prafulla Dhariwal, Alec Radford, and Oleg Klimov. 2017. Proximal policy optimization algorithms. In Proceedings of the 34th International Conference on Machine Learning (ICML). find
  10. Karim Hamade, Reid McIlroy-Young, Siddhartha Sen, Jon Kleinberg, and Ashton Anderson. 2024. Designing skill-compatible ai: Methodologies and frameworks in chess. arXiv preprint arXiv:2405.05066. — [arXiv:2405.05066](https://arxiv.org/abs/2405.05066) find
  11. Zhihong Shao, Peiyi Wang, Qihao Zhu, Runxin Xu, Junxiao Song, and 1 others. 2024. Deepseekmath: Pushing the limits of mathematical reason-arXiv preprint ing in open language models. arXiv:2402.03300.
  12. Michael Hassid, Gabriel Synnaeve, Yossi Adi, and Roy Schwartz. 2025. Don’t overthink it. preferring shorter thinking chains for improved llm reasoning. arXiv preprint arXiv:2505.17813. — [arXiv:2505.17813](https://arxiv.org/abs/2505.17813) find
  13. Nitish Srivastava, Geoffrey Hinton, Alex Krizhevsky, Ilya Sutskever, and Ruslan Salakhutdinov. 2014. Dropout: A simple way to prevent neural networks from overfitting. The Journal of Machine Learning Research, 15(1):1929–1958.
  14. Geoffrey Hinton, Oriol Vinyals, and Jeff Dean. 2015. Distilling the knowledge in a neural network. arXiv preprint arXiv:1503.02531.
  15. Yiping Wang, Qing Yang, Zhiyuan Zeng, Liliang Ren, Lucas Liu, Baolin Peng, Hao Cheng, Xuehai He, Kuan Wang, Jianfeng Gao, Weizhu Chen, Shuohang Wang, Simon Shaolei Du, and Yelong Shen. 2025. Reinforcement learning for reasoning in large language models with one training example. arXiv preprint arXiv…
  16. Edward J Hu, Yelong Shen, Phillip Wallis, Zeyuan Allen-Zhu, Yuanzhi Li, Shean Wang, Lu Wang, Weizhu Chen, and 1 others. 2022. Lora: Low-rank adaptation of large language models. ICLR, 1(2):3.
  17. Geoffrey Irving, Paul Christiano, and Dario Amodei. 2018. Ai safety via debate. arXiv preprint arXiv:1805.00899.
  18. Armand Joulin, Edouard Grave, Piotr Bojanowski, Matthijs Douze, Hérve Jégou, and Tomas Mikolov. 2016a. FastText.zip: Compressing text classification models. arXiv preprint arXiv:1612.03651.
  19. Armand Joulin, Edouard Grave, Piotr Bojanowski, and Tomas Mikolov. 2016b. Bag of tricks for efficient text classification. arXiv preprint arXiv:1607.01759. find
  20. Cezary Kaliszyk, Josef Urban, Henryk Michalewski, and Miroslav Olšák. 2018. Reinforcement learning of theorem proving. Advances in Neural Information Processing Systems, 31. find
  21. Yihao Li, Jiayi Xin, Miranda Muqing Miao, Qi Long, and Lyle Ungar. 2025. The impact of language mixing on bilingual llm reasoning. arXiv preprint arXiv:2507.15849. find
  22. Hadar Lightman and 1 others. 2023. Let’s verify step by step. arXiv preprint arXiv:2305.20050.
  23. Xumeng Wen, Zihan Liu, Shun Zheng, Zhijian Xu, Shengyu Ye, Zhirong Wu, Xiao Liang, Yang Wang, Junjie Li, Ziming Miao, Jiang Bian, and Mao Yang. 2025. Reinforcement learning with verifiable rewards implicitly incentivizes correct reasoning in base llms. arXiv preprint arXiv:2506.14245. find
  24. Ronald J Williams. 1992. Simple statistical gradient-following algorithms for connectionist reinforcement learning. Machine learning, 8(3):229–256. find
  25. Yuyang Wu, Yifei Wang, Ziyu Ye, Tianqi Du, Stefanie Jegelka, and Yisen Wang. 2025. When more is less: Understanding chain-of-thought length in llms. arXiv preprint arXiv:2502.07266.

Updated

↑ back to top

This page in Markdown → https://md.research.elamur.ai/free/en/papers/west-2026-tandem-training-language-models/