Elamur Research⌕
☰
home / papers / 2026 / CoopQ: Cooperative Game Inspired Layerwise Mixed Precision Quantization for LLMs

CoopQ: Cooperative Game Inspired Layerwise Mixed Precision Quantization for LLMs

Paper · 2026 · Junchen Zhao, Ali Derakhshan, Jayden Hyman et al.

Summary

Mixed-precision post-training quantization of LLMs tends to break down below an average of four bits. Most existing schemes score each layer on its own and ignore how quantization errors in one layer interact with others. CoopQ treats layers as players in a cooperative game whose payoff is the change in per-token negative log-likelihood, which tracks Perplexity directly. Its estimator, SPQE, computes Monte-Carlo Shapley values in a progressive way. It starts from a uniformly 4-bit model and drops layers one at a time to 2 bits along sampled permutations. Removing layers outright, as pruning-based estimation does, gives much noisier estimates. The resulting sensitivities and pairwise interactions feed a quadratic surrogate of the loss. Choosing 2 or 4 bits per layer under a memory budget becomes a binary quadratic problem, which is linearized into a MILP and solved to global optimality. On Llama-3, Gemma-2 and Qwen-3 with the Quanto, HQQ and GPTQ backends, CoopQ lowers Perplexity by 20–80% compared with the strongest baseline. The advantage grows as the bit budget shrinks, which matters for deployment on constrained hardware.

Key points

  • Each layer is assigned either 2-bit or 4-bit precision. The allocation is found exactly through a MILP derived from the Shapley-based quadratic objective.
  • On Llama 3.1-8B with Quanto, pruning-based Shapley estimation makes Perplexity diverge once about 5 layers are removed. SPQE's progressive quantization degrades smoothly and yields lower-variance estimates.
  • Improvement levels off at around 90 sampled permutations, so the main experiments use 100.
  • A diagonal shrinkage setting of α = 0.5 performed best on Llama 3.2-3B (Quanto), with an average Perplexity of 2.79 × 10³. Results were fairly flat across α values.
  • The main cost is estimation: 100 SPQE samples for Llama-3.1-8B take about 18 hours on a single A40 GPU. This is a one-time cost per model.

Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.

 Institutions

🎓University of California, IrvineIrvine · United States43 works · 36,565 citations

 Fields, topics and concepts

📚 Shelves 5The field of science this work belongs to (OpenAlex fields).Computer Science · 18,614 worksArtificial Intelligence · 17,965 worksArts and Humanities · 17,450 worksLanguage and Linguistics · 17,296 worksComputer Vision and Pattern Recognition · 3,486 works

🧭 Topics 2Research subjects and languages the work is about.Language models & pretraining · 3,894 worksEfficient models & compression · 297 works

Cite

Cite as Junchen Zhao, Ali Derakhshan, Jayden Hyman, Junhao Dong, Sangeetha Abdu Jyothi and Ian Harris. 2026. CoopQ: Cooperative Game Inspired Layerwise Mixed Precision Quantization for LLMs. Findings of the Association for Computational Linguistics: ACL 2026.

The paper itself, not this page. BibTeX from the library's record of the work. DOI 10.18653/v1/2026.findings-acl.373

@inproceedings{zhao2026coopq,
  title = {{CoopQ: Cooperative Game Inspired Layerwise Mixed Precision Quantization for LLMs}},
  author = {Junchen Zhao, Ali Derakhshan, Jayden Hyman, Junhao Dong, Sangeetha Abdu Jyothi and Ian Harris},
  booktitle = {Findings of the Association for Computational Linguistics: ACL 2026},
  year = {2026},
  doi = {10.18653/v1/2026.findings-acl.373},
  url = {https://doi.org/10.18653/v1/2026.findings-acl.373}
}

References

39 references, 9 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

  1. Hicham Badri and Appu Shaji. 2023. Half-quadratic quantization of large machine learning models. find
  2. Tim Donkiewicz, Jasper van Doornmalen, Leon Eifler, Mohammed Ghannam, Ambros Gleixner, Christoph Graczyk, Katrin Halbig, Ivo Hedtke, Alexander Hoen, Christopher Hojny, Rolf van der Hulst, Dominik Kamp, Thorsten Koch, Kevin Kofler, and 12 others. 2024. The scip optimization suite 9.0. Preprint, arXiv… — [arXiv:2402.17702](https://arxiv.org/abs/2402.17702) find
  3. Tom Brown, Benjamin Mann, Nick Ryder, Melanie Subbiah, Jared D Kaplan, Prafulla Dhariwal, Arvind Neelakantan, Pranav Shyam, Girish Sastry, Amanda Askell, Sandhini Agarwal, Ariel Herbert-Voss, Gretchen Krueger, Tom Henighan, Rewon Child, Aditya Ramesh, Daniel Ziegler, Jeffrey Wu, Clemens Winter, and… find
  4. Zihan Chen, Bike Xie, Jundong Li, and Cong Shen. 2025. Channel-wise mixed-precision quantization for large language models. Preprint, arXiv:2410.13056. — [arXiv:2410.13056](https://arxiv.org/abs/2410.13056) find
  5. Jungwook Choi, Zhuo Wang, Swagath Venkataramani, Pierce I-Jen Chuang, Vijayalakshmi Srinivasan, and Kailash Gopalakrishnan. 2018. Pact: Parameterized clipping activation for quantized neural networks. arXiv preprint arXiv:1805.06085. — [arXiv:1805.06085](https://arxiv.org/abs/1805.06085) find
  6. Tim Dettmers, Ruslan Svirschevski, Vage Egiazarian, Denis Kuznedelev, Elias Frantar, Saleh Ashkboos, Alexander Borzunov, Torsten Hoefler, and Dan Alistarh. 2023. Spqr: A sparse-quantized representation for near-lossless llm weight compression. Preprint, arXiv:2306.03078. find
  7. Zhen Dong, Zhewei Yao, Yaohui Cai, Daiyaan Arfeen, Amir Gholami, Michael W. Mahoney, and Kurt Keutzer. 2019a. Hawq-v2: Hessian aware trace-weighted quantization of neural networks. Preprint, arXiv:1911.03852. — [arXiv:1911.03852](https://arxiv.org/abs/1911.03852) find
  8. Zhen Dong, Zhewei Yao, Amir Gholami, Michael W Mahoney, and Kurt Keutzer. 2019b. Hawq: Hessian aware quantization of neural networks with mixed-precision. In Proceedings of the IEEE/CVF international conference on computer vision, pages 293– 302. find
  9. Razvan-Gabriel Dumitru, Vikas Yadav, Rishabh Maheshwary, Paul-Ioan Clotan, Sathwik Tejaswi Madhusudhan, and Mihai Surdeanu. 2024a. Layer-wise quantization: A pragmatic and effective method for quantizing llms beyond integer bit-levels. Preprint, arXiv:2406.17415. — [arXiv:2406.17415](https://arxiv.org/abs/2406.17415) find
  10. Ron Banner, Yury Nahshan, and Daniel Soudry. 2019. Post training 4-bit quantization of convolutional networks for rapid-deployment. In Advances in Neural Information Processing Systems, volume 32. Curran Associates, Inc. find
  11. Razvan-Gabriel Dumitru, Vikas Yadav, Rishabh Maheshwary, Paul-Ioan Clotan, Sathwik Tejaswi Madhusudhan, and Mihai Surdeanu. 2024b. Layer-wise quantization: A pragmatic and effective method for quantizing llms beyond integer bit-levels. arXiv preprint arXiv:2406.17415. — [arXiv:2406.17415](https://arxiv.org/abs/2406.17415) find
  12. Suresh Bolusani, Mathieu Besançon, Ksenia Bestuzheva, Antonia Chmiela, João Dionísio, Elias Frantar, Saleh Ashkboos, Torsten Hoefler, and Dan Alistarh. 2023. Gptq: Accurate post-training quantization for generative pre-trained transformers. Preprint, arXiv:2210.17323. find
  13. Amir Gholami, Sehoon Kim, Zhen Dong, Zhewei Yao, Michael W Mahoney, and Kurt Keutzer. 2022. A survey of quantization methods for efficient neural network inference. In Low-Power Computer Vision, pages 291–326. Chapman and Hall/CRC. find
  14. Amirata Ghorbani and James Zou. 2020. Neuron shapley: Discovering the responsible neurons. In NeurIPS. find
  15. Aaron Grattafiori, Abhimanyu Dubey, Abhinav Jauhri, Abhinav Pandey, Abhishek Kadian, Ahmad Al-Dahle, Aiesha Letman, Akhil Mathur, Alan Schelten, Alex Vaughan, and 1 others. 2024. The llama 3 herd of models. arXiv preprint arXiv:2407.21783.
  16. Lanxiang Hu, Tajana Rosing, and Hao Zhang. 2025. TrimLLM: Progressive layer dropping for domain-specific LLMs. In Proceedings of the 63rd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 667–681, Vienna, Austria. Association for Computational Linguistics… find
  17. Itay Hubara, Yury Nahshan, Yair Hanani, Ron Banner, and Daniel Soudry. 2021. Accurate post training quantization with small calibration sets. In International Conference on Machine Learning, pages 4466–4475. find
  18. Benoit Jacob, Skirmantas Kligys, Bo Chen, Menglong Zhu, Matthew Tang, Andrew Howard, Hartwig Adam, and Dmitry Kalenichenko. 2018. Quantization and training of neural networks for efficient integer-arithmetic-only inference. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recogni… find
  19. Jason Kong, Lanxiang Hu, Flavio Ponzina, and Tajana Rosing. 2024. Tinyagent: Quantization-aware model compression and adaptation for on-device LLM agent deployment. In Workshop on Efficient Systems for Foundation Models II @ ICML2024. find
  20. Shiyao Li, Xuefei Ning, Ke Hong, Tengxuan Liu, Luning Wang, Xiuhong Li, Kai Zhong, Guohao Dai, Huazhong Yang, and Yu Wang. 2023. Llm-mq: Mixed-precision quantization for efficient llm deployment. In The Efficient Natural Language and Speech Processing Workshop with NeurIPS. find
  21. Ji Lin, Jiaming Tang, Haotian Tang, Shang Yang, Wei-Ming Chen, Wei-Chen Wang, Guangxuan Xiao, Xingyu Dang, Chuang Gan, and Song Han. 2024. Awq: Activation-aware weight quantization for on-device llm compression and acceleration. In Proceedings of Machine Learning and Systems, volume 6, pages 87–100. find
  22. Stephen Merity, Caiming Xiong, James Bradbury, and Richard Socher. 2016. Pointer sentinel mixture models. Preprint, arXiv:1609.07843.
  23. OpenAI, Josh Achiam, Steven Adler, Sandhini Agarwal, Lama Ahmad, Ilge Akkaya, Florencia Leoni Aleman, Diogo Almeida, Janko Altenschmidt, Sam Altman, Shyamal Anadkat, Red Avila, Igor Babuschkin, Suchir Balaji, Valerie Balcom, Paul Baltescu, Haiming Bao, Mohammad Bavarian, Jeff Belgum, and 262 others.… — GPT-4 Technical Report
  24. Optimum Quanto. 2024. Optimum quanto. [Online; accessed 2025-06-07]. find
  25. Jack W. Rae, Anna Potapenko, Siddhant M. Jayakumar, and Timothy P. Lillicrap. 2019. Compressive transformers for long-range sequence modelling. Preprint, arXiv:1911.05507.
  26. Colin Raffel, Noam Shazeer, Adam Roberts, Katherine Lee, Sharan Narang, Michael Matena, Yanqi Zhou, Wei Li, and Peter J. Liu. 2020. Exploring the limits of transfer learning with a unified text-to-text transformer. Journal of Machine Learning Research, 21(140):1–67.
  27. Utkarsh Saxena, Sayeh Sharify, Kaushik Roy, and Xin Wang. 2025. Resq: Mixed-precision quantization of large language models with low-rank residuals. Preprint, arXiv:2412.14363. — [arXiv:2412.14363](https://arxiv.org/abs/2412.14363) find
  28. Lloyd S Shapley. 1953. A value for n-person games. In Harold W. Kuhn and Albert W. Tucker, editors, Contributions to the Theory of Games II, pages 307– 317. Princeton University Press, Princeton. find
  29. Chuan Sun, Han Yu, Lizhen Cui, and Xiaoxiao Li. 2025. Efficient shapley value-based non-uniform pruning of large language models. Preprint, arXiv:2505.01731. — [arXiv:2505.01731](https://arxiv.org/abs/2505.01731) find
  30. Gemma Team, Morgane Riviere, Shreya Pathak, Pier Giuseppe Sessa, Cassidy Hardin, Surya Bhupatiraju, Léonard Hussenot, Thomas Mesnard, Bobak Shahriari, Alexandre Ramé, Johan Ferret, Peter Liu, Pouya Tafti, Abe Friesen, Michelle Casbon, Sabela Ramos, Ravin Kumar, Charline Le Lan, Sammy Jerome, and 179… find
  31. Hugo Touvron, Louis Martin, Kevin Stone, Peter Albert, Amjad Almahairi, Yasmine Babaei, Nikolay Bashlykov, Soumya Batra, Prajjwal Bhargava, Shruti Bhosale, Dan Bikel, Lukas Blecher, Cristian Canton Ferrer, Moya Chen, Guillem Cucurull, David Esiobu, Jude Fernandes, Jeremy Fu, Wenyin Fu, and 49 others… — Llama 2: Open Foundation and Fine-Tuned Chat Models
  32. Haocheng Xi, Changhao Li, Jianfei Chen, and Jun Zhu. 2023. Training transformers with 4-bit integers. Advances in Neural Information Processing Systems, 36:49146–49168. find
  33. Guangxuan Xiao, Ji Lin, Mickael Seznec, Hao Wu, Julien Demouth, and Song Han. 2023. Smoothquant: Accurate and efficient post-training quantization for large language models. In International conference on machine learning, pages 38087–38099. PMLR. find
  34. An Yang, Anfeng Li, Baosong Yang, Beichen Zhang, Binyuan Hui, Bo Zheng, Bowen Yu, Chang Gao, Chengen Huang, Chenxu Lv, Chujie Zheng, Dayiheng Liu, Fan Zhou, Fei Huang, Feng Hu, Hao Ge, Haoran Wei, Huan Lin, Jialong Tang, and 41 others. 2025. Qwen3 technical report. Preprint, arXiv:2505.09388.
  35. Zhewei Yao, Reza Yazdani Aminabadi, Minjia Zhang, Xiaoxia Wu, Conglong Li, and Yuxiong He. 2022. Zeroquant: Efficient and affordable post-training quantization for large-scale transformers. Advances in Neural Information Processing Systems, 35:27168– 27183.
  36. Zhewei Yao, Xiaoxia Wu, Cheng Li, Stephen Youn, and Yuxiong He. 2024. Exploring post-training quantization in llms from comprehensive study to low rank compensation. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 38, pages 19377–19385. find
  37. Yang Zhang, Yanfei Dong, and Kenji Kawaguchi. 2024. Investigating layer importance in large language models. Preprint, arXiv:2409.14381. find
  38. Junchen Zhao, Yurun Song, Simeng Liu, Ian G. Harris, and Sangeetha Abdu Jyothi. 2023. Lingualinked: A distributed large language model inference system for mobile devices. Preprint, arXiv:2312.00388. — [arXiv:2312.00388](https://arxiv.org/abs/2312.00388) find
  39. Xunyu Zhu, Jian Li, Yong Liu, Can Ma, and Weiping Wang. 2024. A survey on model compression for large language models. Transactions of the Association for Computational Linguistics, 12:1556–1577.

Updated

↑ back to top

This page in Markdown → https://md.research.elamur.ai/free/en/papers/zhao-2026-coopq-cooperative-game-inspired-layerwise/