Elamur Research⌕
☰
home / papers / 2026 / Improving the Throughput of Diffusion-based Large Language Models via a Training-Free Confidence-Aware Calibration

Improving the Throughput of Diffusion-based Large Language Models via a Training-Free Confidence-Aware Calibration

Paper · 2026 · Jucheng Shen, Gaurav Sarkar, Yeonju Ro et al.

Summary

Diffusion-based LLMs (dLLMs) such as LLaDA and DREAM generate text by repeatedly denoising masked positions, which makes inference slow. The prior accelerator, Fast-dLLM, decodes tokens in parallel using fixed block sizes, a fixed number of steps per block, a fixed commit threshold, and full-vocabulary sampling. The authors show that token confidence varies considerably both across blocks and across denoising steps. Based on this, they propose CadLLM, a training-free controller. It uses the average confidence of unmasked tokens to set four parameters on the fly:

  • block size
  • number of refinement steps
  • confidence threshold for committing tokens
  • size of the vocabulary subset used for softmax, with a repetition guard

Each policy is a simple clipped linear mapping, so the controller adds almost no overhead. It works with any model and with both prefix and dual KV caching.

The method was tested on a single H100 with GSM8K, MATH, MBPP, and HumanEval at generation lengths 256 and 512. Compared with Fast-dLLM, CadLLM raises throughput by 1.1–2.28× on LLaDA and 1.1–1.4× on DREAM, with comparable accuracy. This makes dLLM inference noticeably cheaper without any retraining.

Key points

  • Speedup on LLaDA-Instruct: throughput rises 1.1–2.28× over the Fast-dLLM threshold baseline. The largest gain is on HumanEval at g=256: 220.81 vs 96.84 tokens/s, with accuracy of 35.97% vs 37.19%.
  • Softmax cost: running softmax over the full ~50K-token vocabulary is roughly ten times slower than over a small subset. CadLLM therefore restricts softmax to the top-logit tokens, typically 35–1,000 of them.
  • Ablation of the threshold (GSM8K, g=256): removing the adaptive threshold cuts throughput by 71.6% and increases the number of function evaluations (NFE) by 289%.
  • Ablation of the vocabulary policy: dropping the adaptive vocabulary size lowers accuracy by 4.6 percentage points.
  • Limitations: results remain sensitive to hyperparameter tuning. The authors suggest learned schedules, such as RL-based or Bayesian optimization, as future work.

Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.

 Institutions

🎓The University of Texas at AustinAustin · United States109 works · 56,942 citations🏢IntelSanta Clara · United States15 works · 41,257 citations🎓Rice UniversityHouston · United States11 works · 1,850 citations

 Fields, topics and concepts

📚 Shelves 5The field of science this work belongs to (OpenAlex fields).Computer Science · 16,667 worksArtificial Intelligence · 16,194 worksArts and Humanities · 16,039 worksLanguage and Linguistics · 15,891 worksComputer Vision and Pattern Recognition · 2,869 works

🧭 Topics 1Research subjects and languages the work is about.Language models & pretraining · 3,320 works

Cite

Cite as Jucheng Shen, Gaurav Sarkar, Yeonju Ro, Sharath Nittur Sridhar, Zhangyang Wang, Aditya Akella and Souvik Kundu. 2026. Improving the Throughput of Diffusion-based Large Language Models via a Training-Free Confidence-Aware Calibration. Findings of the Association for Computational Linguistics: ACL 2026.

The paper itself, not this page. BibTeX from the library's record of the work. DOI 10.18653/v1/2026.findings-acl.478

@inproceedings{shen2026improving,
  title = {{Improving the Throughput of Diffusion-based Large Language Models via a Training-Free Confidence-Aware Calibration}},
  author = {Jucheng Shen, Gaurav Sarkar, Yeonju Ro, Sharath Nittur Sridhar, Zhangyang Wang, Aditya Akella and Souvik Kundu},
  booktitle = {Findings of the Association for Computational Linguistics: ACL 2026},
  year = {2026},
  doi = {10.18653/v1/2026.findings-acl.478},
  url = {https://doi.org/10.18653/v1/2026.findings-acl.478}
}

References

10 references, 4 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

  1. Jacob Austin, Daniel D. Johnson, Jonathan Ho, Daniel Tarlow, and Rianne van den Berg. 2021. Structured denoising diffusion models in discrete state-spaces. Preprint, arXiv:2107.03006.
  2. Andrew Campbell, Joe Benton, Valentin De Bortoli, Tom Rainforth, George Deligiannidis, and Arnaud Doucet. 2022. A continuous time framework for discrete denoising models. Preprint, arXiv:2205.14987. — [arXiv:2205.14987](https://arxiv.org/abs/2205.14987) find
  3. Angela Fan, Mike Lewis, and Yann Dauphin. 2018. Hierarchical neural story generation. arXiv preprint arXiv:1805.04833.
  4. Ari Holtzman, Jan Buys, Maxwell Forbes, Antoine Bosselut, David Golub, and Yejin Choi. 2018. Learning to write with cooperative discriminators. arXiv preprint arXiv:1805.06087.
  5. Pengxiang Li, Yefan Zhou, Dilxat Muhtar, Lu Yin, Shilin Yan, Li Shen, Yi Liang, Soroush Vosoughi, and Shiwei Liu. 2025. Diffusion language models know the answer before decoding. Preprint, arXiv:2508.19982. — [arXiv:2508.19982](https://arxiv.org/abs/2508.19982) find
  6. Guanxi Lu, Hao Mark Chen, Yuto Karashima, Zhican Wang, Daichi Fujiki, and Hongxiang Fan. 2026. Adablock-dllm: Semantic-aware diffusion llm inference via adaptive block size. Preprint, arXiv:2509.26432. — [arXiv:2509.26432](https://arxiv.org/abs/2509.26432) find
  7. Shen Nie, Fengqi Zhu, Zebin You, Xiaolu Zhang, Jingyang Ou, Jun Hu, Jun Zhou, Yankai Lin, Ji-Rong Wen, and Chongxuan Li. 2025. Large language diffusion models. Preprint, arXiv:2502.09992.
  8. Chengyue Wu, Hao Zhang, Shuchen Xue, Zhijian Liu, Shizhe Diao, Ligeng Zhu, Ping Luo, Song Han, and Enze Xie. 2025. Fast-dllm: Training-free acceleration of diffusion llm by enabling kv cache and parallel decoding. Preprint, arXiv:2505.22618. — [arXiv:2505.22618](https://arxiv.org/abs/2505.22618) find
  9. Jiacheng Ye, Zhihui Xie, Lin Zheng, Jiahui Gao, Zirui Wu, Xin Jiang, Zhenguo Li, and Lingpeng Kong. 2025a. Dream 7b: Diffusion large language models. Preprint, arXiv:2508.15487. find
  10. Jiacheng Ye, Zhihui Xie, Lin Zheng, Jiahui Gao, Zirui Wu, Xin Jiang, Zhenguo Li, and Lingpeng Kong. 2025b. Dream 7b: Diffusion large language models. arXiv preprint arXiv:2508.15487. find

Updated

↑ back to top

This page in Markdown → https://md.research.elamur.ai/free/en/papers/shen-2026-improving-throughput-diffusion-based-large/