Elamur Research⌕
☰
home / papers / 2026 / A Universal Avoidance Method for Diverse Multi-branch Generation

A Universal Avoidance Method for Diverse Multi-branch Generation

Paper · 2026 · Kyeongman Park, Minha Jhang and Kyomin Jung

Summary

Sampling a generative model several times from the same prompt often yields near-duplicate results. This is a weakness for creative work such as story writing or illustration, which needs multi-branch diversity. Earlier fixes either operate per token or per pixel, which is costly, or depend on a specific architecture.

The authors describe autoregressive LMs, diffusion LMs and latent image diffusion models as one step-wise "Universal Generation Process". On top of it they build UAG (Universal Avoidance Generation). At each step, UAG measures how similar the current output and hidden state are to a bank of earlier generations. It then nudges the output along the negative gradient of that similarity, steering new samples away from old ones. A logistic schedule stresses a local (token/pixel-level) term early and a global (hidden-state) term later.

Experiments cover LLaMA-3B and LLaDA-8B on ReedsyPrompts and Stable Diffusion on COCO. UAG raises diversity in both automatic and human evaluation, with modest degeneration, at far lower cost than Avoidance Decoding and Procreate. The result is a cheap, plug-in diversity method that works across model families.

Key points

  • Method: The output is corrected as ŷ_t = y_t − ∇L_UAG, a weighted mix of local and global max-similarity losses with normalized gradients.
    • For LMs, the global gradient is projected through the output matrix W.
    • For diffusion models, the Jacobian is approximated by the identity.
    • Analytic gradients are used instead of autodiff.
  • Headline claims:
    • Up to 1.9× higher diversity, on average 1.43× on conventional metrics (BLEU, CLIPscore) and 1.19× on LLM-based judgments.
    • 4.4× faster decoding with about 1/64 of the FLOPs of prior state-of-the-art methods.
  • Efficiency in detail:
    • Avoidance Decoding needs about 126× more FLOPs and 3.73× more runtime than UAG.
    • Procreate needs about 1.25× more FLOPs and 5.01× more runtime.
  • Human evaluation (diversity, degeneration, creativity, coherence):
    • Second-best overall for LLaMA.
    • Best diversity and creativity for LLaDA.
    • Best coherence for Stable Diffusion.
  • Setup and limits:
    • Hyperparameters were chosen from the Pareto front of extensive sweeps.
    • On LLaMA-3B, diversity gains come with higher degeneration.
    • The method does not apply to models outside the framework, such as GANs.

Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.

 Institutions

🎓Seoul National UniversitySeoul · South Korea41 works · 45,671 citations

 Fields, topics and concepts

📚 Shelves 5The field of science this work belongs to (OpenAlex fields).Computer Science · 16,667 worksArtificial Intelligence · 16,194 worksArts and Humanities · 16,039 worksLanguage and Linguistics · 15,891 worksComputer Vision and Pattern Recognition · 2,869 works

Cite

Cite as Kyeongman Park, Minha Jhang and Kyomin Jung. 2026. A Universal Avoidance Method for Diverse Multi-branch Generation. Findings of the Association for Computational Linguistics: ACL 2026.

The paper itself, not this page. BibTeX from the library's record of the work. DOI 10.18653/v1/2026.findings-acl.777

@inproceedings{park2026universal,
  title = {{A Universal Avoidance Method for Diverse Multi-branch Generation}},
  author = {Kyeongman Park, Minha Jhang and Kyomin Jung},
  booktitle = {Findings of the Association for Computational Linguistics: ACL 2026},
  year = {2026},
  doi = {10.18653/v1/2026.findings-acl.777},
  url = {https://doi.org/10.18653/v1/2026.findings-acl.777}
}

References

30 references, 7 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

  1. Stability AI. 2024. Stable diffusion 3.5 large: Multimodal diffusion transformer for text-to-image generation. Model description released via Stability AI and Hugging Face model card. find
  2. Gabriele Corso, Yilun Xu, Valentin De Bortoli, Regina Barzilay, and Tommi Jaakkola. 2023. Particle guidance: non-iid diverse sampling with diffusion models. arXiv preprint arXiv:2310.13102. find
  3. Yuanhao Ding, Esteban Garces Arias, Meimingwei Li, Julian Rodemann, Matthias Aßenmacher, Danlu Chen, Gaojuan Fan, Christian Heumann, and Chongsheng Zhang. 2025. GUARD: Glocal uncertainty-aware robust decoding for effective and efficient open-ended text generation. In Findings of the Association for… find
  4. Abhimanyu Dubey, Abhinav Jauhri, Abhinav Pandey, Abhishek Kadian, Ahmad Al-Dahle, Aiesha Letman, Akhil Mathur, Alan Schelten, Amy Yang, Angela Fan, and 1 others. 2024. The llama 3 herd of models. arXiv e-prints, pages arXiv–2407.
  5. Angela Fan, Mike Lewis, and Yann Dauphin. 2018. Hierarchical neural story generation. arXiv preprint arXiv:1805.04833.
  6. Alisa Fortin and Seth Odoom. 2025. Announcing imagen 4 fast and the general availability of the imagen 4 family in the gemini api. Google Developers Blog. Accessed: 2025-09-29. find
  7. Esteban Garces Arias, Julian Rodemann, Meimingwei Li, Christian Heumann, and Matthias Aßenmacher. 2024. Adaptive contrastive search: Uncertainty-guided decoding for open-ended text generation. In Findings of the Association for Computational Linguistics: EMNLP 2024, pages 15060–15080, Miami, Florida… find
  8. Mario Giulianelli, Joris Baan, Wilker Aziz, Raquel Fernández, and Barbara Plank. 2023. What comes next? evaluating uncertainty in neural text generators against human production variability. In Proceedings of the 2023 Conference on Empirical Methods in Natural Language Processing, pages 14349–14371,… find
  9. Google DeepMind. 2024. Imagen 3. Technical report, Google DeepMind. Accessed: 2025-09-29. find
  10. Jonathan Ho, Ajay Jain, and Pieter Abbeel. 2020. Denoising diffusion probabilistic models. Advances in neural information processing systems, 33:6840– 6851. find
  11. Corinna Jaschek, Tom Beckmann, Jaime A Garcia, and William L Raffe. 2019. Mysterious murder-mcts-driven murder mystery generation. In 2019 IEEE Conference on Games (CoG), pages 1–8. IEEE. find
  12. Shuangqi Li, Hieu Le, Jingyi Xu, and Mathieu Salzmann. 2024. Enhancing compositional text-to-image generation with reliable random seeds. In The Thirteenth International Conference on Learning Representations. find
  13. Tsung-Yi Lin, Michael Maire, Serge Belongie, James Hays, Pietro Perona, Deva Ramanan, Piotr Dollár, and C. Lawrence Zitnick. 2014. Microsoft coco: Common objects in context. In Proceedings of the European Conference on Computer Vision (ECCV), pages 740–755. Springer.
  14. Jack Lu, Ryan Teehan, and Mengye Ren. 2024. Procreate, don’t reproduce! propulsive energy diffusion for creative generation. In European Conference on Computer Vision, pages 397–414. Springer. find
  15. Mariia Zameshina, Olivier Teytaud, and Laurent Najman. 2023. Diverse diffusion: Enhancing image diversity in text-to-image generation. arXiv preprint arXiv:2310.12583. — [arXiv:2310.12583](https://arxiv.org/abs/2310.12583) find
  16. Shen Nie, Fengqi Zhu, Zebin You, Xiaolu Zhang, Jingyang Ou, Jun Hu, Jun Zhou, Yankai Lin, Ji-Rong Wen, and Chongxuan Li. 2025. Large language diffusion models. arXiv preprint arXiv:2502.09992.
  17. Fengqi Zhu, Rongzhen Wang, Shen Nie, Xiaolu Zhang, Chunwei Wu, Jun Hu, Jun Zhou, Jianfei Chen, Yankai Lin, Ji-Rong Wen, and 1 others. 2025. Llada 1.5: Variance-reduced preference optimization for large language diffusion models. arXiv preprint arXiv:2505.19223.
  18. Kolby Nottingham, Ruo-Ping Dong, Ben Kasper, and Wesley N Kerr. 2024. Improving branching language via self-reflection. find
  19. OpenAI. 2023. Dall·e 3 system card. Technical report, OpenAI. Accessed: 2025-09-29. find
  20. OpenAI. 2025a. Gpt-5 system card. Technical report, OpenAI. Accessed: 2025-09-29. find
  21. OpenAI. 2025b. Introducing gpt-4.1 in the api. OpenAI Blog. Accessed: 2025-09-29. find
  22. Kyeongman Park, Minbeom Kim, and Kyomin Jung. 2024a. A character-centric creative story generation via imagination. arXiv preprint arXiv:2409.16667. find
  23. Kyeongman Park, Nakyeong Yang, and Kyomin Jung. 2024b. Longstory: Coherent, complete and length controlled long story generation. In Pacific-Asia Conference on Knowledge Discovery and Data Mining, pages 184–196. Springer. find
  24. Kyeongman Park, Nakyeong Yang, and Kyomin Jung. 2025. Avoidance decoding for diverse multi-branch story generation. arXiv preprint arXiv:2509.02170. find
  25. Seyedmorteza Sadat, Jakob Buhmann, Derek Bradley, Otmar Hilliges, and Romann M Weber. 2023. Cads: Unleashing the diversity of diffusion models through condition-annealed sampling. arXiv preprint arXiv:2310.17347. — [arXiv:2310.17347](https://arxiv.org/abs/2310.17347) find
  26. Stability AI. 2024. Stable diffusion 3. Stability AI News. Accessed: 2025-09-29. find
  27. Yixuan Su and Nigel Collier. 2022. Contrastive search is what you need for neural text generation. In arXiv preprint. Preprint. find
  28. Sean Welleck, Ilia Kulikov, Stephen Roller, Emily Dinan, Kyunghyun Cho, and Jason Weston. 2019. Neural text generation with unlikelihood training. arXiv preprint arXiv:1908.04319.
  29. Zhihua Wen, Zhiliang Tian, Wei Wu, Yuxin Yang, Yanqi Shi, Zhen Huang, and Dongsheng Li. 2023. Grove: a retrieval-augmented complex story generation framework with a forest of evidence. arXiv preprint arXiv:2310.05388. find
  30. An Yang, Anfeng Li, Baosong Yang, Beichen Zhang, Binyuan Hui, Bo Zheng, Bowen Yu, Chang Gao, Chengen Huang, Chenxu Lv, and 1 others. 2025. Qwen3 technical report. arXiv preprint arXiv:2505.09388.

Updated

↑ back to top

This page in Markdown → https://md.research.elamur.ai/free/en/papers/park-2026-universal-avoidance-method-diverse-multi/