Elamur Research⌕
☰
home / papers / 2026 / UniCreative: Unifying Long-form Logic and Short-form Sparkle via Reference-Free Reinforcement Learning

UniCreative: Unifying Long-form Logic and Short-form Sparkle via Reference-Free Reinforcement Learning

Paper · 2026 · Xiaolong Wei, Zerun Zhu, Simin Niu et al.

Summary

LLM creative writing faces two opposite problems. Long stories and scripts need global planning, or they drift off topic, repeat themselves and lose structure. Short pieces such as poems, slogans or blessings get worse when forced through rigid outlines, and models already tend toward bland, high-probability wording. The authors propose UniCreative, a reinforcement learning framework in which a single policy decides for itself whether to outline first (Plan-then-Write) or to answer directly.

Training skips SFT and needs no ground-truth references. AC-GenRM, a generative reward model, writes evaluation criteria specific to each query and gives fine-grained preference judgments on response pairs. ACPO, a GRPO-based policy optimization method, turns these judgments into rewards over groups of sampled outputs.

On Qwen3 models (1.7B, 4B, 8B), AC-GenRM agrees closely with expert evaluators, and ACPO improves both long- and short-form writing. Larger models also learn to choose the right mode on their own, which the authors call emergent meta-cognition. The work points to a cheaper route to creative alignment that avoids expensive human-written data.

Key points

  • Blessing short-text dataset: RL training raised the share of "excellent" ratings over the Thinking baselines.
    • Qwen3-1.7B: 64.2% → 90.0%
    • Qwen3-4B: 74.0% → 91.4%
    • Qwen3-8B: 68.0% → 93.6%, slightly above Claude-Sonnet-4.5 (93.2%)
  • Mode Discrimination Benchmark: the 4B and 8B models reach up to 96.4% accuracy on Easy items and 86.4% on Hard items when choosing between planning and direct generation.
  • Smallest model: Qwen3-1.7B becomes worse at choosing the mode after RL (64.2% → 57.0%), even though its writing quality improves. The authors attribute this to limited model capacity.
  • Compute cost: long-form training samples reach about 12.6k tokens, and training ran on 8 × NVIDIA H800 GPUs.
  • Limitations and code: the switch between modes is strictly binary, so "medium-form" texts remain an open problem. Code is available at github.com/weixiaolong94-hub/UniCreative.

Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.

 Institutions

🏢BaiduBeijing · China74 works · 15,263 citations🎓Renmin University of ChinaBeijing · China114 works · 16,118 citations🎓Beihang UniversityBeijing · China65 works · 8,452 citations

 Fields, topics and concepts

📚 Shelves 5The field of science this work belongs to (OpenAlex fields).Computer Science · 16,667 worksArtificial Intelligence · 16,194 worksArts and Humanities · 16,039 worksLanguage and Linguistics · 15,891 worksComputer Vision and Pattern Recognition · 2,869 works

Cite

Cite as Xiaolong Wei, Zerun Zhu, Simin Niu, Xingyu Zhang, Peiying Yu, Changxuan Xiao, Yuchen Li, Jicheng Yang, Zhejun Zhao, Chong Meng, Long Xia and Daiting Shi. 2026. UniCreative: Unifying Long-form Logic and Short-form Sparkle via Reference-Free Reinforcement Learning. Findings of the Association for Computational Linguistics: ACL 2026.

The paper itself, not this page. BibTeX from the library's record of the work. DOI 10.18653/v1/2026.findings-acl.1179

@inproceedings{wei2026unicreative,
  title = {{UniCreative: Unifying Long-form Logic and Short-form Sparkle via Reference-Free Reinforcement Learning}},
  author = {Xiaolong Wei, Zerun Zhu, Simin Niu, Xingyu Zhang, Peiying Yu, Changxuan Xiao, Yuchen Li, Jicheng Yang, Zhejun Zhao, Chong Meng, Long Xia and Daiting Shi},
  booktitle = {Findings of the Association for Computational Linguistics: ACL 2026},
  year = {2026},
  doi = {10.18653/v1/2026.findings-acl.1179},
  url = {https://doi.org/10.18653/v1/2026.findings-acl.1179}
}

References

56 references, 14 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

  1. Morris Alper, Moran Yanuka, Raja Giryes, and Gašper Beguš. 2025. Conlangcrafter: Constructing languages with a multi-hop llm pipeline. arXiv preprint arXiv:2508.06094. find
  2. Anirudh Atmakuru, Jatin Nainani, Rohith Siddhartha Reddy Bheemreddy, Anirudh Lakkaraju, Zonghai Yao, Hamed Zamani, and Haw-Shiuan Chang. 2024. Cs4: Measuring the creativity of large language models automatically by controlling the number of story-writing constraints. arXiv preprint arXiv:2410.04197. — [arXiv:2410.04197](https://arxiv.org/abs/2410.04197) find
  3. Yushi Bai, Shangqing Tu, Jiajie Zhang, Hao Peng, Xiaozhi Wang, Xin Lv, Shulin Cao, Jiazheng Xu, Lei Hou, Yuxiao Dong, and 1 others. 2025. Longbench v2: Towards deeper understanding and reasoning on realistic long-context multitasks. In Proceedings of the 63rd Annual Meeting of the Association for Co…
  4. Bin Chen, Xinzge Gao, Chuanrui Hu, Penghang Yu, Hua Zhang, and Bing-Kun Bao. 2025a. Reasongrm: Enhancing generative reward models through large reasoning models. arXiv preprint arXiv:2506.16712. — [arXiv:2506.16712](https://arxiv.org/abs/2506.16712) find
  5. Jianghao Chen, Wei Sun, Qixiang Yin, Lingxing Kong, Zhixing Tan, and Jiajun Zhang. 2025b. Ace-rl: Adaptive constraint-enhanced reward for long-form generation reinforcement learning. arXiv preprint arXiv:2509.04903. — [arXiv:2509.04903](https://arxiv.org/abs/2509.04903) find
  6. Xiusi Chen, Gaotang Li, Ziqi Wang, Bowen Jin, Cheng Qian, Yu Wang, Hongru Wang, Yu Zhang, Denghui Zhang, Tong Zhang, and 1 others. 2025c. Rmr1: Reward modeling as reasoning. arXiv preprint arXiv:2505.02387.
  7. Paul F Christiano, Jan Leike, Tom Brown, Miljan Martic, Shane Legg, and Dario Amodei. 2017. Deep reinforcement learning from human preferences. Advances in neural information processing systems, 30. find
  8. Xiaoxi Cui, Weihai Lu, Yu Tong, Yiheng Li, and Zhejun Zhao. 2025a. Diffusion-based multi-modal synergy interest network for click-through rate prediction. In Proceedings of the 48th International ACM SIGIR Conference on Research and Development in Information Retrieval, pages 581–591. find
  9. Xiaoxi Cui, Weihai Lu, Yu Tong, Yiheng Li, and Zhejun Zhao. 2025b. Multi-modal multi-behavior sequential recommendation with conditional diffusion-based feature denoising. In Proceedings of the 48th International ACM SIGIR Conference on Research and Development in Information Retrieval, pages 1593–1… find
  10. Angela Fan, Mike Lewis, and Yann Dauphin. 2018. Hierarchical neural story generation. arXiv preprint arXiv:1805.04833.
  11. Daniel Fein, Sebastian Russo, Violet Xiang, Kabir Jolly, Rafael Rafailov, and Nick Haber. 2025. Litbench: A benchmark and dataset for reliable evaluation of creative writing. arXiv preprint arXiv:2507.00769. find
  12. Zhaoyi Joey Hou, Bowei Alvin Zhang, Yining Lu, Bhiman Kumar Baghel, Anneliese Brei, Ximing Lu, Meng Jiang, Faeze Brahman, Snigdha Chaturvedi, Haw-Shiuan Chang, and 1 others. 2025. Creativi-typrism: A holistic benchmark for large language model creativity. arXiv preprint arXiv:2510.20091. — [arXiv:2510.20091](https://arxiv.org/abs/2510.20091) find
  13. Zhongzhan Huang, Shanshan Zhong, Pan Zhou, Shanghua Gao, Marinka Zitnik, and Liang Lin. 2025. A causality-aware paradigm for evaluating creativity of multimodal large language models. IEEE Transactions on Pattern Analysis and Machine Intelligence. find
  14. Kong, and Dawei Yin. 2025g. Rankelectra: Semi-supervised pre-training of learning-to-rank electra for web-scale search. In Proceedings of the 31st ACM SIGKDD Conference on Knowledge Discovery and Data Mining V. 1, pages 2415–2425. find
  15. Ruipeng Jia, Yunyi Yang, Yongbo Gai, Kai Luo, Shihao Huang, Jianhe Lin, Xiaoxi Jiang, and Guanjun Jiang. 2025. Writing-zero: Bridge the gap between non-verifiable tasks and verifiable rewards. arXiv e-prints, pages arXiv–2506. find
  16. Yuchen Li, Hao Zhang, Yongqi Zhang, Hengyi Cai, Mingxin Cai, Shuaiqiang Wang, Haoyi Xiong, Linghe Kong, Dawei Yin, and Lei Chen. 2025h. Rankexpert: A mixture of textual-and-behavioral experts for multi-objective learning-to-rank in web search. In Proceedings of the 31st ACM SIGKDD Conference on Know… find
  17. Ruizhe Li, Chiwei Zhu, Benfeng Xu, Xiaorui Wang, and Zhendong Mao. 2025a. Automated creativity evaluation for large language models: A reference-based approach. arXiv preprint arXiv:2504.15784. find
  18. Yanshu Li, Yi Cao, Hongyang He, Qisen Cheng, Xiang Fu, Xi Xiao, Tianyang Wang, and Ruixiang Tang. 2025b. M²IV: Towards efficient and fine-grained multimodal in-context learning via representation engineering. In Second Conference on Language Modeling. find
  19. Yanshu Li, Jianjiang Yang, Zhennan Shen, Ligong Han, Haoyan Xu, and Ruixiang Tang. 2025c. Catp: Contextually adaptive token pruning for efficient and enhanced multimodal in-context learning. arXiv preprint arXiv:2508.07871. — [arXiv:2508.07871](https://arxiv.org/abs/2508.07871) find
  20. Yanshu Li, Jianjiang Yang, Tian Yun, Pinyuan Feng, Jinfa Huang, and Ruixiang Tang. 2025d. Taco: Enhancing multimodal in-context learning via task mapping-guided sequence configuration. In Proceedings of the 2025 Conference on Empirical Methods in Natural Language Processing, pages 736–763.
  21. Yuchen Li, Hengyi Cai, Rui Kong, Xinran Chen, Jiamin Chen, Jun Yang, Haojie Zhang, Jiayi Li, Jiayi Wu, Yiqun Chen, and 1 others. 2025e. Towards ai search paradigm. arXiv preprint arXiv:2506.17188. — [arXiv:2506.17188](https://arxiv.org/abs/2506.17188) find
  22. Yuchen Li, Rui Kong, Zhonghao Lyu, Qiyang Li, Xinran Chen, Hengyi Cai, Lingyong Yan, Shuaiqiang Wang, Jiashu Zhao, Guangxu Zhu, and 1 others. 2026. Flexspec: Frozen drafts meet evolving targets in edge-cloud collaborative llm speculative decoding. IEEE Transactions on Mobile Computing. find
  23. Yuchen Li, Zhonghao Lyu, Yongqi Zhang, Hao Zhang, Tianhao Peng, Haoyi Xiong, Shuaiqiang Wang, Linghe Kong, Guihai Chen, and Dawei Yin. 2025f. S3prank: Towards satisfaction-oriented learning to rank with semi-supervised pre-training. IEEE Transactions on Knowledge and Data Engineering. find
  24. Yuchen Li, Haoyi Xiong, Qingzhong Wang, Linghe Kong, Hao Liu, Haifang Li, Jiang Bian, Shuaiqiang Wang, Guihai Chen, Dejing Dou, and 1 others. 2023. Coltr: Semi-supervised learning to rank with co-training and over-parameterization for web search. IEEE Transactions on Knowledge and Data Engineering,… find
  25. Yuchen Li, Haoyi Xiong, Yongqi Zhang, Jiang Bian, Tianhao Peng, Xuhong Li, Shuaiqiang Wang, Linghe Yuchen Li, Hao Zhang, Yongqi Zhang, Xinyu Ma, Wenwen Ye, Naifei Song, Shuaiqiang Wang, Haoyi Xiong, Dawei Yin, and Lei Chen. 2025i. M 2 oerank: Multi-objective mixture-of-experts enhanced ranking for s… find
  26. Xun Liang, Simin Niu, Zhiyu Li, Sensen Zhang, Hanyu Wang, Feiyu Xiong, Zhaoxin Fan, Bo Tang, Jihao Zhao, Jiawei Yang, and 1 others. 2025. Saferag: benchmarking security in retrieval-augmented generation of large language model. In Proceedings of the 63rd Annual Meeting of the Association for Computa… find
  27. Xun Liang, Mengwei Wang, Yuefeng Ma, and Simin Niu. 2026. Safe rag by rag: Untying the bell that rag rang with the rag hand. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 40, pages 31925–31933. find
  28. Jianxing Liao, Tian Zhang, Xiao Feng, Yusong Zhang, Rui Yang, Haorui Wang, Bosi Wen, Ziying Wang, and Runzhi Shi. 2025. Rlmr: Reinforcement learning with mixed rewards for creative writing. arXiv preprint arXiv:2508.18642. — [arXiv:2508.18642](https://arxiv.org/abs/2508.18642) find
  29. Nelson F Liu, Kevin Lin, John Hewitt, Ashwin Paranjape, Michele Bevilacqua, Fabio Petroni, and Percy Liang. 2024. Lost in the middle: How language models use long contexts. Transactions of the Association for Computational Linguistics, 12:157–173.
  30. Yang Liu, Dan Iter, Yichong Xu, Shuohang Wang, Ruochen Xu, and Chenguang Zhu. 2023. G-eval: Nlg evaluation using gpt-4 with better human alignment. arXiv preprint arXiv:2303.16634.
  31. Zijun Liu, Peiyi Wang, Runxin Xu, Shirong Ma, Chong Ruan, Peng Li, Yang Liu, and Yu Wu. 2025. Inference-time scaling for generalist reward modeling. arXiv preprint arXiv:2504.02495.
  32. Weihai Lu, Yu Tong, and Zhiqiu Ye. 2025. Dammfnd: Domain-aware multimodal multi-view fake news detection. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 39, pages 559–567. find
  33. Xun Lu. 2025. Writing-zero: Bridge the gap between non-verifiable problems and verifiable rewards. arXiv preprint arXiv:2506.00103. — [arXiv:2506.00103](https://arxiv.org/abs/2506.00103) find
  34. Dakota Mahan, Duy Van Phung, Rafael Rafailov, Chase Blagden, Nathan Lile, Louis Castricato, Jan-Philipp Fränken, Chelsea Finn, and Alon Albalak. 2024. Generative reward models. arXiv preprint arXiv:2410.12832. find
  35. Moran Mizrahi, Chen Shani, Gabriel Stanovsky, Dan Jurafsky, and Dafna Shahaf. 2025. Cooking up creativity: A cognitively-inspired approach for enhancing llm creativity through structured representations. arXiv preprint arXiv:2504.20643. — [arXiv:2504.20643](https://arxiv.org/abs/2504.20643) find
  36. Simin Niu, Mengwei Wang, Xun Liang, Zhiyu Li, Sensen Zhang, Shichao Song, Hanyu Wang, Jiawei Yang, Feiyu Xiong, and Chenyang Xi. 2025. Empowering large language models to set up knowledge retrieval indexing via self-learning. IEEE Transactions on Knowledge and Data Engineering. find
  37. Long Ouyang, Jeffrey Wu, Xu Jiang, Diogo Almeida, Carroll Wainwright, Pamela Mishkin, Chong Zhang, Sandhini Agarwal, Katarina Slama, Alex Ray, and 1 others. 2022. Training language models to follow instructions with human feedback. Advances in neural information processing systems, 35:27730–27744.
  38. Haoran Que, Feiyu Duan, Liqun He, Yutao Mou, Wangchunshu Zhou, Jiaheng Liu, Wenge Rong, Zekun Moore Wang, Jian Yang, Ge Zhang, and 1 others. 2024. Hellobench: Evaluating long text generation capabilities of large language models. arXiv preprint arXiv:2409.16191. — [arXiv:2409.16191](https://arxiv.org/abs/2409.16191) find
  39. Rafael Rafailov, Archit Sharma, Eric Mitchell, Christopher D Manning, Stefano Ermon, and Chelsea Finn. 2023. Direct preference optimization: Your language model is secretly a reward model. Advances in neural information processing systems, 36:53728–53741.
  40. Xiaolong Wei, Bo Lu, Xingyu Zhang, Zhejun Zhao, Dongdong Shen, Long Xia, and Dawei Yin. 2025b. Igniting creative writing in small language models: Llm-as-a-judge versus multi-agent refined rewards. In Proceedings of the 2025 Conference on Empirical Methods in Natural Language Processing, pages 17171…
  41. Yuhao Wu, Yushi Bai, Zhiqiang Hu, Roy Ka-Wei Lee, and Juanzi Li. 2025a. Longwriter-zero: Mastering ultra-long text generation via reinforcement learning. arXiv preprint arXiv:2506.18841. — [arXiv:2506.18841](https://arxiv.org/abs/2506.18841) find
  42. Yuning Wu, Jiahao Mei, Ming Yan, Chenliang Li, Shaopeng Lai, Yuran Ren, Zijia Wang, Ji Zhang, Mengyue Wu, Qin Jin, and 1 others. 2025b. Writingbench: A comprehensive benchmark for generative writing. arXiv preprint arXiv:2503.05244. — [arXiv:2503.05244](https://arxiv.org/abs/2503.05244) find
  43. Zikai Xiao, Fei Huang, Jianhong Tu, Jianhui Wei, Wen Ma, Yuxuan Zhou, Jian Wu, Bowen Yu, Zuozhu Liu, and Junyang Lin. 2025. Longweave: A long-form generation benchmark bridging real-world relevance and verifiability. arXiv preprint arXiv:2510.24345. find
  44. Zhuohan Xie, Trevor Cohn, and Jey Han Lau. 2023. The next chapter: A study of large language models in storytelling. arXiv preprint arXiv:2301.09790. find
  45. Ruihan Yang, Caiqi Zhang, Zhisong Zhang, Xinting Huang, Dong Yu, Nigel Collier, and Deqing Yang. 2025. Uncle: Benchmarking uncertainty expressions in long-form generation. In Proceedings of the 2025 Conference on Empirical Methods in Natural Language Processing, pages 30328–30344. find
  46. Lili Yao, Nanyun Peng, Ralph Weischedel, Kevin Knight, Dongyan Zhao, and Rui Yan. 2019. Plan-and-write: Towards better automatic storytelling. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 33, pages 7378–7385. find
  47. Aishik Sanyal, Samuel Schapiro, Sumuk Shashidhar, Royce Moon, Lav R Varshney, and Dilek Hakkani-Tur. 2025. Spark: A system for scientifically creative idea generation. arXiv preprint arXiv:2504.20090. find
  48. Peiying Yu, Guoxin Chen, and Jingjing Wang. 2025a. Table-critic: A multi-agent framework for collaborative criticism and refinement in table reasoning. In Proceedings of the 63rd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 17432–17451.
  49. Yu Tong, Weihai Lu, Xiaoxi Cui, Yifan Mao, and Zhejun Zhao. 2025. Dapt: Domain-aware prompt-tuning for multimodal fake news detection. In Proceedings of the 33rd ACM International Conference on Multimedia, pages 7902–7911. find
  50. Qiying Yu, Zheng Zhang, Ruofei Zhu, Yufeng Yuan, Xiaochen Zuo, Yu Yue, Weinan Dai, Tiantian Fan, Gaohong Liu, Lingjun Liu, and 1 others. 2025b. Dapo: An open-source llm reinforcement learning system at scale. arXiv preprint arXiv:2503.14476.
  51. Chenglong Wang, Yang Gan, Yifu Huo, Yongyu Mu, Qiaozhi He, Murun Yang, Bei Li, Tong Xiao, Chun-liang Zhang, Tongran Liu, and 1 others. 2025. Gram: A generative foundation reward model for reward generalization. arXiv preprint arXiv:2506.14175. — [arXiv:2506.14175](https://arxiv.org/abs/2506.14175) find
  52. Zhi Zeng, Jiaying Wu, Minnan Luo, Xiangzheng Kong, Zihan Ma, Guang Dai, and Qinghua Zheng. 2025a. Understand, refine and summarize: Multi-view knowledge progressive enhancement learning for fake news video detection. In Proceedings of the 33rd ACM International Conference on Multimedia, pages 9216–9… find
  53. Xiaolong Wei, Yuehu Dong, Xingliang Wang, Xingyu Zhang, Zhejun Zhao, Dongdong Shen, Long Xia, and Dawei Yin. 2025a. Beyond react: A planner-centric framework for complex tool-augmented llm reasoning. arXiv preprint arXiv:2511.10037. — [arXiv:2511.10037](https://arxiv.org/abs/2511.10037) find
  54. Zhi Zeng, Jiaying Wu, Minnan Luo, Herun Wan, Xiangzheng Kong, Zihan Ma, Guang Dai, and Qinghua Zheng. 2025b. Imol: Incomplete-modality-tolerant learning for multi-domain fake news video detection. In Proceedings of the 63rd Annual Meeting of the Association for Computational Linguistics (Volume 1: L…
  55. Lunjun Zhang, Arian Hosseini, Hritik Bansal, Mehran Kazemi, Aviral Kumar, and Rishabh Agarwal. 2024. Generative verifiers: Reward modeling as next-token prediction. arXiv preprint arXiv:2408.15240.
  56. Yunpu Zhao, Rui Zhang, Wenyi Li, and Ling Li. 2025. Assessing and understanding creativity in large language models. Machine Intelligence Research, 22(3):417–436. find

Updated

↑ back to top

This page in Markdown → https://md.research.elamur.ai/free/en/papers/wei-2026-unicreative-unifying-long-form-logic/