Elamur Research⌕
☰
home / papers / 2026 / SuperWriter: Reflection-Driven Long-Form Generation with Large Language Models

SuperWriter: Reflection-Driven Long-Form Generation with Large Language Models

Paper · 2026 · Yuhao Wu, Yushi Bai, Zhiqiang Hu et al.

Summary

Large language models tend to produce long texts in one uninterrupted pass. The results often start well but lose coherence and contradict themselves as length grows. The authors address this by imitating how human authors work. Their SuperWriter-Agent pipeline splits writing into three stages:

  • Planning: discussion agents, inspired by the Story Workshop method, build an outline with themes, structure and word budgets.
  • Writing: for each paragraph, a Thinker step reasons before a Writer step produces text.
  • Refining: a Checker flags problems and an Editor fixes them.

Outputs of this pipeline, including the intermediate reasoning, become a stage-segmented SFT dataset. The dataset is used to train SuperWriter-LM, a 7B model built on Qwen2.5-7B-Instruct. The model is then aligned with a hierarchical DPO scheme, where Monte Carlo Tree Search passes judgments of the final text back to each intermediate step.

On WritingBench, SuperWriter-LM beats same-size models and, in some domains, even the 671B DeepSeek-R1. The work shows that explicit structured thinking data can help small models write better long-form text.

Key points

  • Ablation on WritingBench (average score): base Qwen2.5-7B-Instruct scores 7.43. Training only on the agent's final outputs gives 8.21, the three-stage SFT model 8.47, and adding hierarchical DPO 8.51 (ZH 8.6, EN 8.5).
  • Pairwise win rates: comparisons against seven baselines were judged by GPT-4.1. A human study used about 200 real user queries and three annotators, comparing against Writing-Model-Qwen-7B. The conclusion reports winning over 98% of real-user comparisons against top open-source baselines.
  • Inference cost: the model needs three sequential passes (plan → write → refine). Multi-round agent pipelines need roughly 30–40 calls per output.
  • Stated limitations:
    • added latency compared with single-pass models such as LongWriter and Suri;
    • limited world knowledge at 7B scale;
    • reliance on offline DPO rather than online RLHF.

Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.

 Fields, topics and concepts

📚 Shelves 5The field of science this work belongs to (OpenAlex fields).Computer Science · 16,667 worksArtificial Intelligence · 16,194 worksArts and Humanities · 16,039 worksLanguage and Linguistics · 15,891 worksComputer Vision and Pattern Recognition · 2,869 works

Cite

Cite as Yuhao Wu, Yushi Bai, Zhiqiang Hu, Juanzi Li and Roy Ka-Wei Lee. 2026. SuperWriter: Reflection-Driven Long-Form Generation with Large Language Models. Findings of the Association for Computational Linguistics: ACL 2026.

The paper itself, not this page. BibTeX from the library's record of the work. DOI 10.18653/v1/2026.findings-acl.428

@inproceedings{wu2026superwriter,
  title = {{SuperWriter: Reflection-Driven Long-Form Generation with Large Language Models}},
  author = {Yuhao Wu, Yushi Bai, Zhiqiang Hu, Juanzi Li and Roy Ka-Wei Lee},
  booktitle = {Findings of the Association for Computational Linguistics: ACL 2026},
  year = {2026},
  doi = {10.18653/v1/2026.findings-acl.428},
  url = {https://doi.org/10.18653/v1/2026.findings-acl.428}
}

References

64 references, 19 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

  1. 2025a. Deepseek-r1: Incentivizing reasoning capability in llms via reinforcement learning. Preprint, arXiv:2501.12948.
  2. Meta AI. 2025. The llama 4 herd: The beginning of a new era of natively multimodal ai innovation. find
  3. Chenxin An, Fei Huang, Jun Zhang, Shansan Gong, Xipeng Qiu, Chang Zhou, and Lingpeng Kong. 2024. Training-free long-context scaling of large language models. arXiv preprint arXiv:2402.17463. — [arXiv:2402.17463](https://arxiv.org/abs/2402.17463) find
  4. Yuntao Bai, Andy Jones, Kamal Ndousse, Amanda Askell, Anna Chen, Nova DasSarma, Dawn Drain, Stanislav Fort, Deep Ganguli, Tom Henighan, and 1 others. 2022. Training a helpful and harmless assistant with reinforcement learning from human feedback. arXiv preprint arXiv:2204.05862. find
  5. Yushi Bai, Xin Lv, Jiajie Zhang, Yuze He, Ji Qi, Lei Hou, Jie Tang, Yuxiao Dong, and Juanzi Li. 2024a. LongAlign: A recipe for long context alignment of large language models. In Findings of the Association for Computational Linguistics: EMNLP 2024, pages 1376–1395, Miami, Florida, USA. Association…
  6. Yushi Bai, Jiahao Ying, Yixin Cao, Xin Lv, Yuze He, Xiaozhi Wang, Jifan Yu, Kaisheng Zeng, Yijia Xiao, Haozhe Lyu, and 1 others. 2023. Benchmarking foundation models with language-model-as-an-examiner. Advances in Neural Information Processing Systems, 36:78142–78167. find
  7. Yushi Bai, Jiajie Zhang, Xin Lv, Linzhi Zheng, Siqi Zhu, Lei Hou, Yuxiao Dong, Jie Tang, and Juanzi Li. 2024b. Longwriter: Unleashing 10,000+ word generation from long context llms. arXiv preprint arXiv:2408.07055.
  8. DeepSeek-AI, Aixin Liu, Bei Feng, Bing Xue, Bingxuan Wang, Bochao Wu, Chengda Lu, Chenggang Zhao, Chengqi Deng, Chenyu Zhang, Chong Ruan, Damai Dai, Daya Guo, Dejian Yang, Deli Chen, Dongjie Ji, Erhang Li, Fangyun Lin, Fucong Dai, and 181 others. 2025b. Deepseek-v3 technical report. Preprint, arXiv:… find
  9. L Flower. 1981. A cognitive process theory of writing. Composition and communication. find
  10. Jinlan Fu, Shenzhen Huangfu, Hao Fei, Xiaoyu Shen, Bryan Hooi, Xipeng Qiu, and See-Kiong Ng. 2025. Chip: Cross-modal hierarchical direct preference optimization for multimodal llms. arXiv preprint arXiv:2501.16629. — [arXiv:2501.16629](https://arxiv.org/abs/2501.16629) find
  11. Yao Fu, Rameswar Panda, Xinyao Niu, Xiang Yue, Hannaneh Hajishirzi, Yoon Kim, and Hao Peng. 2024. Data engineering for scaling language models to 128k context. arXiv preprint arXiv:2402.10171.
  12. Leo Gao, John Schulman, and Jacob Hilton. 2022. Scaling laws for reward model overoptimization. Preprint, arXiv:2210.10760.
  13. Seraphina Goldfarb-Tarrant, Haining Feng, and Nanyun Peng. 2019. Plan, write, and revise: an interactive system for open-domain story generation. Preprint, arXiv:1904.02357. find
  14. Anne Becker. 2006. Revision: History, theory, and practice, pages 25–49. find
  15. Allan Gollins and Dedre Gentner. 2016. A framework for a cognitive theory of writing. In Cognitive processes in writing, pages 51–72. Routledge. find
  16. Shouyuan Chen, Sherman Wong, Liangjian Chen, and Yuandong Tian. 2023a. Extending context window of large language models via positional interpolation. arXiv preprint arXiv:2306.15595. find
  17. Chi Han, Qifan Wang, Wenhan Xiong, Yu Chen, Heng Ji, and Sinong Wang. 2023. Lm-infinite: Simple on-the-fly length generalization for large language models. arXiv preprint arXiv:2308.16137. — [arXiv:2308.16137](https://arxiv.org/abs/2308.16137) find
  18. Yukang Chen, Shengju Qian, Haotian Tang, Xin Lai, Zhijian Liu, Song Han, and Jiaya Jia. 2023b. Longlora: Efficient fine-tuning of long-context large language models. arXiv preprint arXiv:2309.12307.
  19. Shousheng Jia Haosheng Zou, Xiaowei Lv and Xiangzheng Zhang. 2024. 360-llama-factory. find
  20. Paul Christiano, Jan Leike, Tom B. Brown, Miljan Martic, Shane Legg, and Dario Amodei. 2023. Deep reinforcement learning from human preferences. Preprint, arXiv:1706.03741. find
  21. Pradeep Dasigi, Kyle Lo, Iz Beltagy, Arman Cohan, Noah A. Smith, and Matt Gardner. 2021. A dataset of information-seeking questions and answers anchored in research papers. In Proc. of NAACL, pages 4599– 4610.
  22. DeepSeek-AI, Daya Guo, Dejian Yang, Haowei Zhang, Junxiao Song, Ruoyu Zhang, Runxin Xu, Qihao Zhu, Shirong Ma, Peiyi Wang, Xiao Bi, Xiaokang Zhang, Xingkai Yu, Yu Wu, Z. F. Wu, Zhibin Gou, Zhihong Shao, Zhuoshu Li, Ziyi Gao, and 181 others. find
  23. Xinyu Hua and Lu Wang. 2020. Pair: Planning and iterative refinement in pre-trained transformers for long text generation. Preprint, arXiv:2010.02301. find
  24. Hongye Jin, Xiaotian Han, Jingfeng Yang, Zhimeng Jiang, Zirui Liu, Chia-Yuan Chang, Huiyuan Chen, and Xia Hu. 2024. Llm maybe longlm: Self-extend llm context window without tuning. arXiv preprint arXiv:2401.01325.
  25. Ishita Kumar, Snigdha Viswanathan, Sushrita Yerra, Alireza Salemi, Ryan A. Rossi, Franck Dernoncourt, Hanieh Deilamsalehy, Xiang Chen, Ruiyi Zhang, Shubham Agarwal, Nedim Lipka, Chien Van Nguyen, Thien Huu Nguyen, and Hamed Zamani. 2024. Longlamp: A benchmark for personalized long-form text generati… — [arXiv:2407.11016](https://arxiv.org/abs/2407.11016) find
  26. Yoonjoo Lee, Kyungjae Lee, Sunghyun Park, Dasol Hwang, Jaehyeon Kim, Hong-In Lee, and Moontae Lee. 2023. QASA: Advanced question answering on scientific articles. In Proc. of ICML, pages 19036– 19052. find
  27. Shanghaoran Quan, Tianyi Tang, Bowen Yu, An Yang, Dayiheng Liu, Bofei Gao, Jianhong Tu, Yichang Zhang, Jingren Zhou, and Junyang Lin. 2024. Language models can self-lengthen to generate long texts. arXiv preprint arXiv:2410.23933.
  28. Cheng Li, Mingyang Zhang, Qiaozhu Mei, Yaqing Wang, Spurthi Amba Hombaiah, Yi Liang, and Michael Bendersky. 2023. Teach llms to personalize - an approach inspired by writing education. ArXiv. find
  29. Haoran Que, Feiyu Duan, Liqun He, Yutao Mou, Wangchunshu Zhou, Jiaheng Liu, Wenge Rong, Zekun Moore Wang, Jian Yang, Ge Zhang, Junran Peng, Zhaoxiang Zhang, Songyang Zhang, and Kai Chen. 2024. Hellobench: Evaluating long text generation capabilities of large language models. Preprint, arXiv:2409.161… — [arXiv:2409.16191](https://arxiv.org/abs/2409.16191) find
  30. Weibin Liao, Xu Chu, and Yasha Wang. 2025. TPO: Aligning large language models with multi-branch & multi-step preference trees. In The Thirteenth International Conference on Learning Representations. find
  31. Siyang Liu, Naihao Deng, Sahand Sabour, Yilin Jia, Minlie Huang, and Rada Mihalcea. 2023. Task-adaptive tokenization: Enhancing long-form text generation efficacy in mental health and beyond. In Proc. of EMNLP, pages 15264–15281. find
  32. Shervin Minaee, Tomas Mikolov, Narjes Nikzad, Meysam Chenaghlu, Richard Socher, Xavier Amatriain, and Jianfeng Gao. 2024. Large language models: A survey. Preprint, arXiv:2402.06196.
  33. OpenAI, :, Aaron Hurst, Adam Lerer, Adam P. Goucher, Adam Perelman, Aditya Ramesh, Aidan Clark, AJ Ostrow, Akila Welihinda, Alan Hayes, Alec Radford, Aleksander M adry, Alex Baker-Whitcomb, Alex Beutel, Alex Borzunov, Alex Carney, Alex Chow, Alex Kirillov, and 401 others. 2024. Gpt-4o system card.…
  34. OpenAI. 2024. Learning to reason with llms. find
  35. OpenAI. 2025. Introducing gpt-4.1 in the api. Accessed: 2025-05-02. find
  36. Long Ouyang, Jeffrey Wu, Xu Jiang, Diogo Almeida, Carroll Wainwright, Pamela Mishkin, Chong Zhang, Sandhini Agarwal, Katarina Slama, Alex Ray, and 1 others. 2022. Training language models to follow instructions with human feedback. Advances in neural information processing systems, 35:27730–27744.
  37. Jupinder Parmar, Shrimai Prabhumoye, Joseph Jennings, Bo Liu, Aastha Jhunjhunwala, Zhilin Wang, Mostofa Patwary, Mohammad Shoeybi, and Bryan Catanzaro. 2024. Data, data everywhere: A guide for pretraining dataset construction. arXiv preprint arXiv:2407.06380. find
  38. Bowen Peng, Jeffrey Quesnelle, Honglu Fan, and Enrico Shippole. 2023. Yarn: Efficient context window extension of large language models. arXiv preprint arXiv:2309.00071. find
  39. Chau Minh Pham, Simeng Sun, and Mohit Iyyer. 2024. Suri: Multi-constraint instruction following for long-form text generation. arXiv preprint arXiv:2406.19371. — [arXiv:2406.19371](https://arxiv.org/abs/2406.19371) find
  40. Qwen, :, An Yang, Baosong Yang, Beichen Zhang, Binyuan Hui, Bo Zheng, Bowen Yu, Chengyuan Li, Dayiheng Liu, Fei Huang, Haoran Wei, Huan Lin, Jian Yang, Jianhong Tu, Jianwei Zhang, Jianxin Yang, Jiaxi Yang, Jingren Zhou, and 25 others. 2025. Qwen2.5 technical report. Preprint, arXiv:2412.15115. find
  41. Rafael Rafailov, Archit Sharma, Eric Mitchell, Christopher D Manning, Stefano Ermon, and Chelsea Finn. 2023. Direct preference optimization: Your language model is secretly a reward model. In Thirty-seventh Conference on Neural Information Processing Systems.
  42. Rafael Rafailov, Archit Sharma, Eric Mitchell, Christopher D Manning, Stefano Ermon, and Chelsea Finn. 2024. Direct preference optimization: Your language model is secretly a reward model. Advances in Neural Information Processing Systems, 36.
  43. Zeeshan Rasheed, Muhammad Waseem, Kai Kristian Kemell, Aakash Ahmad, Malik Abdul Sami, Jussi Rasku, Kari Systä, and Pekka Abrahamsson. 2025. Large language models for code generation: The practitioners perspective. Preprint, arXiv:2501.16998. — [arXiv:2501.16998](https://arxiv.org/abs/2501.16998) find
  44. Jeff Rasley, Samyam Rajbhandari, Olatunji Ruwase, and Yuxiong He. 2020. Deepspeed: System optimizations enable training deep learning models with over 100 billion parameters. In Proceedings of the 26th ACM SIGKDD International Conference on Knowledge Discovery & Data Mining, pages 3505–3506. find
  45. Alireza Salemi, Julian Killingback, and Hamed Zamani. 2025a. Expert: Effective and explainable evaluation of personalized long-form text generation. Preprint, arXiv:2501.14956. find
  46. Alireza Salemi, Cheng Li, Mingyang Zhang, Qiaozhu Mei, Weize Kong, Tao Chen, Zhuowan Li, Michael Bendersky, and Hamed Zamani. 2025b. Reasoning-enhanced self-training for long-form personalized text generation. Preprint, arXiv:2501.04167. — [arXiv:2501.04167](https://arxiv.org/abs/2501.04167) find
  47. Samuel Schmidgall, Yusheng Su, Ze Wang, Ximeng Sun, Jialian Wu, Xiaodong Yu, Jiang Liu, Zicheng Liu, and Emad Barsoum. 2025. Agent laboratory: Using llm agents as research assistants. Preprint, arXiv:2501.04227.
  48. John Schultz. 1977. The story workshop method: Writing from start to finish. College English, 39(4):381– 389. find
  49. Betty Sheflett. 1973. Story workshop as a method of teaching writing. College English, 35(2):141–160. find
  50. Kezhen Chen, Max Ryabinin, Tri Dao, Percy Liang, Christopher Ré, Irina Rish, and Ce Zhang. 2024. Redpajama: an open dataset for training large language models. NeurIPS Datasets and Benchmarks Track.
  51. Yuhao Wu, Yushi Bai, Zhiqing Hu, Shangqing Tu, Ming Shan Hee, Juanzi Li, and Roy Ka-Wei Lee. 2025a. Shifting long-context llms research from input to output. Preprint, arXiv:2503.04723. — [arXiv:2503.04723](https://arxiv.org/abs/2503.04723) find
  52. Daria Soboleva, Faisal Al-Khateeb, Robert Myers, Jacob R Steeves, Joel Hestness, and Nolan Dey. 2023. SlimPajama: A 627B token cleaned and deduplicated version of Yuhao Wu, Ming Shan Hee, Zhiqing Hu, and Roy Ka-RedPajama. https://www.cerebras.net/blog/ ↗
  53. Wei Lee. 2024. Longgenbench: Benchmarking long-slimpajama-a-627b-token-cleaned-and-deduplicated-version-of-redpajama. form generation in long context llms. arXiv preprint arXiv:2409.02076. — [arXiv:2409.02076](https://arxiv.org/abs/2409.02076) find
  54. Ivan Stelmakh, Yi Luan, Bhuwan Dhingra, and Ming-Wei Chang. 2022. ASQA: Factoid questions meet Yuning Wu, Jiahao Mei, Ming Yan, Chenliang Li, long-form answers. In Proc. of EMNLP, pages 8273– Shaopeng Lai, Yuran Ren, Zijia Wang, Ji Zhang, 8288. find
  55. Mengyue Wu, Qin Jin, and Fei Huang. 2025b. Writ-Haochen Tan, Zhijiang Guo, Zhan Shi, Lu Xu, Zhili ingbench: A comprehensive benchmark for genera-Liu, Yunlong Feng, Xiaoguang Li, Yasheng Wang, tive writing. Preprint, arXiv:2503.05244. — [arXiv:2503.05244](https://arxiv.org/abs/2503.05244) find
  56. Lifeng Shang, Qun Liu, and Linqi Song. 2024. ProxyQA: An alternative framework for evaluating long-Guangxuan Xiao, Yuandong Tian, Beidi Chen, Song form text generation with large language models. In Han, and Mike Lewis. 2023. Efficient streaming Proceedings of the 62nd Annual Meeting of the As-langu… find
  57. Wenhan Xiong, Jingyu Liu, Igor Molybog, Hejia Zhang, Prajjwal Bhargava, Rui Hou, Louis Martin, Rashi Pospelova Tatiana. 2021. The collaborative discussion Rungta, Karthik Abinav Sankararaman, Barlas Oguz, model: Developing writing skills through prewriting and 1 others. 2024. Effective long-context… find
  58. Qwen Team. 2025. Qwq-32b: Embracing the power of An Yang, Baosong Yang, Binyuan Hui, Bo Zheng, reinforcement learning. Bowen Yu, Chang Zhou, Chengpeng Li, Chengyuan Peiyi Wang, Lei Li, Zhihong Shao, Runxin Xu, Damai Li, Dayiheng Liu, Fei Huang, Guanting Dong, Hao-Dai, Yifei Li, Deli Chen, Yu Wu, and… find
  59. Tian. 2023. DOC: Improving long story coherence with detailed outline control. In Proc. of ACL, pages Qianyue Wang, Jinwu Hu, Zhengping Li, Yufeng Wang, 3378–3465. daiyuan li, Yu Hu, and Mingkui Tan. 2024b. Generating long-form story using dynamic hierarchi-Kevin Yang, Yuandong Tian, Nanyun Peng, an…
  60. Preprint, Klein. 2022. Re3: Generating longer stories with arXiv:2412.13575. recursive reprompting and revision. In Proc. of EMNLP, pages 4393–4479. — [arXiv:2412.13575](https://arxiv.org/abs/2412.13575) find
  61. Yidong Wang, Qi Guo, Wenjin Yao, Hongbo Zhang, Xin Zhang, Zhen Wu, Meishan Zhang, Xinyu Dai, Min Zhang, Qingsong Wen, Wei Ye, Shikun Zhang, and Yue Zhang. 2024c. Autosurvey: Large language models can automatically write surveys. In Proc. of NeurIPS. find
  62. Lili Yao, Nanyun Peng, Ralph Weischedel, Kevin Knight, Dongyan Zhao, and Rui Yan. 2019. Plan-and-write: Towards better automatic storytelling. Preprint, arXiv:1811.05701. find
  63. Maurice Weber, Daniel Y. Fu, Quentin Anthony, Yonatan Oren, Shane Adams, Anton Alexandrov, Xiaozhong Lyu, Huu Nguyen, Xiaozhe Yao, Virginia Adams, Ben Athiwaratkun, Rahul Chalamala, Peitian Zhang, Zheng Liu, Shitao Xiao, Ninglu Shao, Qiwei Ye, and Zhicheng Dou. 2024. Soaring from 4k to 400k: Extendi…
  64. Wenting Zhao, Xiang Ren, Jack Hessel, Claire Cardie, Yejin Choi, and Yuntian Deng. 2024. Wildchat: 1m chatgpt interaction logs in the wild. arXiv preprint arXiv:2405.01470.

Updated

↑ back to top

This page in Markdown → https://md.research.elamur.ai/free/en/papers/wu-2026-superwriter-reflection-driven-long-form/