Elamur Research⌕
☰
home / papers / 2026 / HyperEdit: Unlocking Instruction-based Text Editing in LLMs via Hypernetworks

HyperEdit: Unlocking Instruction-based Text Editing in LLMs via Hypernetworks

Paper · 2026 · Yiming Zeng, Jinghan Cao, Zexin Li et al.

Summary

LLM-based text editing often fails in two ways. Models may not carry out the user's instruction precisely, and they tend to rewrite parts of the text that should stay untouched, which can break code or degrade a corrected document. HyperEdit addresses both problems. A hypernetwork reads the instruction and the source text, encoded with a frozen sentence transformer, and produces LoRA-style low-rank weights for each request. Its GRU, local embeddings and MLP generate a dynamic matrix A_l, which is combined with a trainable static matrix B_l on top of frozen LLM weights. A difference-aware regularization loss focuses training on the tokens that actually change, so the many unchanged tokens do not dominate supervision. Experiments cover four domains: Code, LaTeX, DSL and Wikipedia. With only 3B parameters, HyperEdit beats the editing model FineEdit-Pro and the much larger Qwen-2.5-14B-Instruct. This suggests that adapting weights to each instruction is a practical route to precise, minimal editing.

Key points

  • Reported gains:
    • 9%–30% relative BLEU improvement on modified regions over FineEdit-Pro.
    • About 50% relative improvement over Qwen-2.5-14B-Instruct, using a 3B-parameter model.
  • Multi-turn editing on LaTeX: HyperEdit scores 0.3894 / 0.5182, compared with 0.2500 / 0.3729 for FineEdit-Pro. It also leads on Wikipedia, with 0.4665 / 0.6083 against 0.4446 / 0.5858.
  • Domain-specific weights: The authors ran a t-SNE analysis of the generated A_l matrices on 80 cases, 20 per domain.
    • The A_l matrices form separate clusters for each domain.
    • Average similarity within a domain is 1.49× higher than across domains.
    • B_l appears to act as a shared basis across domains.
  • Ablation results:
    • The hypernetwork alone raises LaTeX BLEU from 0.1278 to 0.1339 and DSL BLEU from 0.0356 to 0.0521.
    • The full model reaches 0.1706 on LaTeX.
    • On Code, the hypernetwork-only variant scores slightly higher (0.0504 vs 0.0492).
  • Limitations and release: Training does not include explicit multi-round optimization, and models of 14B parameters or larger were not tested in the multi-turn setting. The code is released at github.com/StuRinDQB/hyper_edit.

Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.

 Institutions

🎓University of Notre DameNotre Dame · United States59 works · 37,127 citations🎓Florida State UniversityTallahassee · United States10 works · 4,835 citations🎓University of California, RiversideRiverside · United States10 works · 420 citations🎓University of North Carolina at CharlotteCharlotte · United States8 works · 494 citations🎓University of ConnecticutStorrs · United States13 works · 357 citations🎓San Francisco State UniversitySan Francisco · United States2 works · 3 citations

 Fields, topics and concepts

📚 Shelves 5The field of science this work belongs to (OpenAlex fields).Computer Science · 18,614 worksArtificial Intelligence · 17,965 worksArts and Humanities · 17,450 worksLanguage and Linguistics · 17,296 worksInformation Systems · 1,730 works

🧭 Topics 1Research subjects and languages the work is about.Language models & pretraining · 3,894 works

Cite

Cite as Yiming Zeng, Jinghan Cao, Zexin Li, Wanhao Yu, Zhankai Ye, Dawei Xiang, Ting Hua, Xin Liu, Shangqian Gao and Tingting Yu. 2026. HyperEdit: Unlocking Instruction-based Text Editing in LLMs via Hypernetworks. Findings of the Association for Computational Linguistics: ACL 2026.

The paper itself, not this page. BibTeX from the library's record of the work. DOI 10.18653/v1/2026.findings-acl.22

@inproceedings{zeng2026hyperedit,
  title = {{HyperEdit: Unlocking Instruction-based Text Editing in LLMs via Hypernetworks}},
  author = {Yiming Zeng, Jinghan Cao, Zexin Li, Wanhao Yu, Zhankai Ye, Dawei Xiang, Ting Hua, Xin Liu, Shangqian Gao and Tingting Yu},
  booktitle = {Findings of the Association for Computational Linguistics: ACL 2026},
  year = {2026},
  doi = {10.18653/v1/2026.findings-acl.22},
  url = {https://doi.org/10.18653/v1/2026.findings-acl.22}
}

References

47 references, 11 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

  1. Bashar Alhafni and Nizar Habash. 2025. Enhancing text editing for grammatical error correction: Arabic as a case study. In Proceedings of the 63rd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 17892–17914, Vienna, Austria. Association for Computationa… find
  2. Anysphere. 2023. Cursor: Ai-powered code editor. Software. find
  3. Ekaterina Artemova, Jason S Lucas, Saranya Venkatraman, Jooyoung Lee, Sergei Tilga, Adaku Uchendu, and Vladislav Mikhailov. 2025. Beemo: Benchmark of expert-edited machine-generated outputs. In Proceedings of the 2025 Conference of the Nations of the Americas Chapter of the Association for Computati… find
  4. Jimmy Lei Ba, Jamie Ryan Kiros, and Geoffrey E Hinton. 2016. Layer normalization. arXiv preprint arXiv:1607.06450.
  5. Federico Cassano, Luisa Li, Akul Sethi, Noah Shinn, Abby Brennan-Jones, Jacob Ginesin, Edward Berman, George Chakhnashvili, Anton Lozhkov, Carolyn Jane Anderson, and Arjun Guha. 2024. Can it edit? evaluating the ability of large language models to follow code editing instructions. Preprint, arXiv:23… — [arXiv:2312.12450](https://arxiv.org/abs/2312.12450) find
  6. Adam Casson. 2023. Transformer flops. find
  7. Yiming Chen, Xianghu Yue, Xiaoxue Gao, Chen Zhang, Luis Fernando D’Haro, Robby T. Tan, and Haizhou Li. 2024. Beyond single-audio: Advancing multi-audio processing in audio large language models. In Findings of the Association for Computational Linguistics: EMNLP 2024, pages 10917–10930, Miami, Flori… find
  8. Abhimanyu Dubey, Abhinav Jauhri, Abhinav Pandey, Abhishek Kadian, Ahmad Al-Dahle, Aiesha Letman, Akhil Mathur, Alan Schelten, Amy Yang, Angela Fan, et al. 2024. The llama 3 herd of models. arXiv e-prints, pages arXiv–2407.
  9. Lishui Fan, Jiakun Liu, Zhongxin Liu, David Lo, Xin Xia, and Shanping Li. 2025. Exploring the capabilities of llms for code-change-related tasks. ACM Transactions on Software Engineering and Methodology, 34(6):1–36. find
  10. Shangqian Gao, Ting Hua, Reza Shirkavand, Chi-Heng Lin, Zhen Tang, Zhengao Li, Longge Yuan, Fangyi Li, Zeyu Zhang, Alireza Ganjdanesh, et al. 2025. Tomoe: Converting dense large language models to mixture-of-experts through dynamic structural pruning. arXiv preprint arXiv:2501.15316. — [arXiv:2501.15316](https://arxiv.org/abs/2501.15316) find
  11. Grammarly Inc. 2009. Grammarly: Writing assistant. Software. find
  12. David Ha, Andrew M. Dai, and Quoc V. Le. 2017. Hypernetworks. In International Conference on Learning Representations (ICLR). find
  13. Dan Hendrycks and Kevin Gimpel. 2016. Gaussian error linear units (gelus). arXiv preprint arXiv:1606.08415. find
  14. Edward J Hu, Yelong Shen, Phillip Wallis, Zeyuan Allen-Zhu, Yuanzhi Li, Shean Wang, Lu Wang, Weizhu Chen, et al. 2022. Lora: Low-rank adaptation of large language models. ICLR, 1(2):3.
  15. Hayate Iso, Chao Qiao, and Hang Li. 2020. Fact-based Text Editing. In Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics, pages 171–182, Online. Association for Computational Linguistics. find
  16. Yuelyu Ji, Zhuochun Li, Rui Meng, and Daqing He. 2025. Reason-to-rank: Distilling direct and comparative reasoning from large language models for document reranking. In Proceedings of the 48th International ACM SIGIR Conference on Research and Development in Information Retrieval, SIGIR ’25, page 23… find
  17. Philippe Laban, Jesse Vig, Marti Hearst, Caiming Xiong, and Chien-Sheng Wu. 2024. Beyond the chat: Executable and verifiable text-editing with llms. In Proceedings of the 37th Annual ACM Symposium on User Interface Software and Technology, UIST ’24, New York, NY, USA. Association for Computing Machi… find
  18. Dongfang Li, Zetian Sun, Baotian Hu, Zhenyu Liu, Xinshuo Hu, Xuebo Liu, and Min Zhang. 2024. Improving attributed text generation of large language models via preference learning. In Findings of the Association for Computational Linguistics: ACL 2024, pages 5079–5101, Bangkok, Thailand. Association… find
  19. Zhuochun Li, Yuelyu Ji, Rui Meng, and Daqing He. 2025. Learning from committee: Reasoning distillation from a mixture of teachers with peer-review. In Findings of the Association for Computational Linguistics: ACL 2025, pages 4190–4205. find
  20. Chin-Yew Lin. 2004. Rouge: A package for automatic evaluation of summaries. In Text Summarization Branches Out: Proceedings of the ACL-04 Workshop, pages 74–81.
  21. Junnan Liu, Hongwei Liu, Linchen Xiao, Ziyi Wang, Kuikun Liu, Songyang Gao, Wenwei Zhang, Songyang Zhang, and Kai Chen. 2025. Are your LLMs capable of stable reasoning? In Findings of the Association for Computational Linguistics: ACL 2025, pages 17594–17632, Vienna, Austria. Association for Computa… find
  22. Xiang Liu, Xueling Liu, Jing Diao, Mengyao Zheng, Jihe Li, Yongyi Xie, Kang Lai, Xiao Geng, Yijun Song, and Linshan Jiang. 2024. Novel truncated-rank graph-structured and tree-guided sparse linear mixed models for variable selection on genome-wide association studies. In 2024 IEEE International Conf… find
  23. Shishira R. Maiya, Anubhav Gupta, Matthew Gwilliam, Max Ehrlich, and Abhinav Shrivastava. 2024. Latent-inr: A flexible framework for implicit representations of videos with discriminative semantics. In European Conference on Computer Vision (ECCV). find
  24. Notion Labs Inc. 2022. Notion ai. Software. find
  25. Long Ouyang, Jeff Wu, Xu Jiang, Diogo Almeida, Carroll L. Wainwright, Pamela Mishkin, Chong Zhang, Sandhini Agarwal, Katarina Slama, Alex Ray, John Schulman, Jacob Hilton, Fraser Kelton, Luke Miller, Maddie Simens, Amanda Askell, Peter Welinder, Paul Christiano, Jan Leike, and Ryan Lowe. 2022. Train…
  26. Kishore Papineni, Salim Roukos, Todd Ward, and Wei-Jing Zhu. 2002. Bleu: a method for automatic evaluation of machine translation. In Proceedings of the 40th Annual Meeting of the Association for Computational Linguistics (ACL), pages 311–318.
  27. Maria Pilligua, Danna Xue, and Javier Vázquez-Corral. 2025. Hypernvd: Accelerating neural video decomposition via hypernetworks. In IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR). find
  28. Leigang Qu, Haochuan Li, Wenjie Wang, Xiang Liu, Juncheng Li, Liqiang Nie, and Tat-Seng Chua. 2025. Silmm: Self-improving large multimodal models for compositional text-to-image generation. In Proceedings of the Computer Vision and Pattern Recognition Conference, pages 18497–18508. find
  29. Vipul Raheja, Dimitris Alikaniotis, Vivek Kulkarni, Bashar Alhafni, and Dhruv Kumar. 2024. mEdIT: Multilingual text editing via instruction tuning. In Proceedings of the 2024 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies (Volum… find
  30. Vipul Raheja, Dhruv Kumar, Ryan Koo, and Dongyeop Kang. 2023. CoEdIT: Text editing by task-specific instruction tuning. In Findings of the Association for Computational Linguistics: EMNLP 2023, pages 5274–5291, Singapore. Association for Computational Linguistics. find
  31. Mathieu Ravaut, Aixin Sun, Nancy Chen, and Shafiq Joty. 2024. On context utilization in summarization with large language models. In Proceedings of the 62nd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 2764–2781, Bangkok, Thailand. Association for Co…
  32. Nils Reimers and Iryna Gurevych. 2019. Sentence-bert: Sentence embeddings using siamese bert-networks. In Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing (EMNLP-IJCNLP), pages 3982–3992.
  33. Bipasha Sen, Gaurav Singh, Aditya Agarwal, Rohith Agaram, Madhava Krishna, and Srinath Sridhar. 2024. Hypnerf: Learning improved nerf priors using a hypernetwork. In Advances in Neural Information Processing Systems (NeurIPS). find
  34. Lei Shu, Lei Luo, Jayant Hoskere, Yichong Zhu, Yicheng Liu, Shilin Tong, Jinxin Chen, and Lingyu Meng. 2024. Rewritelm: An instruction-tuned large language model for text rewriting. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 38, pages 18970–18980. find
  35. Hwanjun Song, Hang Su, Igor Shalyminov, Jason Cai, and Saab Mansour. 2024. FineSurE: Fine-grained summarization evaluation using LLMs. In Proceedings of the 62nd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 906–922, Bangkok, Thailand. Association for…
  36. Felix Stahlberg and Shankar Kumar. 2020. Seq2Edits: Sequence transduction using span-level edit operations. In Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing (EMNLP), pages 5147–5159, Online. Association for Computational Linguistics.
  37. Johannes von Oswald, Christian Henning, Benjamin F. Grewe, and João Sacramento. 2020. Continual learning with hypernetworks. In International Conference on Learning Representations (ICLR). 476 11 find
  38. Dingbang Wang, Yu Zhao, Sidong Feng, Zhaoxu Zhang, William G. J. Halfond, Chunyang Chen, Xiaoxia Sun, Jiangfan Shi, and Tingting Yu. 2024. Feedback-driven automated whole bug report reproduction for android apps. In Proceedings of the 33rd ACM SIG-SOFT International Symposium on Software Testing and… find
  39. Li Zeng, Zeming Liu, Chong Feng, Heyan Huang, and Yuhang Guo. 2025a. DocMEdit: Towards document-level model editing. In Findings of the Association for Computational Linguistics: ACL 2025, pages 19725–19743, Vienna, Austria. Association for Computational Linguistics. find
  40. Xidong Wu, Shangqian Gao, Zeyu Zhang, Zhenzhen Li, Runxue Bao, Yanfu Zhang, Xiaoqian Wang, and Heng Huang. 2024a. Auto-train-once: Controller network guided automatic network pruning from scratch. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 16163– 1617… find
  41. Yiming Zeng, Wanhao Yu, Zexin Li, Tao Ren, Yu Ma, Jinghan Cao, Xiyan Chen, and Tingting Yu. 2025b. Bridging the editing gap in LLMs: FineEdit for precise and targeted text modifications. In Findings of the Association for Computational Linguistics: EMNLP 2025, pages 2193–2206, Suzhou, China. Associa… find
  42. Xidong Wu, Sumin Jo, Yiming Zeng, Arun Das, Ting-He Zhang, Parth Patel, Yuanjing Wei, Lei Li, Shou-Jiang Gao, Jianqiu Zhang, Dexter Pratt, Yu-Chiao Chiu, and Yufei Huang. 2024b. Regulogpt: Harnessing gpt for end-to-end knowledge graph construction of molecular regulatory pathways. In 2024 IEEE EMBS… find
  43. Dawei Xiang, Wenyan Xu, Kexin Chu, Tianqi Ding, Zixu Shen, Yiming Zeng, Jianchang Su, and Wei Zhang. 2025. PromptSculptor: Multi-agent based text-to-image prompt optimization. In Proceedings of the 2025 Conference on Empirical Methods in Natural Language Processing: System Demonstrations, pages 774–… find
  44. Haopeng Zhang, Hayate Iso, Sairam Gurajada, and Nikita Bhutani. 2024. XATU: A fine-grained instruction-based benchmark for explainable text updates. In Proceedings of the 2024 Joint International Conference on Computational Linguistics, Language Resources and Evaluation (LREC-COLING 2024), pages 177… find
  45. Zhenjun Zhao. 2024. Balf: Simple and efficient blur aware local feature detector. In Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision, pages 3362–3372. find
  46. An Yang, Anfeng Li, Baosong Yang, Beichen Zhang, Binyuan Hui, Bo Zheng, Bowen Yu, Chang Gao, Chengen Huang, Chenxu Lv, Chujie Zheng, Dayiheng Liu, Fan Zhou, Fei Huang, Feng Hu, Hao Ge, Haoran Wei, Huan Lin, Jialong Tang, Jian Yang, Jianhong Tu, Jianwei Zhang, Jianxin Yang, Jiaxi Yang, Jing Zhou, Jin… — Qwen3 Technical Report
  47. An Yang, Baosong Yang, Beichen Zhang, Binyuan Hui, Bo Zheng, Bowen Yu, Chengyuan Li, Dayiheng Liu, Fei Huang, Haoran Wei, Huan Lin, Jian Yang, Jianhong Tu, Jianwei Zhang, Jianxin Yang, Jiaxi Yang, Jingren Zhou, Junyang Lin, Kai Dang, Keming Lu, Keqin Bao, Kexin Yang, Le Yu, Mei Li, Mingfeng Xue, Pei… find

Updated

↑ back to top

This page in Markdown → https://md.research.elamur.ai/free/en/papers/zeng-2026-hyperedit-unlocking-instruction-based-text/