Elamur Research⌕
☰
home / papers / 2026 / PaperMentor: A Human-Centered Multi-Agent Writing Tutor for AI Research Papers in Overleaf

PaperMentor: A Human-Centered Multi-Agent Writing Tutor for AI Research Papers in Overleaf

Paper · 2026 · Jiarui Liu, Terry Jingchen Zhang, Ryan Faulkner et al.

Summary

Early-career AI researchers often lack experienced mentors who can critique their drafts. Existing tools only partly fill that gap: grammar checkers fix individual sentences, and automated reviewers mostly assign quality judgments. PaperMentor is an open-source assistant that posts mentor-style feedback as inline comments directly in Overleaf. It never rewrites the text, so authors keep full control of their revisions.

The system works in three stages:

  1. It merges the LaTeX project, detects the paper type and maps sections to review domains, taking into account the target venue and an optional role model paper.
  2. Twelve specialized agents review the paper. Each agent draws on a curated library of expert writing advice.
  3. The system removes duplicate comments and anchors each remaining one to a span in the source files.

In a user study, 14 AI researchers rated comments on 80 papers. Most comments were judged actionable, and about two-thirds were judged valid. Both scores were significantly higher than those of the same LLM prompted without the skill library. The result suggests that encoding expert knowledge in modular form can make LLM writing feedback more useful.

Key points

  • The skill library has more than 40 markdown skill files (over 16,000 words) in six categories: setup, venues, paper types, sections, figures/tables, and writing style.
    • Sources include faculty feedback, public writing guides, 32 exemplary papers and 350 reviews from NeurIPS, ICLR and COLM 2025.
    • The material was restructured with Claude Opus 4.5 and then checked by humans.
  • Each comment includes a file, a character span, the feedback text and a severity label (CRITICAL, WARNING or SUGGESTION).
  • Six paper types are supported: analysis, dataset, method, engineering, interdisciplinary and position.
  • Evaluation covered ICLR 2026 submissions and internal student drafts. Against a GPT-5.2 baseline without the skill library:
    • 90.6% of comments were rated actionable, an improvement of 4.1 percentage points.
    • 67.5% were rated valid, an improvement of 6.5 percentage points.
  • Known limitations:
    • Section-level agents lack context from the whole document, so they sometimes flag content as missing when it appears elsewhere.
    • There is no comparison against comments written by human experts.
    • The code is released under AGPL-3.0.

Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.

 Institutions

🎓University of Illinois Urbana-ChampaignUrbana · United States184 works · 69,372 citations🎓University of TorontoToronto · Canada96 works · 142,007 citations🎓École Polytechnique Fédérale de LausanneLausanne · Switzerland44 works · 7,050 citations🎓ETH ZurichZurich · Switzerland72 works · 5,176 citations🔬Vector InstituteToronto · Canada33 works · 4,535 citations🔬Max Planck SocietyMunich · Germany16 works · 42,253 citations🔬Max Planck Institute for Intelligent SystemsStuttgart · Germany8 works · 854 citations

 Fields, topics and concepts

📚 Shelves 5The field of science this work belongs to (OpenAlex fields).Computer Science · 16,667 worksArtificial Intelligence · 16,194 worksArts and Humanities · 16,039 worksInformation Systems · 1,437 worksHistory and Philosophy of Science · 53 works

Cite

Cite as Jiarui Liu, Terry Jingchen Zhang, Ryan Faulkner, Xuanqiang Angelo Huang, Vilém Zouhar, Dominik Glandorf, Isabel Dahlgren, Rishit Dagli, Yuen Chen, Felix Leeb, Van Q. Truong, Punya Syon Pandey, Yves Bicker, Suvajit Majumder, Wenyuan Jiang, Zeju Qiu, Sankalan Pal Chowdhury, Mrinmaya Sachan, Bernhard Schölkopf, Mona T. Diab and Zhijing Jin. 2026. PaperMentor: A Human-Centered Multi-Agent Writing Tutor for AI Research Papers in Overleaf. Proceedings of the 64th Annual Meeting of the Association for Computational Linguistics (Volume 3: System Demonstrations).

The paper itself, not this page. BibTeX from the library's record of the work. DOI 10.18653/v1/2026.acl-demo.39

@inproceedings{liu2026papermentor,
  title = {{PaperMentor: A Human-Centered Multi-Agent Writing Tutor for AI Research Papers in Overleaf}},
  author = {Jiarui Liu, Terry Jingchen Zhang, Ryan Faulkner, Xuanqiang Angelo Huang, Vilém Zouhar, Dominik Glandorf, Isabel Dahlgren, Rishit Dagli, Yuen Chen, Felix Leeb, Van Q. Truong, Punya Syon Pandey, Yves Bicker, Suvajit Majumder, Wenyuan Jiang, Zeju Qiu, Sankalan Pal Chowdhury, Mrinmaya Sachan, Bernhard Schölkopf, Mona T. Diab and Zhijing Jin},
  booktitle = {Proceedings of the 64th Annual Meeting of the Association for Computational Linguistics (Volume 3: System Demonstrations)},
  year = {2026},
  doi = {10.18653/v1/2026.acl-demo.39},
  url = {https://doi.org/10.18653/v1/2026.acl-demo.39}
}

References

43 references, 1 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

  1. Jordan Boyd-Graber. Style. Online guide. University of Maryland. Lele Cao, Lei You, and R&d Team. 2025. CSPaper review: Fast, rubric-faithful conference feedback. In Proceedings of the 18th International Natural Language Generation Conference: System Demonstrations, pages 3–7, Hanoi, Vietnam. Associ… find
  2. Eric Chamoun, Michael Schlichtkrull, and Andreas Vlachos. 2024. Automated focused feedback generation for scientific writing assistance. In Findings of the Association for Computational Linguistics: ACL 2024, pages 9742–9763. find
  3. Mike D’Arcy, Tom Hope, Larry Birnbaum, and Doug Downey. 2024. Marg: Multi-agent review generation for scientific papers. arXiv preprint arXiv:2401.04259. find
  4. Paramveer S Dhillon, Somayeh Molaei, Jiaqi Li, Maximilian Golub, Shaochun Zheng, and Lionel Peter Robert. 2024. Shaping human-ai collaboration: Varied scaffolding levels in co-writing with language models. In Proceedings of the 2024 CHI conference on human factors in computing systems, pages 1–18. find
  5. Jason Eisner. 2010. How to write a paper? Online guide. Johns Hopkins University. find
  6. Maxwell Forbes. 2021. Figure creation tutorial: Making a figure 1. Online guide. University of Washington. find
  7. Xian Gao, Jiacheng Ruan, Zongyun Zhang, Jingsheng Gao, Ting Liu, and Yuzhuo Fu. 2025. Reviewagents: Bridging the gap between human and ai-generated paper reviews. Preprint, arXiv:2503.08506. — [arXiv:2503.08506](https://arxiv.org/abs/2503.08506) find
  8. Timnit Gebru, Jamie Morgenstern, Briana Vecchione, Jennifer Wortman Vaughan, Hanna Wallach, Hal Daumé III, and Kate Crawford. 2021. Datasheets for datasets. Communications of the ACM, 64(12):86– 92. find
  9. AAAI. 2026. AAAI launches AI-powered peer review assessment system. https://aaai.org/aaai-lau nches-ai-powered-peer-review-assessment-s ystem/. Accessed 2026-02-27. ↗
  10. ACL. 2021. Ethics FAQ: How to write ethical considerations. Online guide. find
  11. Grammarly. 2026. Grammarly. https://www.gramma rly.com/. Accessed 2026-02-27. ↗
  12. Anthropic. 2025. Introducing claude opus 4.5. https://www.anthropic.com/news/claude-opus-4-5. ↗
  13. Jieun Han, Haneul Yoo, Junho Myung, Minsun Kim, Hyunseung Lim, Yoonsu Kim, Tak Yeon Lee, Hwajung Hong, Juho Kim, So-Yeon Ahn, and 1 others. 2024. LLM-as-a-tutor in EFL writing education: Focusing on evaluation of student-llm interaction. In Proceedings of the 1st Workshop on Customizable NLP: Progre… find
  14. Emily M. Bender and Batya Friedman. 2018. Data statements for natural language processing: Toward mitigating system bias and enabling better science. In Transactions of the Association for Computational Linguistics, volume 6, pages 587–604. find
  15. Michael Black. Writing is laying out your logical thoughts. Twitter thread. Max Planck Institute Tuebingen. find
  16. Nicolas Bougie and Narimawa Watanabe. 2025. Generative reviewer agents: Scalable simulacra of peer review. In Proceedings of the 2025 Conference on Empirical Methods in Natural Language Processing: Industry Track, pages 98–116, Suzhou (China). Association for Computational Linguistics. find
  17. Jia-Bin Huang. 2023. How to write math in a paper? Twitter post. University of Maryland. find
  18. Yiqiao Jin, Qinlin Zhao, Yiyang Wang, Hao Chen, Kaijie Zhu, Yijia Xiao, and Jindong Wang. 2024. Agentreview: Exploring peer review dynamics with llm agents. In Proceedings of the 2024 Conference on Empirical Methods in Natural Language Processing, pages 1208–1226. find
  19. Zhijing Jin. 2021. Resources to help global equality for PhDs in NLP / AI. GitHub repository. Open resources and information for people to succeed in PhD in CS and career in AI/NLP, including writing suggestions from various professors. find
  20. Stanford. 2026. Paperreview.ai. https://paperrevie w.ai/. Accessed 2026-02-27. find
  21. Pawin Taechoyotin, Guanchao Wang, Tong Zeng, Bradley Sides, and Daniel Acuna. 2024. Mamorx: Multi-agent multi-modal scientific review generation with external knowledge. In Neurips 2024 Workshop Foundation Models for Science: Progress, Opportunities, and Challenges. find
  22. Zhijing Jin. 2024. Nlp phd global equality: Writing suggestions from various professors. https://gith ub.com/zhijing-jin/nlp-phd-global-equalit y. Max Planck Institute for Intelligent Systems. find
  23. Mina Lee, Katy Ilonka Gero, John Joon Young Chung, Simon Buckingham Shum, Vipul Raheja, Hua Shen, Subhashini Venugopalan, Thiemo Wambsganss, David Zhou, Emad A Alghamdi, and 1 others. 2024. A design space for intelligent and interactive writing assistants. In Proceedings of the 2024 CHI Conference o… find
  24. Weixin Liang, Yuhui Zhang, Hancheng Cao, Binglu Wang, Daisy Yi Ding, Xinyu Yang, Kailas Vodrahalli, Siyu He, Daniel Scott Smith, Yian Yin, and 1 others. 2024. Can large language models provide useful feedback on research papers? a large-scale empirical analysis. NEJM AI, 1(8):AIoa2400196. find
  25. Nitya Thakkar, Mert Yuksekgonul, Jake Silberg, Animesh Garg, Nanyun Peng, Fei Sha, Rose Yu, Carl Vondrick, and James Zou. 2025. Can llm feedback enhance review quality? a randomized study of 20k reviews at iclr 2025. arXiv preprint arXiv:2504.09737. find
  26. Xin Wang, Kaiwen Shi, and Carlos Oliver. 2025. Bioblobs: Unsupervised discovery of functional substructures for protein function prediction. arXiv preprint arXiv:2510.01632. — [arXiv:2510.01632](https://arxiv.org/abs/2510.01632) find
  27. Jennifer Widom. 2006. Tips for writing technical papers. Online guide. Stanford University. find
  28. Shomir Wilson. Guide for scholarly writing. Penn State University. find
  29. Ryan Liu and Nihar B Shah. 2023. Reviewergpt? an exploratory study on using large language models for paper reviewing. arXiv preprint arXiv:2306.00622. find
  30. Writefull. 2026. Writefull. https://writefull.com/. Accessed 2026-02-27. ↗
  31. Chris Maddison. How to write an ML paper. Notion page. Step-by-step writing guide. find
  32. Weizhe Yuan, Pengfei Liu, and Graham Neubig. 2022. Can we automate scientific reviewing? Journal of Artificial Intelligence Research, 75:171–212. find
  33. OpenAI. 2025. Introducing gpt-5.2. find
  34. OpenAI. 2026. Prism. https://prism.openai.com/. Accessed 2026-02-27. ↗
  35. Devi Parikh. Shortening papers to fit page limits. Medium blog post. find
  36. Simon Peyton Jones. 2014. How to write a great research paper. https://www.microsoft.com/en-u s/research/video/how-to-write-a-great-res earch-paper-3/. Microsoft Research. ↗
  37. Simon Peyton Jones. 2014. How to write a great research paper: Seven simple suggestions. Slides and talk. Microsoft Research. Talk available at https://www.microsoft.com/en-us/research/video/how-to-write-a-great-research-paper-3/. ↗
  38. Tim Rocktäschel and Jakob Foerster. 2022. How to ML paper. Twitter post. UCL/DeepMind and University of Oxford. find
  39. Ruiyang Zhou, Lu Chen, and Kai Yu. 2024. Is llm a reliable reviewer? a comprehensive evaluation of llm on automatic paper reviewing tasks. In Proceedings of the 2024 joint international conference on computational linguistics, language resources and evaluation (LREC-COLING 2024), pages 9340–9351.
  40. Minjun Zhu, Yixuan Weng, Linyi Yang, and Yue Zhang. 2025. DeepReview: Improving LLM-based paper review with human-like deep thinking process. In Proceedings of the 63rd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 29330–29355, Vienna, Austria. Associ… find
  41. Zhenzhen Zhuang, Jiandong Chen, Hongfeng Xu, Yuwen Jiang, and Jialiang Lin. 2025. Large language models for automated scholarly paper review: A survey. Information Fusion, 124:103332. find
  42. Anna Rogers and Isabelle Augenstein. 2020. What can we do to improve peer review in NLP? In Findings of the Association for Computational Linguistics: EMNLP 2020, pages 1256–1262. Association for Computational Linguistics. find
  43. Nihar B. Shah. 2022. Challenges, experiments, and computational solutions in peer review. Communications of the ACM, 65(6):76–87. find

Updated

↑ back to top

This page in Markdown → https://md.research.elamur.ai/free/en/papers/liu-2026-papermentor-human-centered-multi-agent/