Elamur Research⌕
☰
home / papers / 2022 / Revise and Resubmit: An Intertextual Model of Text-based Collaboration in Peer Review

Revise and Resubmit: An Intertextual Model of Text-based Collaboration in Peer Review

Paper · 2022 · Ilia Kuznetsov, Jan Buchmann, Max Eichler et al.

Summary

Peer review is a clear case of text-based collaboration. Referees comment on a manuscript, authors revise it, and the cycle repeats. Supporting this work computationally requires reasoning across pairs of documents, yet NLP has mostly analyzed texts in isolation. Existing review datasets also cover only pre-publication conference reviewing in machine learning. Drawing on intertextuality theory from literary studies, the authors formalize one revision round as three tasks: pragmatic tagging, linking, and version alignment. Pragmatic tagging labels review statements by communicative function. Linking connects review passages to locations in the paper. Version alignment matches parts of consecutive paper revisions. Documents are represented with the Intertextual Graph, a structure-aware data model that replaces flat plain text and encodes cross-document relations. The framework is instantiated in F1000RD, an annotated corpus built from F1000Research's open, journal-style, post-publication reviews across many disciplines. Annotation studies reveal each task's strengths and weak spots, and a joint analysis shows how the relation types interact. The openly licensed resource supports multidomain editorial-assistance tools and broader models of collaborative writing.

Key points

  • F1000RD is presented as the first multidomain NLP corpus of open, journal-style, post-publication peer review. It contains reviews, papers, and paper revisions sourced from F1000Research under CC-BY/CC0 licenses.
  • The authors compare F1000RD to prior data sources. ICLR (OpenReview) lacks a clear license, NeurIPS offers no revision history, and arXiv has revisions but no reviews. F1000Research provides all three.
  • Three new tasks (pragmatic tagging, linking, version alignment) are defined over the Intertextual Graph, which models document structure and non-textual elements.
  • The version aligner works at paragraph granularity and cannot yet handle split or merged paragraphs. The joint analysis suggests that link scope may extend beyond the single sentence where a link appears.
  • The corpus, annotation guidelines, and code are released at github.com/UKPLab/f1000rd. The work was funded by the DFG PEER project and the EU InterText grant.

Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.

 Institutions

🎓Technische Universität DarmstadtDarmstadt · Germany109 works · 35,920 citations

 Fields, topics and concepts

📚 Shelves 5The field of science this work belongs to (OpenAlex fields).Computer Science · 16,667 worksArtificial Intelligence · 16,194 worksArts and Humanities · 16,039 worksLanguage and Linguistics · 15,891 worksHistory and Philosophy of Science · 53 works

Cite

Cite as Ilia Kuznetsov, Jan Buchmann, Max Eichler and Iryna Gurevych. 2022. Revise and Resubmit: An Intertextual Model of Text-based Collaboration in Peer Review. Computational Linguistics.

The paper itself, not this page. BibTeX from the library's record of the work. DOI 10.1162/coli_a_00455

@article{kuznetsov2022revise,
  title = {{Revise and Resubmit: An Intertextual Model of Text-based Collaboration in Peer Review}},
  author = {Ilia Kuznetsov, Jan Buchmann, Max Eichler and Iryna Gurevych},
  journal = {Computational Linguistics},
  year = {2022},
  doi = {10.1162/coli\_a\_00455},
  url = {https://doi.org/10.1162/coli_a_00455}
}

References

28 references, 13 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

  1. Afrin, Tazin and Diane Litman. 2018. Annotation and classification of sentence-level revision improvement. In Proceedings of the Thirteenth Workshop on Innovative Use of NLP for Building Educational Applications, pages 240–246. find
  2. Anjum, Omer, Hongyu Gong, Suma Bhat, Wen-Mei Hwu, and JinJun Xiong. 2019. PaRe: A paper-reviewer matching approach using a common topic space. In Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Proc… find
  3. Beltagy, Iz, Matthew E. Peters, and Arman Cohan. 2020. Longformer: The long-document transformer. arXiv:2004.05150. Broich, Ulrich, editor. 1985. Intertextualität: Formen, Funktionen, anglistische Fallstudien. Number 35 in Konzepte der Sprach- und Literaturwissenschaft. Niemeyer, Tübingen. https:/…
  4. John, Noah Constant, Mario Guajardo-Cespedes, Steve Yuan, Chris Tar, Brian Strope, and Ray Kurzweil. 2018. Universal sentence encoder for English. In Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing: System Demonstrations, pages 169–174. https://doi.org/10.18653…
  5. Chandrasekaran, Muthu Kumar, Guy Feigenblat, Eduard Hovy, Abhilasha Ravichander, Michal Shmueli-Scheuer, and Anita de Waard. 2020. Overview and insights from the shared tasks at scholarly document processing 2020: CL-SciSumm, LaySumm and LongSumm. In Proceedings of the First Workshop on Scholarly Do… find
  6. Pretrained language models for sequential sentence classification. In Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing (EMNLP-IJCNLP), pages 3693–3699. find
  7. Cohan, Arman, Sergey Feldman, Iz Beltagy, Doug Downey, and Daniel Weld. 2020. SPECTER: Document-level representation learning using citation-informed transformers. In Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics, pages 2270–2282. https://doi.org/10.18653/v1… ↗
  8. Daxenberger, Johannes and Iryna Gurevych. 2014. Automatically detecting corresponding edit-turn-pairs in Wikipedia. In Proceedings of the 52nd Annual Meeting of the Association for Computational Linguistics (Volume 2: Short Papers), pages 187–192. https://doi.org/10.3115/v1/P14-2031 Devlin, Jacob,… ↗
  9. Dycke, Nils, Edwin Simpson, Ilia Kuznetsov, and Iryna Gurevych. 2021. Ranking scientific papers using preference learning. arXiv:2109.01190. — [arXiv:2109.01190](https://arxiv.org/abs/2109.01190) find
  10. Esteva, Andre, Anuprit Kale, Romain Paulus, Kazuma Hashimoto, Wenpeng Yin, Dragomir Radev, and Richard Socher. 2021. COVID-19 information retrieval with deep-learning based semantic search, question answering, and abstractive summarization. NPJ Digital Medicine, 4(1):68. https://doi.org/10.1038 /s41… find
  11. Ghosal, Tirthankar, Sandeep Kumar, Prabhat Kumar Bharti, and Asif Ekbal. 2022. Peer review analyze: A novel benchmark resource for computational analysis of peer reviews. PLOS ONE, 17(1):1–29. https://doi.org/10.1371/journal .pone.0259238, PubMed: 35085252 Ghosal, Tirthankar, Rajeev Verma, Asif Ekba… ↗
  12. Hua, Xinyu, Mitko Nikolov, Nikhil Badugu, and Lu Wang. 2019. Argument mining for understanding peer reviews. In Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long and Short Papers), pages 2131… find
  13. Kwiatkowski, Tom, Jennimaria Palomaki, Olivia Redfield, Michael Collins, Ankur Parikh, Chris Alberti, Danielle Epstein, Illia Polosukhin, Jacob Devlin, Kenton Lee, Kristina Toutanova, Llion Jones, Matthew Kelcey, Ming-Wei Chang, Andrew M. Dai, Jakob Uszkoreit, Quoc Le, and Slav Petrov. 2019. Natural…
  14. Liu, Yinhan, Myle Ott, Naman Goyal, Jingfei Du, Mandar Joshi, Danqi Chen, Omer Levy, Mike Lewis, Luke Zettlemoyer, and Veselin Stoyanov. 2019. RoBERTa: A robustly optimized BERT pretraining approach. arXiv:1907.11692, abs/1907.11692.
  15. Lo, Kyle, Lucy Lu Wang, Mark Neumann, Rodney Kinney, and Daniel Weld. 2020. S2ORC: The semantic scholar open research corpus. In Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics, pages 4969–4983. https://doi.org/10.18653/v1/2020 .acl-main.447 Luan, Yi, Luheng H… ↗
  16. Mann, William C. and Sandra A. Thompson. 1988. Rhetorical structure theory: Toward a functional theory of text organization. Text – Interdisciplinary Journal for the Study of Discourse, 8(3):243–281. https://doi.org/10.1515/text.1.1988.8.3.243 Marcus, Mitchell P., Mary Ann Marcinkiewicz, and Beatri… ↗
  17. Mimno, David and Andrew McCallum. 2007. Expertise modeling for matching papers with reviewers. In Proceedings of the 13th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, KDD ’07, pages 500–509. find
  18. Moreno, Laura, Gabriele Bavota, Massimiliano Di Penta, Rocco Oliveto, Andrian Marcus, and Gerardo Canfora. 2017. ARENA: An approach for the automated generation of release notes. IEEE Transactions on Software Engineering, 43(2):106–127. https://doi.org/10 .1109/TSE.2016.2591536 Mussmann, Stephen, Ro… find
  19. Nury, Elisa and Elena Spadini. 2020. From giant despair to a new heaven: The early years of automatic collation. IT -Information Technology, 62(2):61–73. https://doi.org/10.1515/itit-2019 -0047 Nye, Benjamin, Ani Nenkova, Iain Marshall, and Byron C. Wallace. 2020. Trialstreamer: Mapping and browsing… ↗
  20. Reimers, Nils and Iryna Gurevych. 2019. Sentence-BERT: Sentence embeddings using Siamese BERT-networks. In Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing (EMNLP-IJCNLP), pages 3982–3992.…
  21. Stab, Christian and Iryna Gurevych. 2014. Identifying argumentative discourse structures in persuasive essays. In Proceedings of the 2014 Conference on Empirical Methods in Natural Language Processing (EMNLP), pages 46–56.
  22. Stab, Christian and Iryna Gurevych. 2017. Parsing argumentation structures in persuasive essays. Computational Linguistics, 43(3):619–659. https://doi.org/10.1162/COLI a 00295 Steyer, Kathrin. 2015. Irgendwie hängt alles mit allem zusammen – Grenzen und Möglichkeiten einer linguistischen Kategori… ↗
  23. Teufel, Simone. 2006. Argumentative zoning for improved citation indexing. Computing Attitude and Affect in Text: Theory and Applications, pages 159–169. https://doi.org/10.1007/1-4020-4102-0_13 Teufel, Simone, Advaith Siddharthan, and Dan Tidhar. 2006. Automatic classification of citation function… ↗
  24. Thelwall, Mike, Eleanor-Rose Papas, Zena Nyakoojo, Liz Allen, and Verena Weigert. 2020. Automatically detecting open academic review praise and criticism. Online Information Review, 44(5):1057–1076. https://doi.org/10.1108/OIR-11 -2019-0347 Thorne, James, Andreas Vlachos, Christos Christodoulopoulos… ↗
  25. White, Aaron Steven, Elias Stengel-Eskin, Siddharth Vashishtha, Venkata Subrahmanyan Govindarajan, Dee Ann Reisinger, Tim Vieira, Keisuke Sakaguchi, Sheng Zhang, Francis Ferraro, Rachel Rudinger, Kyle Rawlins, and Benjamin Van Durme. 2020. The Universal Decompositional Semantics dataset and Decomp t…
  26. Yang, Diyi, Aaron Halfaker, Robert E. Kraut, and Eduard H. Hovy. 2017. Identifying semantic edit intentions from revisions in Wikipedia. In Proceedings of the 2017 Conference on Empirical Methods in Natural Language Processing, EMNLP 2017, pages 2000–2010. find
  27. Yuan, Weizhe, Pengfei Liu, and Graham Neubig. 2021. Can we automate scientific reviewing? arXiv:2102.00176. find
  28. Zhang, Fan and Diane Litman. 2015. Annotation and classification of argumentative writing revisions. In Proceedings of the Tenth Workshop on Innovative Use of NLP for Building Educational Applications, pages 133–143. https://doi.org/10.3115/v1/W15-0616 ↗

Updated

↑ back to top

This page in Markdown → https://md.research.elamur.ai/free/en/papers/kuznetsov-2022-revise-resubmit-intertextual-model-text/