Elamur Research⌕
☰
home / papers / 2026 / The Double Bind: Revisiting Preprinting and Peer Review Two Years After the Removal of the ACL Anonymity Period

The Double Bind: Revisiting Preprinting and Peer Review Two Years After the Removal of the ACL Anonymity Period

Paper · 2026 · A Pranav, Shane Storks and Anne Lauscher

Summary

In February 2024, ACL dropped its anonymity period, which had barred posting preprints during review. This paper asks who gained from that change and whether the embargo ever protected underrepresented researchers.

The authors used mixed methods:

  • They matched 46,923 ACL Anthology papers (2019–2025) against arXiv to see whether a preprint existed before the submission deadline.
  • They grouped papers by institution tier and country tier, using publication volume as the tier measure.
  • They surveyed 75 NLP researchers and interviewed 14 community members.
  • They examined about 1.9k ARR reviews, splitting them by tier and by whether reviewers said they knew the authors. Review quality was rated by LLM judges.

Preprinting during review rose sharply after the repeal, and the rise was largest at top-ranked institutions and countries. When reviewers recognized the authors, scores went up only for elite institutions, and review quality went down. Even so, the embargo appears to have done little for marginalized researchers. Participants pointed instead to:

  • weak visibility,
  • poor-quality reviews,
  • structural obstacles such as conference costs.

The study matters because it shifts the policy discussion away from anonymity rules and toward concrete diversity and inclusion investment.

Key points

  • Preprint trends: Overall preprinting during review grew from 19.9% (2023) to 39.3% (2024) and 47.8% (2025).
    • By 2025, Tier 1 (top 50) institutions reached 52.2%, compared with 35.8% for Tier 3 (rank 251+).
    • The gap between country tiers widened from 3.8 to 17.2 points.
  • Effect of author recognition: Knowing the authors raised review scores only at elite institutions (d = 0.43, p < 0.001).
    • Review quality was assessed with the RevUtil and LazyReviewPlus models.
    • The analysis covers accepted papers only.
  • Survey results: Most respondents agreed that preprinting favours elite institutions. Only about 20% believed the policy change affected their own paper decisions.
  • Recommendations: The authors call on ACL to:
    • adopt ACM-style tiered fees;
    • seek more D&I sponsors (the number fell from eight in 2022 to one in 2024 and 2025);
    • pursue Scopus indexing of proceedings;
    • set up ARR mentoring programs;
    • give more weight to affinity groups.

Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.

 Institutions

🎓University of MichiganAnn Arbor · United States70 works · 13,693 citations🎓Universität HamburgHamburg · Germany79 works · 5,422 citations

 Fields, topics and concepts

📚 Shelves 5The field of science this work belongs to (OpenAlex fields).Arts and Humanities · 16,039 worksDecision Sciences · 209 worksInformation Systems and Management · 41 worksStatistics, Probability and Uncertainty · 61 worksHistory and Philosophy of Science · 53 works

Cite

Cite as A Pranav, Shane Storks and Anne Lauscher. 2026. The Double Bind: Revisiting Preprinting and Peer Review Two Years After the Removal of the ACL Anonymity Period. Findings of the Association for Computational Linguistics: ACL 2026.

The paper itself, not this page. BibTeX from the library's record of the work. DOI 10.18653/v1/2026.findings-acl.222

@inproceedings{pranav2026double,
  title = {{The Double Bind: Revisiting Preprinting and Peer Review Two Years After the Removal of the ACL Anonymity Period}},
  author = {A Pranav, Shane Storks and Anne Lauscher},
  booktitle = {Findings of the Association for Computational Linguistics: ACL 2026},
  year = {2026},
  doi = {10.18653/v1/2026.findings-acl.222},
  url = {https://doi.org/10.18653/v1/2026.findings-acl.222}
}

References

50 references, 4 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

  1. Nils Dycke, Ilia Kuznetsov, and Iryna Gurevych. 2023. NLPeer: A unified resource for the computational study of peer review. In Proceedings of the 61st Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 5049–5073, Toronto, Canada. Association for Computati…
  2. Marah Abdin, Jyoti Aneja, Harkirat Behl, Sébastien Bubeck, Ronen Eldan, Suriya Gunasekar, Michael Harrison, Russell J. Hewett, Mojan Javaheripi, Piero Kauffmann, James R. Lee, Yin Tat Lee, Yuanzhi Li, Weishung Liu, Caio C. T. Mendes, Anh Nguyen, Eric Price, Gustavo de Rosa, Olli Saarikivi, and 8 oth… — Phi-4 Technical Report
  3. ACL. 2025. Diversity and inclusion chair handbook. find
  4. Organizing Committee of ACL. 2016. Call for papers: ACL 2016. find
  5. Organizing Committee of ACL. 2022. find
  6. Organizing Committee of ACL. 2024. Sponsors. find
  7. Organizing Committee of ACL. 2025. Sponsors. find
  8. ACL Admin Wiki. 2025a. ACL author guidelines. find
  9. ACL Admin Wiki. 2025b. ACL policies for review and citation. find
  10. Organizing Committee of ACL-IJCNLP. 2009. find
  11. ACL-IJCNLP 2009 2nd call for papers. find
  12. ACM. 2025. Policy on geographic apc waivers and discounts. find
  13. Organizing Committee of EACL. 2014. Call for papers. find
  14. Organizing Committee of EMNLP. 2015. Call for papers for EMNLP 2015. find
  15. Aaron Grattafiori, Abhimanyu Dubey, Abhinav Jauhri, Abhinav Pandey, Abhishek Kadian, Ahmad Al-Dahle, Aiesha Letman, Akhil Mathur, Alan Schelten, Alex Vaughan, Amy Yang, Angela Fan, Anirudh Goyal, Anthony Hartshorn, Aobo Yang, Archi Mitra, Archie Sravankumar, Artem Korenev, Arthur Hinsvark, and 542 o… — The Llama 3 Herd of Models
  16. Organizing Committee of HLT/EMNLP. 2005. HLT/EMNLP 2005 call for papers. find
  17. Adina R. Kern-Goldberger, Richard James, Vincenzo Berghella, and Emily S. Miller. 2022. The impact of double-blind peer review on gender bias in scientific publishing: a systematic review. American Journal of Obstetrics and Gynecology, 227(1):43–50.e4. find
  18. Yoav Benjamini and Yosef Hochberg. 1995. Controlling the false discovery rate: a practical and powerful approach to multiple testing. Journal of the Royal Statistical Society: Series B, 57(1):289–300. find
  19. Ilia Kuznetsov, Osama Mohammed Afzal, Koen Dercksen, Nils Dycke, Alexander Goldberg, Tom Hope, Dirk Hovy, Jonathan K. Kummerfeld, Anne Lauscher, Kevin Leyton-Brown, Sheng Lu, Mausam, Margot Mieskes, Aurélie Névéol, Danish Pruthi, Lizhen Qu, Roy Schwartz, Noah A. Smith, Thamar Solorio, and 5 others.… find
  20. Homanga Bharadhwaj, Dylan Turpin, Animesh Garg, and Ashton Anderson. 2020. De-anonymization of authors through arXiv submissions during double-blind review. Preprint, arXiv:2007.00177. — [arXiv:2007.00177](https://arxiv.org/abs/2007.00177) find
  21. C. Le Goues, Y. Brun, S. Apel, E. Berger, S. Khurshid, and Y. Smaragdakis. 2018. Effectiveness of anonymization in double-blind review. Commun. ACM, 61(6):30–33. find
  22. Cornelia Caragea, Ana Uban, and Liviu P. Dinu. 2019. The myth of double-blind review revisited: ACL vs. EMNLP. In Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing (EMNLP-IJCNLP), pages 231… find
  23. NAACL. 2022. Affinity group workshops. find
  24. ARR. 2024. Update to anonymity policy. find
  25. ARR. 2025. ARR reviewer guidelines. find
  26. Jacob Cohen. 1968. Weighted kappa: Nominal scale agreement provision for scaled disagreement or partial credit. Psychological bulletin, 70(4):213. find
  27. Jacob Cohen. 1988. Statistical Power Analysis for the Behavioral Sciences, 2nd edition. Lawrence Erlbaum Associates. find
  28. Nils Dycke, Ilia Kuznetsov, and Iryna Gurevych. 2022. Yes-yes-yes: Proactive data collection for ACL rolling NAACL. 2024. find
  29. NAACL 2024 theme track: Languages of Latin America. find
  30. Organizing Committee of NAACL. 2001. find
  31. NAACL 2001 call for papers. find
  32. Organizing Committee of NAACL. 2013. Call for papers for NAACL HLT 2013. find
  33. E. E. O’Connor, M. Cousar, J. A. Lentini, M. Castillo, K. Halm, and T. A. Zeffiro. 2017. Efficacy of Double-Blind Peer Review in an Imaging Subspecialty Journal. AJNR. American journal of neuroradiology, 38(2):230– 235. find
  34. OpenAI, Aaron Hurst, Adam Lerer, Adam P. Goucher, Adam Perelman, Aditya Ramesh, Aidan Clark, AJ Ostrow, Akila Welihinda, Alan Hayes, Alec Radford, Aleksander M adry, Alex Baker-Whitcomb, Alex Beutel, Alex Borzunov, Alex Carney, Alex Chow, Alex Kirillov, Alex Nichol, and 400 others. 2024. GPT-4o sys…
  35. Sukannya Purkayastha, Qile Wan, Anne Lauscher, Lizhen Qu, and Iryna Gurevych. 2026. Reviewing the reviewer: Elevating peer review quality through llm-guided feedback. In Findings of the Association for Computational Linguistics: ACL 2026. find
  36. RAF. 2024. NAACL Regional Americas Fund. find
  37. Organizing Committee of RANLP. 2025. Final call for papers, RANLP 2025. find
  38. Charvi Rastogi, Ivan Stelmakh, Xinwei Shen, Marina Meila, Federico Echenique, Shuchi Chawla, and Nihar B. Shah. 2022. To arXiv or not to arXiv: A study quantifying pros and cons of posting preprints online. Preprint, arXiv:2203.17259. — [arXiv:2203.17259](https://arxiv.org/abs/2203.17259) find
  39. Terne Thorn Jakobsen and Anna Rogers. 2022. What factors should paper-reviewer assignments rely on? community perspectives on issues and ideals in conference peer-review. In Proceedings of the 2022 Conference of the North American Chapter of the Association for Computational Linguistics: Human Langu… find
  40. Andrew Tomkins, Min Zhang, and William D. Heavlin. 2017. Reviewer bias in single- versus double-blind peer review. Proceedings of the National Academy of Sciences, 114(48):12708–12713. find
  41. Transactions of the Association for Computational Linguistics. 2024. Submissions. find
  42. Anna Rogers, Marzena Karpinska, Jordan Boyd-Graber, and Naoaki Okazaki. 2023. Program chairs’ report on peer review at ACL 2023. In Proceedings of the 61st Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages xl–lxxv, Toronto, Canada. Association for Computa… find
  43. Abdelrahman Sadallah, Tim Baumgärtner, Iryna Gurevych, and Ted Briscoe. 2025. The good, the bad and the constructive: Automatically measuring peer review’s utility for authors. In Proceedings of the 2025 Conference on Empirical Methods in Natural Language Processing, pages 28979–29009, Suzhou, China… find
  44. Scopus. 2025a. International conference recent advances in natural language processing, RANLP. find
  45. Scopus. 2025b. Proceedings of the annual meeting of the Association for Computational Linguistics. find
  46. Nihar B Shah. 2022. Challenges, experiments, and computational solutions in peer review. Communications of the ACM, 65(6):76–87. find
  47. Olivia M Smith, Kayla L Davis, Riley B Pizza, Robin Waterman, Kara C Dobson, Brianna Foster, Julie C Jarvey, Leonard N Jones, Wendy Leuenberger, Nan Nourn, and 1 others. 2023. Peer review perpetuates barriers for historically excluded groups. Nature Ecology & Evolution, 7(4):512–523. find
  48. Ivan Stelmakh, Nihar B. Shah, Aarti Singh, and Hal Daumé III. 2020. A novice-reviewer experiment to address scarcity of qualified reviewers in large conferences. AAAI-21. find
  49. Charles Sutton and Linan Gong. 2017. Popularity of arXiv.org within computer science. Preprint, arXiv:1710.05225. — [arXiv:1710.05225](https://arxiv.org/abs/1710.05225) find
  50. Péter Szluka, Edit Csajbók, and Balázs Győrffy. 2023. Relationship between bibliometric indicators and university ranking positions. Scientific Reports, 13(1):14193. find

Updated

↑ back to top

This page in Markdown → https://md.research.elamur.ai/free/en/papers/pranav-2026-double-bind-revisiting-preprinting-peer/