Elamur Research⌕
☰
home / papers / 2024 / AI-Driven Review Systems: Evaluating LLMs in Scalable and Bias-Aware Academic Reviews

AI-Driven Review Systems: Evaluating LLMs in Scalable and Bias-Aware Academic Reviews

Paper · 2024 · Keith Tyser, Ben Segev, Gaston Longhitano et al.

Summary

Peer review struggles to keep up with fast-growing paper volumes. This work asks whether large language models can produce useful, calibrated reviews at scale and how such reviews should be judged. The authors built three connected tools. OpenReviewer is a GPT-4-based reviewer that reads both the text and the figures of a paper. Papers with Reviews is a public site that automatically reviews new arXiv and open-access Nature papers. Reviewer Arena collects pairwise human judgments of competing reviews. Review quality is assessed in four ways: human preference voting, GPT-4 acting as a judge, a fine-tuned model that predicts which review people will favour, and controlled insertion of flaws into papers to see what the reviewer catches. Giving the model venue materials, such as guidelines, ethics codes and past score statistics, brought its score distributions close to human ones. Blind comparisons showed LLM reviews aligning well with human reviews. This matters because it maps where automated reviewing can be trusted and where it cannot.

Key points

  • Three systems are released: OpenReviewer, Papers with Reviews and Reviewer Arena.
  • Papers with Reviews ingests about 500 arXiv papers per day and roughly 1,000 open-access Nature papers per month.
  • Supplying venue-specific context reduced inflated or skewed ratings and made the model use the full score range. This context included the review form, reviewer and area chair guides, ethics and conduct codes, and prior-year statistics.
  • The error-injection experiments gave mixed results. The model reliably flagged problems like overclaiming but missed cases where authors departed from expected norms.
  • GPT-4 personas (PC, SAC, AC, reviewers, authors) simulate the full editorial cycle, including rebuttals, discussions and meta-reviews.

Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.

 Institutions

🎓Stanford UniversityStanford · United States567 works · 335,190 citations🎓Columbia UniversityNew York · United States263 works · 57,821 citations🎓Massachusetts Institute of TechnologyCambridge · United States274 works · 82,013 citations🎓Tel Aviv UniversityTel Aviv · Israel91 works · 11,470 citations🎓Boston UniversityBoston · United States35 works · 4,458 citations

 Fields, topics and concepts

📚 Shelves 7The field of science this work belongs to (OpenAlex fields).Computer Science · 19,503 worksArtificial Intelligence · 18,805 worksArts and Humanities · 17,959 worksLanguage and Linguistics · 17,789 worksSocial Sciences · 2,009 worksSafety Research · 665 worksHistory and Philosophy of Science · 83 works

🧭 Topics 1Research subjects and languages the work is about.Language models & pretraining · 4,145 works

Cite

Cite as Keith Tyser, Ben Segev, Gaston Longhitano, Xin-Yu Zhang, Zachary Meeks, Jason Lee, Uday Garg, Nicholas Belsten, Avi Shporer, Madeleine Udell, Dov Te'eni and Iddo Drori. 2024. AI-Driven Review Systems: Evaluating LLMs in Scalable and Bias-Aware Academic Reviews. arXiv:2408.10365.

The paper itself, not this page. BibTeX from the library's record of the work. DOI 10.48550/arxiv.2408.10365 · arXiv 2408.10365

@misc{tyser2024ai,
  title = {{AI-Driven Review Systems: Evaluating LLMs in Scalable and Bias-Aware Academic Reviews}},
  author = {Keith Tyser, Ben Segev, Gaston Longhitano, Xin-Yu Zhang, Zachary Meeks, Jason Lee, Uday Garg, Nicholas Belsten, Avi Shporer, Madeleine Udell, Dov Te'eni and Iddo Drori},
  year = {2024},
  doi = {10.48550/arxiv.2408.10365},
  eprint = {2408.10365},
  archivePrefix = {arXiv},
  url = {https://doi.org/10.48550/arxiv.2408.10365}
}

References

38 references, 19 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

  1. Abdin, M.; Jacobs, S. A.; Awan, A. A.; Aneja, J.; Awadallah, A.; Awadalla, H.; Bach, N.; Bahree, A.; Bakhtiari, A.; Behl, H.; et al. 2024. Phi-3 technical report: A highly capable language model locally on your phone. arXiv preprint arXiv:2404.14219.
  2. Angelopoulos, A. N.; Duchi, J. C.; and Zrnic, T. 2023. PPI++: Efficient prediction-powered inference. arXiv preprint arXiv:2311.01453. arXiv Sanity Lite. 2024. https://github.com/karpathy/arxiv-sanity-lite. — [arXiv:2311.01453](https://arxiv.org/abs/2311.01453) ↗
  3. Bao, P.; Hong, W.; and Li, X. 2021. Predicting paper acceptance via interpretable decision sets. In Companion Proceedings of the Web Conference 2021, 461–467. Bergstra, J.; Bardenet, R.; Bengio, Y.; and Kégl, B. 2011. Algorithms for hyper-parameter optimization. Advances in neural information proce… find
  4. Boyeau, P.; Angelopoulos, A. N.; Yosef, N.; Malik, J.; and Jordan, M. I. 2024. AutoEval Done Right: Using Synthetic Data for Model Evaluation. arXiv preprint arXiv:2403.07008. — [arXiv:2403.07008](https://arxiv.org/abs/2403.07008) find
  5. Checco, A.; Bracciale, L.; Loreti, P.; Pinfield, S.; and Bianchi, G. 2021. AI-assisted peer review. Humanities and Social Sciences Communications, 8(1): 1–11. find
  6. Chiang, W.-L.; Zheng, L.; Dunlap, L.; Gonzalez, J. E.; Stoica, I.; Mooney, P.; Dane, S.; Howard, A.; and Keating, N. 2024a. LMSYS - Chatbot Arena Human Preference Predictions. find
  7. Chiang, W.-L.; Zheng, L.; Sheng, Y.; Angelopoulos, A. N.; Li, T.; Li, D.; Zhang, H.; Zhu, B.; Jordan, M.; Gonzalez, J. E.; et al. 2024b. Chatbot Arena: An Open Platform for Evaluating LLMs by Human Preference. arXiv preprint arXiv:2403.04132.
  8. D’Arcy, M.; Hope, T.; Birnbaum, L.; and Downey, D. 2024. MARG: Multi-Agent Review Generation for Scientific Papers. arXiv preprint arXiv:2401.04259.
  9. Gao, H. 2022. Improved sentiment analysis using a customized distilbert NLP configuration. Advances in Engineering: An International Journal (ADEIJ), 3(2). find
  10. Goldberg, A.; Stelmakh, I.; Cho, K.; Oh, A.; Agarwal, A.; Belgrave, D.; and Shah, N. B. 2023. Peer Reviews of Peer Reviews: A Randomized Controlled Trial and Other Experiments. arXiv preprint arXiv:2311.09497.
  11. Kaddour, J.; Harris, J.; Mozes, M.; Bradley, H.; Raileanu, R.; and McHardy, R. 2023. Challenges and applications of large language models. arXiv preprint arXiv:2307.10169.
  12. Koivisto, M.; and Grassini, S. 2023. Best humans still outperform artificial intelligence in a creative divergent thinking task. Scientific Reports, 13(1): 13601. find
  13. Kuznetsov, I.; Afzal, O. M.; Dercksen, K.; Dycke, N.; Goldberg, A.; Hope, T.; Hovy, D.; Kummerfeld, J. K.; Lauscher, A.; Leyton-Brown, K.; et al. 2024. What Can Natural Language Processing Do for Peer Review? arXiv preprint arXiv:2405.06563.
  14. Latona, G. R.; Ribeiro, M. H.; Davidson, T. R.; Veselovsky, V.; and West, R. 2024. The AI Review Lottery: Widespread AI-Assisted Peer Reviews Boost Paper Scores and Acceptance Rates. arXiv preprint arXiv:2405.02150.
  15. Liang, W.; Zhang, Y.; Cao, H.; Wang, B.; Ding, D.; Yang, X.; Vodrahalli, K.; He, S.; Smith, D.; Yin, Y.; McFarland, D.; and Zou, J. 2023. Can large language models provide useful feedback on research papers? A large-scale empirical analysis. arXiv preprint arXiv:2310.01783.
  16. Liu, N. F.; Lin, K.; Hewitt, J.; Paranjape, A.; Bevilacqua, M.; Petroni, F.; and Liang, P. 2024. Lost in the middle: How language models use long contexts. Transactions of the Association for Computational Linguistics, 12: 157–173.
  17. Liu, R.; and Shah, N. B. 2023. ReviewerGPT? An exploratory study on using large language models for paper reviewing. arXiv preprint arXiv:2306.00622.
  18. Lu, C.; Lu, C.; Lange, R. T.; Foerster, J.; Clune, J.; and Ha, D. 2024. The AI Scientist: Towards Fully Automated Open-Ended Scientific Discovery. arXiv preprint arXiv:2408.06292.
  19. McKay, C. 2024. Anthropic’s New Tool Will Write Better Prompts For You. Maginative. Accessed: 2024-08-10. find
  20. Miller, G. A. 1995. WordNet: A Lexical Database for English. Commun. ACM, 38: 39–41.
  21. Paivio, A. 2010. Dual coding theory and the mental lexicon. The Mental Lexicon, 5(2): 205–230. find
  22. Park, R.; Rafailov, R.; Ermon, S.; and Finn, C. 2024. Disentangling length from quality in direct preference optimization. arXiv preprint arXiv:2403.19159.
  23. Petrescu, M.; and Krishen, A. S. 2022. The evolving crisis of the peer-review process. Journal of Marketing Analytics, 10(3): 185–186. find
  24. Powell, T. E.; Boomgaarden, H. G.; De Swert, K.; and De Vreese, C. H. 2015. A clearer picture: The contribution of visuals and text to framing effects. Journal of communication, 65(6): 997–1017. find
  25. Robertson, Z. 2023. GPT4 is slightly helpful for peer-review assistance: A pilot study. arXiv preprint arXiv:2307.05492. — [arXiv:2307.05492](https://arxiv.org/abs/2307.05492) find
  26. Saito, K.; Wachi, A.; Wataoka, K.; and Akimoto, Y. 2023. Verbosity bias in preference labeling by large language models. arXiv preprint arXiv:2310.10076. — [arXiv:2310.10076](https://arxiv.org/abs/2310.10076) find
  27. Sanh, V.; Debut, L.; Chaumond, J.; and Wolf, T. 2019. DistilBERT, a distilled version of BERT: smaller, faster, cheaper and lighter. ArXiv, abs/1910.01108. Santu, S. K. K.; Sinha, S. K.; Bansal, N.; Knipper, A.; Sarkar, S.; Salvador, J.; Mahajan, Y.; Guttikonda, S.; Akter, M.; Freestone, M.; and au2…
  28. Schnotz, W.; and Bannert, M. 2003. Construction and interference in learning from multiple representation. Learning and instruction, 13(2): 141–156. find
  29. Schulz, R.; Barnett, A.; Bernard, R.; Brown, N. J.; Byrne, J. A.; Eckmann, P.; Gazda, M. A.; Kilicoglu, H.; Prager, E. M.; Salholz-Hillel, M.; et al. 2022. Is the future of peer review automated? BMC Research Notes, 15(1): 1–5. find
  30. Spitale, G.; Biller-Andorno, N.; and Germani, F. 2023. AI model GPT-3 (dis) informs us better than humans. Science Advances, 9(26). find
  31. Suzgun, M.; and Kalai, A. T. 2024. Meta-Prompting: Enhancing Language Models with Task-Agnostic Scaffolding. arXiv preprint arXiv:2401.12954.
  32. Tal, A.; and Wansink, B. 2016. Blinded with science: Trivial graphs and formulas increase ad persuasiveness and belief in product efficacy. Public Understanding of Science, 25(1): 117–125. find
  33. Tan, C.; Lyu, D.; Li, S.; Gao, Z.; Wei, J.; Ma, S.; Liu, Z.; and Li, S. Z. 2024. Peer Review as A Multi-Turn and Long-Context Dialogue with Role-Based Interactions. arXiv preprint arXiv:2406.05688. — [arXiv:2406.05688](https://arxiv.org/abs/2406.05688) find
  34. Vesper, I. 2018. Peer reviewers unmasked: Largest global survey reveals trends. Nature, 7–8. find
  35. Wei, J.; and Zou, K. 2019. Eda: Easy data augmentation techniques for boosting performance on text classification tasks. arXiv preprint arXiv:1901.11196.
  36. Zhang, Y.; Li, Y.; Cui, L.; Cai, D.; Liu, L.; Fu, T.; Huang, X.; Zhao, E.; Zhang, Y.; Chen, Y.; et al. 2023. Siren’s song in the ai ocean: A survey on hallucination in large language models. arXiv preprint arXiv:2309.01219.
  37. Zheng, L.; Chiang, W.-L.; Sheng, Y.; Zhuang, S.; Wu, Z.; Zhuang, Y.; Lin, Z.; Li, Z.; Li, D.; Xing, E.; et al. 2024. Judging llm-as-a-judge with mt-bench and chatbot arena. Advances in Neural Information Processing Systems, 36.
  38. Zou, A.; Wang, Z.; Kolter, J. Z.; and Fredrikson, M. 2023. Universal and transferable adversarial attacks on aligned language models. arXiv preprint arXiv:2307.15043.

Updated

↑ back to top

This page in Markdown → https://md.research.elamur.ai/free/en/papers/tyser-2024-ai-driven-review-systems/