Elamur Research⌕
☰
home / papers / 2024 / Monitoring AI-Modified Content at Scale: A Case Study on the Impact of ChatGPT on AI Conference Peer Reviews

Monitoring AI-Modified Content at Scale: A Case Study on the Impact of ChatGPT on AI Conference Peer Reviews

Paper · 2024 · Weixin Liang, Zachary Izzo, Yaohui Zhang et al.

Summary

Detecting LLM-written text one document at a time is unreliable, and humans perform barely better than chance at it. So the scale of real-world LLM use has been hard to measure. The authors therefore estimate usage for a whole corpus rather than for single texts. Their framework, distributional GPT quantification, builds word-frequency distributions from reference texts known to be written by human experts or generated by AI. It then applies maximum likelihood estimation to infer what share of a target corpus is AI-generated or heavily AI-edited, without classifying any individual review.

They applied the method to peer reviews from ICLR 2024, NeurIPS 2023, CoRL 2023 and EMNLP 2023, and used Nature portfolio journal reviews as a control. Between 6.5% and 16.9% of review text at these AI venues appears substantially modified by LLMs, beyond simple grammar fixes. The Nature reviews show no such signal. Higher estimated LLM use coincides with:

  • low reviewer confidence
  • submission close to the deadline
  • lower engagement with author rebuttals

The approach offers a cheap, scalable way to monitor AI influence on information ecosystems.

Key points

  • Venue estimates: About 10.6% of ICLR 2024 review sentences and 16.9% of EMNLP 2023 review sentences are estimated to be substantially LLM-modified. Nature portfolio reviews show no significant evidence of ChatGPT use.
  • Efficiency and accuracy: The estimator is roughly 10 million times (7 orders of magnitude) cheaper computationally than state-of-the-art instance-level detectors. It also cuts estimation error by a factor of 3.4 in-distribution and 4.6 out-of-distribution.
  • Adjective signal: Certain adjectives jumped sharply in ICLR 2024 reviews, including "commendable" (9.8×), "meticulous" (34.7×) and "intricate" (11.2×). Adjectives serve as the main features, although adverbs, verbs and non-technical nouns give similar results.
  • Behavioural correlates:
    • Estimated AI use rises within 3 days of review deadlines.
    • It is higher among reviewers who skip ICLR/NeurIPS author rebuttals.
    • It is lower in reviews that cite scholarly work.
    • It is linked to more homogeneous review content in embedding space.
  • Robustness and availability: A model trained only on GPT-3.5 data also detects GPT-4 text accurately. The paper appeared at ICML 2024 (PMLR 235), and the code is available on GitHub (Weixin-Liang/Mapping-the-Increasing-Use-of-LLMs-in-Scientific-Papers).

Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.

 Institutions

🎓Stanford UniversityStanford · United States506 works · 330,053 citations🎓University of California, Santa BarbaraSanta Barbara · United States67 works · 10,550 citations🏢NEC (United States)Irving · United States10 works · 658 citations

 Fields, topics and concepts

📚 Shelves 6The field of science this work belongs to (OpenAlex fields).Computer Science · 18,614 worksArtificial Intelligence · 17,965 worksArts and Humanities · 17,450 worksLanguage and Linguistics · 17,296 worksSocial Sciences · 1,954 worksSafety Research · 626 works

Cite

Cite as Weixin Liang, Zachary Izzo, Yaohui Zhang, Haley Lepp, Hancheng Cao, Xuandong Zhao, Lingjiao Chen, Haotian Ye, Sheng Liu, Zhi Huang, Daniel A. McFarland and James Y. Zou. 2024. Monitoring AI-Modified Content at Scale: A Case Study on the Impact of ChatGPT on AI Conference Peer Reviews. arXiv:2403.07183.

The paper itself, not this page. BibTeX from the library's record of the work. DOI 10.48550/arxiv.2403.07183 · arXiv 2403.07183

@misc{liang2024monitoring,
  title = {{Monitoring AI-Modified Content at Scale: A Case Study on the Impact of ChatGPT on AI Conference Peer Reviews}},
  author = {Weixin Liang, Zachary Izzo, Yaohui Zhang, Haley Lepp, Hancheng Cao, Xuandong Zhao, Lingjiao Chen, Haotian Ye, Sheng Liu, Zhi Huang, Daniel A. McFarland and James Y. Zou},
  year = {2024},
  doi = {10.48550/arxiv.2403.07183},
  eprint = {2403.07183},
  archivePrefix = {arXiv},
  url = {https://doi.org/10.48550/arxiv.2403.07183}
}

References

87 references, 31 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

  1. Arora, A., Kaffee, L.-A., and Augenstein, I. Probing pre-trained language models for cross-cultural differences in values. arXiv preprint arXiv:2203.13722, 2022.
  2. Atallah, M. J., Raskin, V., Crogan, M., Hempelmann, C. F., Kerschbaum, F., Mohamed, D., and Naik, S. Natural Language Watermarking: Design, Analysis, and a Proof-of-Concept Implementation. In Information Hiding, 2001. find
  3. Atallah, M. J., Raskin, V., Hempelmann, C. F., Topkara, M., Sion, R., Topkara, U., and Triezenberg, K. E. Natural Language Watermarking and Tamperproofing. In Information Hiding, 2002. find
  4. Badaskar, S., Agarwal, S., and Arora, S. Identifying Real or Fake Articles: Towards better Language Modeling. In International Joint Conference on Natural Language Processing, 2008. URL https://api.semanticscholar.org/CorpusID:4324753. ↗
  5. Bakhtin, A., Gross, S., Ott, M., Deng, Y., Ranzato, M., and Szlam, A. Real or Fake? Learning to Discriminate Machine from Human Generated Text. ArXiv, abs/1906.03351, 2019. URL https://api.semanticscholar.org/CorpusID: 182952342. — [arXiv:1906.03351](https://arxiv.org/abs/1906.03351) ↗
  6. Bao, G., Zhao, Y., Teng, Z., Yang, L., and Zhang, Y. Fast-DetectGPT: Efficient Zero-Shot Detection of Machine-Generated Text via Conditional Probability Curvature. ArXiv, abs/2310.05130, 2023. URL https://api.semanticscholar.org/CorpusID:263831345. ↗
  7. Bearman, M., Ryan, J., and Ajjawi, R. Discourses of artificial intelligence in higher education: A critical literature review. Higher Education, 86(2):369–385, 2023. find
  8. Beresneva, D. Computer-Generated Text Detection Using Machine Learning: A Systematic Review. In International Conference on Applications of Natural Language to Data Bases, 2016. URL https://api.semanticscholar.org/CorpusID:1175726. ↗
  9. Bhagat, R. and Hovy, E. H. Squibs: What Is a Paraphrase? Computational Linguistics, 39:463–472, 2013. URL https://api.semanticscholar.org/CorpusID:32452685. ↗
  10. Bhattacharjee, A., Kumarage, T., Moraffah, R., and Liu, H. ConDA: Contrastive Domain Adaptation for AI-generated Text Detection. ArXiv, abs/2309.03992, 2023. URL https://api.semanticscholar.org/CorpusID:261660497. ↗
  11. Bommasani, R., Creel, K. A., Kumar, A., Jurafsky, D., and Liang, P. S. Picking on the same person: Does algorithmic monoculture lead to outcome homogenization? Advances in Neural Information Processing Systems, 35: 3663–3678, 2022. find
  12. Cantor, M. Nearly 50 news websites are ‘AI-generated’, a study says. Would I be able to tell?, 2023. URL https://www.theguardian.com/technology/2023/may/ 08/ai-generated-news-websites-study. Accessed: 2024-02-24. ↗
  13. Cao, Y., Zhou, L., Lee, S., Cabello, L., Chen, M., and Hershcovich, D. Assessing cross-cultural alignment between chatgpt and human societies: An empirical study, 2023.
  14. Chakraborty, S., Bedi, A. S., Zhu, S., An, B., Manocha, D., and Huang, F. On the possibilities of ai-generated text detection. arXiv preprint arXiv:2304.04736, 2023. — [arXiv:2304.04736](https://arxiv.org/abs/2304.04736) find
  15. Chen, Y., Kang, H., Zhai, V., Li, L., Singh, R., and Ramakrishnan, B. GPT-Sentinel: Distinguishing Human and ChatGPT Generated Content. ArXiv, abs/2305.07969, 2023. URL https://api.semanticscholar.org/CorpusID:258686680. — [arXiv:2305.07969](https://arxiv.org/abs/2305.07969) ↗
  16. Chiang, Y.-L., Chang, L.-P., Hsieh, W.-T., and Chen, W.-C. Natural Language Watermarking Using Semantic Substitution for Chinese Text. In International Workshop on Digital Watermarking, 2003. URL https://api.semanticscholar.org/CorpusID:40971354. ↗
  17. Christin, A. What data can do: A typology of mechanisms. International Journal of Communication, 14:20, 2020. find
  18. Clark, E., August, T., Serrano, S., Haduong, N., Gururangan, S., and Smith, N. A. All That’s ‘Human’Is Not Gold: Evaluating Human Evaluation of Generated Text. In Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Conference on Na… find
  19. Crothers, E., Japkowicz, N., and Viktor, H. Machine Generated Text: A Comprehensive Survey of Threat Models and Detection Methods. arXiv preprint arXiv:2210.07321, 2022. find
  20. Dycke, N., Kuznetsov, I., and Gurevych, I. Yes-yes-yes: Proactive data collection for ACL rolling review and beyond. In Goldberg, Y., Kozareva, Z., and Zhang, Y. (eds.), Findings of the Association for Computational Linguistics: EMNLP 2022, pp. 300–318, Abu Dhabi, United Arab Emirates, December 2022… find
  21. Dycke, N., Kuznetsov, I., and Gurevych, I. NLPeer: A unified resource for the computational study of peer review. In Rogers, A., Boyd-Graber, J., and Okazaki, N. (eds.), Proceedings of the 61st Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pp. 5049–5073, To…
  22. Else, H. Abstracts written by ChatGPT fool scientists. Nature, Jan 2023. URL https://www.nature.com/articles/d41586-023-00056-7. ↗
  23. Fagni, T., Falchi, F., Gambini, M., Martella, A., and Tesconi, M. TweepFake: About detecting deepfake tweets. Plos one, 16(5):e0251415, 2021.
  24. Fernandez, P., Chaffin, A., Tit, K., Chappelier, V., and Furon, T. Three Bricks to Consolidate Watermarks for Large Language Models. 2023 IEEE International Workshop on Information Forensics and Security (WIFS), pp. 1– 6, 2023. URL https://api.semanticscholar.org/CorpusID:260351507. ↗
  25. Fu, Y., Xiong, D., and Dong, Y. Watermarking Conditional Text Generation for AI Detection: Unveiling Challenges and a Semantic-Aware Watermark Remedy. ArXiv, abs/2307.13808, 2023. URL https://api.semanticscholar.org/CorpusID: 260164516. — [arXiv:2307.13808](https://arxiv.org/abs/2307.13808) ↗
  26. Gao, C. A., Howard, F. M., Markov, N. S., Dyer, E. C., Ramesh, S., Luo, Y., and Pearson, A. T. Comparing scientific abstracts generated by ChatGPT to original abstracts using an artificial intelligence output detector, plagiarism detector, and blinded human reviewers. bioRxiv, pp. 2022–12, 2022. find
  27. Gehrmann, S., Strobelt, H., and Rush, A. M. GLTR: Statistical Detection and Visualization of Generated Text. In Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics: System Demonstrations, pp. 111–116, 2019.
  28. Heikkilä, M. How to spot ai-generated text. MIT Technology Review, Dec 2022. URL https://www.technologyreview.com/2022/12/19/1065596/ how-to-spot-ai-generated-text/. ↗
  29. Hofmann, V., Kalluri, P. R., Jurafsky, D., and King, S. Dialect prejudice predicts AI decisions about people’s character, employability, and criminality, 2024. — Dialect prejudice predicts AI decisions about people's character, employability, and criminality
  30. Hou, A. B., Zhang, J., He, T., Wang, Y., Chuang, Y.-S., Wang, H., Shen, L., Durme, B. V., Khashabi, D., and Tsvetkov, Y. SemStamp: A Semantic Watermark with Paraphrastic Robustness for Text Generation. ArXiv, abs/2310.03991, 2023. URL https://api.semanticscholar.org/CorpusID: 263831179. ↗
  31. Hu, X., Chen, P.-Y., and Ho, T.-Y. RADAR: Robust AI-Text Detection via Adversarial Learning. ArXiv, abs/2307.03838, 2023a. URL https://api.semanticscholar.org/CorpusID: 259501842. ↗
  32. Hu, Z., Chen, L., Wu, X., Wu, Y., Zhang, H., and Huang, H. Unbiased Watermark for Large Language Models. ArXiv, abs/2310.10669, 2023b. URL https://api.semanticscholar.org/CorpusID:264172471. — [arXiv:2310.10669](https://arxiv.org/abs/2310.10669) ↗
  33. Ippolito, D., Duckworth, D., Callison-Burch, C., and Eck, D. Automatic detection of generated text is easiest when humans are fooled. arXiv preprint arXiv:1911.00650, 2019.
  34. Jawahar, G., Abdul-Mageed, M., and Lakshmanan, L. V. Automatic detection of machine generated text: A critical survey. arXiv preprint arXiv:2011.01314, 2020.
  35. Kelly, S. M. ChatGPT creator pulls AI detection tool due to ‘low rate of accuracy’. CNN Business, Jul 2023. URL https://www.cnn.com/2023/07/25/tech/openai-ai-detection-tool/index.html. ↗
  36. Kirchenbauer, J., Geiping, J., Wen, Y., Katz, J., Miers, I., and Goldstein, T. A watermark for large language models. International Conference on Machine Learning, 2023.
  37. Kirchner, J. H., Ahmad, L., Aaronson, S., and Leike, J. New AI classifier for indicating AI-written text, 2023. OpenAI. find
  38. Kleinberg, J. and Raghavan, M. Algorithmic monoculture and social welfare. Proceedings of the National Academy of Sciences, 118(22):e2018340118, 2021. find
  39. Kreps, S., McCain, R., and Brundage, M. All the news that’s fit to fabricate: Ai-generated text as a tool of media misinformation. Journal of Experimental Political Science, 9(1):104–117, 2022. doi: 10.1017/XPS.2020.37. — [doi:10.1017/xps.2020.37](https://doi.org/10.1017/xps.2020.37) find
  40. Kuditipudi, R., Thickstun, J., Hashimoto, T., and Liang, P. Robust Distortion-free Watermarks for Language Models. ArXiv, abs/2307.15593, 2023. URL https://api.semanticscholar.org/CorpusID:260315804. ↗
  41. Lamont, M. How professors think: Inside the curious world of academic judgment. Harvard University Press, 2009. find
  42. Lamont, M. Toward a comparative sociology of valuation and evaluation. Annual review of sociology, 38:201–221, 2012. find
  43. Lavergne, T., Urvoy, T., and Yvon, F. Detecting Fake Content with Relative Entropy Scoring. In Pan, 2008. URL https://api.semanticscholar.org/CorpusID:12098535. ↗
  44. Li, Y., Li, Q., Cui, L., Bi, W., Wang, L., Yang, L., Shi, S., and Zhang, Y. Deepfake Text Detection in the Wild. ArXiv, abs/2305.13242, 2023. URL https://api.semanticscholar.org/CorpusID:258832454. — [arXiv:2305.13242](https://arxiv.org/abs/2305.13242) ↗
  45. Liang, W., Yuksekgonul, M., Mao, Y., Wu, E., and Zou, J. Y. GPT detectors are biased against non-native English writers. ArXiv, abs/2304.02819, 2023a.
  46. Liang, W., Zhang, Y., Cao, H., Wang, B., Ding, D., Yang, X., Vodrahalli, K., He, S., Smith, D., Yin, Y., McFarland, D., and Zou, J. Can large language models provide useful feedback on research papers? A large-scale empirical analysis. In arXiv preprint arXiv:2310.01783, 2023b.
  47. Liang, W., Zhang, Y., Cao, H., Wang, B., Ding, D., Yang, X., Vodrahalli, K., He, S., Smith, D., Yin, Y., et al. Can large language models provide useful feedback on research papers? A large-scale empirical analysis. arXiv preprint arXiv:2310.01783, 2023c.
  48. Lin, B. Y., Ravichander, A., Lu, X., Dziri, N., Sclar, M., Chandu, K., Bhagavatula, C., and Choi, Y. The unlocking spell on base llms: Rethinking alignment via in-context learning. arXiv preprint arXiv:2312.01552, 2023.
  49. Liu, A., Pan, L., Hu, X., Meng, S., and Wen, L. A Semantic Invariant Robust Watermark for Large Language Models. ArXiv, abs/2310.06356, 2023. URL https://api.semanticscholar.org/CorpusID:263830310. — [arXiv:2310.06356](https://arxiv.org/abs/2310.06356) ↗
  50. Liu, Q., Zhou, Y., Huang, J., and Li, G. When chatgpt is gone: Creativity reverts and homogeneity persists, 2024. find
  51. Liu, R. and Shah, N. B. Reviewergpt? an exploratory study on using large language models for paper reviewing. arXiv preprint arXiv:2306.00622, 2023.
  52. Liu, X., Zhang, Z., Wang, Y., Lan, Y., and Shen, C. CoCo: Coherence-Enhanced Machine-Generated Text Detection Under Data Limitation With Contrastive Learning. ArXiv, abs/2212.10341, 2022. URL https://api.semanticscholar.org/CorpusID:254877728. — [arXiv:2212.10341](https://arxiv.org/abs/2212.10341) ↗
  53. Liu, Y., Ott, M., Goyal, N., Du, J., Joshi, M., Chen, D., Levy, O., Lewis, M., Zettlemoyer, L., and Stoyanov, V. RoBERTa: A Robustly Optimized BERT Pretraining Approach. ArXiv, abs/1907.11692, 2019.
  54. Longino, H. E. Science as social knowledge: Values and objectivity in scientific inquiry. Princeton university press, 1990. find
  55. Messeri, L. and Crockett, M. J. Artificial intelligence and illusions ofunderstanding in scientific research. Nature, 627:49–58, 2024. find
  56. Mitchell, E., Lee, Y., Khazatsky, A., Manning, C. D., and Finn, C. DetectGPT: Zero-Shot Machine-Generated Text Detection using Probability Curvature. ArXiv, abs/2301.11305, 2023a.
  57. Mitchell, E., Lee, Y., Khazatsky, A., Manning, C. D., and Finn, C. DetectGPT: Zero-shot machine-generated text detection using probability curvature. arXiv preprint arXiv:2301.11305, 2023b.
  58. Naous, T., Ryan, M. J., Ritter, A., and Xu, W. Having beer after prayer? measuring cultural bias in large language models, 2024.
  59. Newman, D. J. The double dixie cup problem. The American Mathematical Monthly, 67(1):58–61, 1960. find
  60. NewsGuard. Tracking AI-enabled Misinformation: 713 ‘Unreliable AI-Generated News’ Websites (and Counting), Plus the Top False Narratives Generated by Artificial Intelligence Tools, 2023. URL https://www.newsguardtech.com/special-reports/ai-tracking-center/. Accessed: 2024-02-24. ↗
  61. OpenAI. GPT-2: 1.5B release. https://openai.com/research/gpt-2-1-5b-release, 2019. Accessed: 2019-11-05. ↗
  62. Papadimitriou, I., Lopez, K., and Jurafsky, D. Multilingual bert has an accent: Evaluating english influences on fluency in multilingual models, 2023. find
  63. Raffel, C., Shazeer, N., Roberts, A., Lee, K., Narang, S., Matena, M., Zhou, Y., Li, W., and Liu, P. J. Exploring the limits of transfer learning with a unified text-to-text transformer. The Journal of Machine Learning Research, 21(1):5485–5551, 2020.
  64. Sadasivan, V. S., Kumar, A., Balasubramanian, S., Wang, W., and Feizi, S. Can AI-Generated Text be Reliably Detected? ArXiv, abs/2303.11156, 2023.
  65. Shi, Z., Wang, Y., Yin, F., Chen, X., Chang, K.-W., and Hsieh, C.-J. Red Teaming Language Model Detectors with Language Models. ArXiv, abs/2305.19713, 2023. URL https://api.semanticscholar.org/CorpusID:258987266. ↗
  66. Singh, K. and Zou, J. New Evaluation Metrics Capture Quality Degradation due to LLM Watermarking. arXiv preprint arXiv:2312.02382, 2023. — [arXiv:2312.02382](https://arxiv.org/abs/2312.02382) find
  67. Solaiman, I., Brundage, M., Clark, J., Askell, A., Herbert-Voss, A., Wu, J., Radford, A., Krueger, G., Kim, J. W., Kreps, S., et al. Release strategies and the social impacts of language models. arXiv preprint arXiv:1908.09203, 2019a.
  68. Solaiman, I., Brundage, M., Clark, J., Askell, A., Herbert-Voss, A., Wu, J., Radford, A., and Wang, J. Release Strategies and the Social Impacts of Language Models. ArXiv, abs/1908.09203, 2019b. URL https://api.semanticscholar.org/CorpusID:201666234. ↗
  69. Sulik, J., Rim, N., Pontikes, E., Evans, J., and Lupyan, G. Why do scientists disagree? PsyArXiv, 2023. Preprint. find
  70. Teplitskiy, M., Acuna, D., Elamrani-Raoult, A., Körding, K., and Evans, J. The sociology of scientific validity: How professional networks shape judgement in peer review. Research Policy, 47(9):1825–1841, 2018. find
  71. Topkara, M., Riccardi, G., Hakkani-Tür, D. Z., and Atallah, M. J. Natural language watermarking: challenges in building a practical system. In Electronic imaging, 2006a. URL https://api.semanticscholar.org/CorpusID:16650373. ↗
  72. Topkara, U., Topkara, M., and Atallah, M. J. The hiding virtues of ambiguity: quantifiably resilient watermarking of natural language text through synonym substitutions. In Workshop on Multimedia & Security, 2006b. find
  73. Tulchinskii, E., Kuznetsov, K., Kushnareva, L., Cherniavskii, D., Barannikov, S., Piontkovskaya, I., Nikolenko, S. I., and Burnaev, E. Intrinsic Dimension Estimation for Robust Detection of AI-Generated Texts. ArXiv, abs/2306.04723, 2023. URL https://api.semanticscholar.org/CorpusID: 259108779. ↗
  74. Uchendu, A., Le, T., Shu, K., and Lee, D. Authorship Attribution for Neural Text Generation. In Conference on Empirical Methods in Natural Language Processing, 2020. URL https://api.semanticscholar.org/CorpusID:221835708. ↗
  75. Van Noorden, R. and Perkel, J. M. Ai and science: what 1,600 researchers think. Nature, 621(7980):672–675, 2023. find
  76. Van Rossum, D. Generative AI Top 150: The World’s Most Used AI Tools. https://www.flexos.work/learn/generative-ai-top-150, February 2024. ↗
  77. Walters, W. H. and Wilder, E. I. Fabrication and errors in the bibliographic citations generated by ChatGPT. Scientific Reports, 13(1):14045, 2023. find
  78. Weber-Wulff, D., Anohina-Naumeca, A., Bjelobaba, S., Foltýnek, T., Guerrero-Dib, J., Popoola, O., Šigut, P., and Waddington, L. Testing of detection tools for AI-generated text. International Journal for Educational Integrity, 19(1):26, 2023. ISSN 1833-2595. doi: 10.1007/s40979-023-00146-z. URL ht… — [doi:10.1007/s40979-023-00146-z](https://doi.org/10.1007/s40979-023-00146-z) find
  79. Wolff, M. Attacking Neural Text Detectors. abs/2002.11768, 2020. — [arXiv:2002.11768](https://arxiv.org/abs/2002.11768) find
  80. ArXiv, Wu, Y., Hu, Z., Zhang, H., and Huang, H. DiPmark: A Stealthy, Efficient and Resilient Watermark for Large Language Models. ArXiv, abs/2310.07710, 2023. URL https://api.semanticscholar.org/CorpusID:263834753. — [arXiv:2310.07710](https://arxiv.org/abs/2310.07710) ↗
  81. Yang, X., Cheng, W., Petzold, L., Wang, W. Y., and Chen, H. DNA-GPT: Divergent N-Gram Analysis for Training-Free Detection of GPT-Generated Text. ArXiv, abs/2305.17359, 2023a. URL https://api.semanticscholar.org/CorpusID: 258960101. — [arXiv:2305.17359](https://arxiv.org/abs/2305.17359) ↗
  82. Yang, X., Pan, L., Zhao, X., Chen, H., Petzold, L. R., Wang, W. Y., and Cheng, W. A Survey on Detection of LLMs-Generated Content. ArXiv, abs/2310.15654, 2023b. URL https://api.semanticscholar.org/CorpusID:264439179. ↗
  83. Yoo, K., Ahn, W., Jang, J., and Kwak, N. J. Robust Multi-bit Natural Language Watermarking through Invariant Features. In Annual Meeting of the Association for Computational Linguistics, 2023. URL https://api.semanticscholar.org/CorpusID:259129912. ↗
  84. Yu, X., Qi, Y., Chen, K., Chen, G., Yang, X., Zhu, P., Zhang, W., and Yu, N. H. GPT Paternity Test: GPT Generated Text Detection with GPT Genetic Inheritance. ArXiv, abs/2305.12519, 2023. URL https://api.semanticscholar.org/CorpusID:258833423. — [arXiv:2305.12519](https://arxiv.org/abs/2305.12519) ↗
  85. Zellers, R., Holtzman, A., Rashkin, H., Bisk, Y., Farhadi, A., Roesner, F., and Choi, Y. Defending Against Neural Fake News. ArXiv, abs/1905.12616, 2019. URL https://api.semanticscholar.org/CorpusID:168169824. ↗
  86. Zhang, Y.-F., Zhang, Z., Wang, L., Tan, T.-P., and Jin, R. Assaying on the Robustness of Zero-Shot Machine-Generated Text Detectors. ArXiv, abs/2312.12918, 2023. URL https://api.semanticscholar.org/CorpusID:266375086. — [arXiv:2312.12918](https://arxiv.org/abs/2312.12918) ↗
  87. Zhao, X., Ananth, P. V., Li, L., and Wang, Y.-X. Provable Robust Watermarking for AI-Generated Text. ArXiv, abs/2306.17439, 2023. URL https://api.semanticscholar.org/CorpusID: 259308864. ↗

Updated

↑ back to top

This page in Markdown → https://md.research.elamur.ai/free/en/papers/liang-2024-monitoring-ai-modified-content/