Elamur Research⌕
☰
home / papers / 2024 / Spotting LLMs With Binoculars: Zero-Shot Detection of Machine-Generated Text

Spotting LLMs With Binoculars: Zero-Shot Detection of Machine-Generated Text

Paper · 2024 · Abhimanyu Hans, Avi Schwarzschild, Valeriia Cherepanova et al.

Summary

The paper addresses how to tell machine-written text from human writing when no examples from the generating model are available. Detectors trained on ChatGPT outputs often fail on other models. Plain perplexity breaks down because an unseen prompt can make LLM output look surprising, which the authors call the "capybara problem."

Their method, Binoculars, uses two related pre-trained LLMs that share a tokenizer. An "observer" model computes the log-perplexity of the text. The observer also scores the next-token distributions of a "performer" model, a quantity called cross-perplexity. The detection score is the ratio of perplexity to cross-perplexity, which normalizes for how predictable a given context is. The method needs no training data and uses unmodified Falcon models.

On News, Creative Writing and Student Essay data, Binoculars catches more than 90% of ChatGPT-generated samples at a 0.01% false-positive rate. It beats Ghostbuster, GPTZero, DetectGPT, Fast-DetectGPT and DNA-GPT, even though it never saw ChatGPT data. Because one detector covers many LLMs, it is useful for content moderation.

Key points

  • Score: Binoculars divides observer perplexity by observer–performer cross-perplexity. Both models must share a tokenizer. The code is at github.com/ahans30/Binoculars, and the paper appeared at ICML 2024.
  • Evaluation metric: The authors report the true-positive rate at a very low false-positive rate (0.01%) rather than accuracy or AUC. They argue that false accusations against human writers are the main harm to avoid.
  • Non-native writing: Commercial detectors have flagged TOEFL essays as machine-written 48–76% of the time (Liang et al.). Binoculars labels essays by non-native speakers as human, both before and after grammar correction.
  • Stylistic prompts: Prompts asking for a particular style (Carl Sagan's voice, pirate speech) barely change results. The worst case, pirate style, lowers sensitivity by about 1%.
  • Limitations:
    • No tests with open models of 30B+ parameters.
    • No guarantees against deliberate evasion.
    • Source code was not studied.
    • Predictions come with no explanation.

Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.

 Institutions

🎓Carnegie Mellon UniversityPittsburgh · United States598 works · 191,225 citations🎓New York UniversityNew York · United States219 works · 103,487 citations🎓University of Maryland, College ParkCollege Park · United States141 works · 29,174 citations🔬ELLIS Institute TübingenTübingen · Germany4 works · 226 citations

 Fields, topics and concepts

📚 Shelves 5The field of science this work belongs to (OpenAlex fields).Computer Science · 18,614 worksArtificial Intelligence · 17,965 worksArts and Humanities · 17,450 worksLanguage and Linguistics · 17,296 worksComputer Vision and Pattern Recognition · 3,486 works

Cite

Cite as Abhimanyu Hans, Avi Schwarzschild, Valeriia Cherepanova, Hamid Kazemi, Aniruddha Saha, Micah Goldblum, Jonas Geiping and Tom Goldstein. 2024. Spotting LLMs With Binoculars: Zero-Shot Detection of Machine-Generated Text. arXiv:2401.12070.

The paper itself, not this page. BibTeX from the library's record of the work. DOI 10.48550/arxiv.2401.12070 · arXiv 2401.12070

@misc{hans2024spotting,
  title = {{Spotting LLMs With Binoculars: Zero-Shot Detection of Machine-Generated Text}},
  author = {Abhimanyu Hans, Avi Schwarzschild, Valeriia Cherepanova, Hamid Kazemi, Aniruddha Saha, Micah Goldblum, Jonas Geiping and Tom Goldstein},
  year = {2024},
  doi = {10.48550/arxiv.2401.12070},
  eprint = {2401.12070},
  archivePrefix = {arXiv},
  url = {https://doi.org/10.48550/arxiv.2401.12070}
}

References

52 references, 21 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

  1. Almazrouei, E., Alobeidli, H., Alshamsi, A., Cappelli, A., Cojocaru, R., Debbah, M., Goffinet, E., Heslow, D., Launay, J., Malartic, Q., Noune, B., Pannier, B., and Penedo, G. Falcon-40B: An open large language model with state-of-the-art performance, 2023. URL https://falconllm.tii.ae/. ↗
  2. Bail, C., Pinheiro, L., and Royer, J. Difficulty Of Detecting AI Content Poses Legal Challenges. Law360, April 2023. find
  3. Bao, G., Zhao, Y., Teng, Z., Yang, L., and Zhang, Y. Fast-detectgpt: Efficient zero-shot detection of machine-generated text via conditional probability curvature. In The Twelfth International Conference on Learning Representations, 2023.
  4. Bhat, M. M. and Parthasarathy, S. How Effectively Can Machines Defend Against Machine-Generated Fake News? An Empirical Study. In Proceedings of the First Workshop on Insights from Negative Results in NLP, pp. 48–53, Online, November 2020. Association for Computational Linguistics. doi: 10.18653/v1/… find
  5. Brown, T. B., Mann, B., Ryder, N., Subbiah, M., Kaplan, J., Dhariwal, P., Neelakantan, A., Shyam, P., Sastry, G., Askell, A., Agarwal, S., Herbert-Voss, A., Krueger, G., Henighan, T., Child, R., Ramesh, A., Ziegler, D. M., Wu, J., Winter, C., Hesse, C., Chen, M., Sigler, E., Litwin, M., Gray, S., Ch… — Language Models are Few-Shot Learners
  6. Chakraborty, S., Bedi, A. S., Zhu, S., An, B., Manocha, D., and Huang, F. On the Possibilities of AI-Generated Text Detection. arxiv:2304.04736[cs], April 2023. doi: 10.48550/arXiv.2304.04736. URL http://arxiv.org/abs/2304.04736. — [doi:10.48550/arxiv.2304.04736](https://doi.org/10.48550/arxiv.2304.04736) ↗
  7. Chen, C., Borgeaud, S., Irving, G., Lespiau, J.-B., Sifre, L., and Jumper, J. Accelerating Large Language Model Decoding with Speculative Sampling. arxiv:2302.01318[cs], February 2023. doi: 10.48550/arXiv.2302.01318. URL http://arxiv.org/abs/2302.01318. ↗
  8. Chowdhery, A., Narang, S., Devlin, J., Bosma, M., Mishra, G., Roberts, A., Barham, P., Chung, H. W., Sutton, C., Gehrmann, S., Schuh, P., Shi, K., Tsvyashchenko, S., Maynez, J., Rao, A., Barnes, P., Tay, Y., Shazeer, N., Prabhakaran, V., Reif, E., Du, N., Hutchinson, B., Pope, R., Bradbury, J., Aust… — PaLM: Scaling Language Modeling with Pathways
  9. Crothers, E., Japkowicz, N., and Viktor, H. Machine Generated Text: A Comprehensive Survey of Threat Models and Detection Methods. arxiv:2210.07321[cs], November 2022. doi: 10.48550/arXiv.2210.07321. URL http://arxiv.org/abs/2210.07321. ↗
  10. Dhaini, M., Poelman, W., and Erdogan, E. Detecting ChatGPT: A Survey of the State of Detecting ChatGPT-Generated Text. arxiv:2309.07689[cs], September 2023. doi: 10.48550/arXiv.2309.07689. URL http://arxiv.org/abs/2309.07689. ↗
  11. EssayForum. Nid989/EssayFroum-Dataset · Datasets at Hugging Face, September 2022. URL https://huggingface.co/datasets/nid989/EssayFroum-Dataset. ↗
  12. Gehrmann, S., Strobelt, H., and Rush, A. GLTR: Statistical detection and visualization of generated text. In Costa-jussà, M. R. and Alfonseca, E. (eds.), Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics: System Demonstrations, pp. 111– 116, Florence, Italy, Ju…
  13. Ghosal, S. S., Chakraborty, S., Geiping, J., Huang, F., Manocha, D., and Bedi, A. S. Towards possibilities & impossibilities of ai-generated text detection: A survey. arXiv preprint arXiv:2310.15264, 2023. — [arXiv:2310.15264](https://arxiv.org/abs/2310.15264) find
  14. Guo, B., Zhang, X., Wang, Z., Jiang, M., Nie, J., Ding, Y., Yue, J., and Wu, Y. How Close is ChatGPT to Human Experts? Comparison Corpus, Evaluation, and Detection. arxiv:2301.07597[cs], January 2023. doi: 10.48550/arXiv.2301.07597. URL http://arxiv.org/abs/2301.07597. ↗
  15. Hamborg, F., Meuschke, N., Breitinger, C., and Gipp, B. news-please: A generic news crawler and extractor. In Proceedings of the 15th International Symposium of Information Science, pp. 218–223, March 2017. doi: 10.5281/zenodo.4120316. — [doi:10.5281/zenodo.4120316](https://doi.org/10.5281/zenodo.4120316) find
  16. Helm, H., Priebe, C. E., and Yang, W. A Statistical Turing Test for Generative Models. arxiv:2309.08913[cs], September 2023. doi: 10.48550/arXiv.2309.08913. URL http://arxiv.org/abs/2309.08913. — [doi:10.48550/arxiv.2309.08913](https://doi.org/10.48550/arxiv.2309.08913) ↗
  17. Hermann, K. M., Kočiský, T., Grefenstette, E., Espeholt, L., Kay, W., Suleyman, M., and Blunsom, P. Teaching machines to read and comprehend. In Proceedings of the 28th International Conference on Neural Information Processing Systems - Volume 1, NIPS’15, pp. 1693–1701, Cambridge, MA, USA, Decembe…
  18. Horne, B. D., Nørregaard, J., and Adali, S. Robust fake news detection over time and attack. ACM Trans. Intell. Syst. Technol., 11(1), dec 2019. ISSN 2157-6904. doi: 10.1145/3363818. URL https://doi.org/10.1145/3363818. — [doi:10.1145/3363818](https://doi.org/10.1145/3363818) ↗
  19. Hu, X., Chen, P.-Y., and Ho, T.-Y. RADAR: Robust AI-Text Detection via Adversarial Learning. arxiv:2307.03838[cs], July 2023. doi: 10.48550/arXiv.2307.03838. URL http://arxiv.org/abs/2307.03838. ↗
  20. Kirchenbauer, J., Geiping, J., Wen, Y., Katz, J., Miers, I., and Goldstein, T. A Watermark for Large Language Models. In Proceedings of the 40th International Conference on Machine Learning, pp. 17061–17084. PMLR, July 2023. URL https://proceedings.mlr.press/v202/kirchenbauer23a.html. ↗
  21. Krishna, K., Song, Y., Karpinska, M., Wieting, J., and Iyyer, M. Paraphrasing evades detectors of AI-generated text, but retrieval is an effective defense. arxiv:2303.13408[cs], March 2023. doi: 10.48550/arXiv.2303.13408. URL http://arxiv.org/abs/2303.13408. ↗
  22. Leviathan, Y., Kalman, M., and Matias, Y. Fast Inference from Transformers via Speculative Decoding. In Proceedings of the 40th International Conference on Machine Learning, pp. 19274–19286. PMLR, July 2023. URL https://proceedings.mlr.press/v202/leviathan23a.html. ↗
  23. Li, J. S., Monaco, J. V., Chen, L.-C., and Tappert, C. C. Authorship authentication using short messages from social networking sites. In 2014 IEEE 11th International Conference on e-Business Engineering, pp. 314–319, 2014. doi: 10.1109/ICEBE.2014.61. — [doi:10.1109/icebe.2014.61](https://doi.org/10.1109/icebe.2014.61) find
  24. Li, X. L., Holtzman, A., Fried, D., Liang, P., Eisner, J., Hashimoto, T., Zettlemoyer, L., and Lewis, M. Contrastive Decoding: Open-ended Text Generation as Optimization. In Proceedings of the 61st Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pp. 12286–123… find
  25. Lian, W., Goodson, B., Pentland, E., Cook, A., Vong, C., and ”Teknium”. Openorca: An open dataset of gpt augmented flan reasoning traces. https://https://huggingface.co/Open-Orca/OpenOrca, 2023. find
  26. Liang, W., Yuksekgonul, M., Mao, Y., Wu, E., and Zou, J. GPT detectors are biased against non-native English writers. arxiv:2304.02819[cs], April 2023. doi: 10.48550/arXiv.2304.02819. URL http://arxiv.org/abs/2304.02819. ↗
  27. Liu, Y., Ott, M., Goyal, N., Du, J., Joshi, M., Chen, D., Levy, O., Lewis, M., Zettlemoyer, L., and Stoyanov, V. Roberta: A robustly optimized bert pretraining approach, 2019.
  28. Liyanage, V. and Buscaldi, D. Detecting Artificially Generated Academic Text: The Importance of Mimicking Human Utilization of Large Language Models. In Métais, E., Meziane, F., Sugumaran, V., Manning, W., and Reiff-Marganiec, S. (eds.), Natural Language Processing and Information Systems, Lecture… — [doi:10.1007/978-3-031-35320-8](https://doi.org/10.1007/978-3-031-35320-8) find
  29. Mitchell, E., Lee, Y., Khazatsky, A., Manning, C. D., and Finn, C. DetectGPT: Zero-Shot Machine-Generated Text Detection using Probability Curvature. In Proceedings of the 40th International Conference on Machine Learning, pp. 24950–24962. PMLR, July 2023. URL https://proceedings.mlr. press/v202/mit… ↗
  30. Pu, J., Huang, Z., Xi, Y., Chen, G., Chen, W., and Zhang, R. Unraveling the Mystery of Artifacts in Machine Generated Text. In Proceedings of the Thirteenth Language Resources and Evaluation Conference, pp. 6889–6898, Marseille, France, June 2022. European Language Resources Association. URL https:… find
  31. Radford, A., Wu, J., Child, R., Luan, D., Amodei, D., and Sutskever, I. Language Models are Unsupervised Multi-task Learners. OpenAI, pp. 24, 2019. find
  32. Raffel, C., Shazeer, N., Roberts, A., Lee, K., Narang, S., Matena, M., Zhou, Y., Li, W., and Liu, P. J. Exploring the limits of transfer learning with a unified text-to-text transformer, 2023. find
  33. Sadasivan, V. S., Kumar, A., Balasubramanian, S., Wang, W., and Feizi, S. Can AI-Generated Text be Reliably Detected? arxiv:2303.11156[cs], March 2023. doi: 10.48550/arXiv.2303.11156. URL http://arxiv.org/abs/2303.11156. ↗
  34. Sen, P., Namata, G., Bilgic, M., Getoor, L., Galligher, B., and Eliassi-Rad, T. Collective Classification in Network Data. AI Magazine, 29 (3):93–93, September 2008. ISSN 2371-9621. doi: 10.1609/aimag.v29i3.2157. URL https://ojs.aaai.org/aimagazine/index.php/ aimagazine/article/view/2157. ↗
  35. Solaiman, I., Brundage, M., Clark, J., Askell, A., Herbert-Voss, A., Wu, J., Radford, A., Krueger, G., Kim, J. W., Kreps, S., McCain, M., Newhouse, A., Blazakis, J., McGuffie, K., and Wang, J. Release Strategies and the Social Impacts of Language Models. arxiv:1908.09203[cs], November 2019. doi: 10.…
  36. Su, J., Zhuo, T. Y., Wang, D., and Nakov, P. DetectLLM: Leveraging Log Rank Information for Zero-Shot Detection of Machine-Generated Text. arxiv:2306.05540[cs], May 2023. doi: 10.48550/arXiv.2306.05540. URL http://arxiv.org/abs/2306.05540. ↗
  37. Tang, R., Chuang, Y.-N., and Hu, X. The Science of Detecting LLM-Generated Texts. arxiv:2303.07205[cs], March 2023. doi: 10.48550/arXiv.2303.07205. URL http://arxiv.org/abs/2303.07205. — [doi:10.48550/arxiv.2303.07205](https://doi.org/10.48550/arxiv.2303.07205) ↗
  38. Tian, E. Gptzero update v1, January 2023a. https://gptzero.substack.com/p/gptzero-update-v1. ↗
  39. URL Tian, E. New year, new features, new model, January 2023b. URL https://gptzero.substack.com/p/ new-year-new-features-new-model. ↗
  40. Tian, Y., Chen, H., Wang, X., Bai, Z., Zhang, Q., Li, R., Xu, C., and Wang, Y. Multiscale Positive-Unlabeled Detection of AI-Generated Texts. arxiv:2305.18149[cs], June 2023. doi: 10.48550/arXiv.2305.18149. URL http://arxiv.org/abs/2305.18149. — [doi:10.48550/arxiv.2305.18149](https://doi.org/10.48550/arxiv.2305.18149) ↗
  41. Touvron, H., Martin, L., Stone, K., Albert, P., Almahairi, A., Babaei, Y., Bashlykov, N., Batra, S., Bhargava, P., Bhosale, S., Bikel, D., Blecher, L., Ferrer, C. C., Chen, M., Cucurull, G., Esiobu, D., Fernandes, J., Fu, J., Fu, W., Fuller, B., Gao, C., Goswami, V., Goyal, N., Hartshorn, A., Hossei… — Llama 2: Open Foundation and Fine-Tuned Chat Models
  42. Tulchinskii, E., Kuznetsov, K., Kushnareva, L., Cherniavskii, D., Barannikov, S., Piontkovskaya, I., Nikolenko, S., and Burnaev, E. Intrinsic Dimension Estimation for Robust Detection of AI-Generated Texts. arxiv:2306.04723[cs], June 2023. doi: 10.48550/arXiv.2306.04723. URL http://arxiv.org/abs/230… ↗
  43. TurnitIn.com. URL https://www.turnitin.com/. Varshney, L. R., Shirish Keskar, N., and Socher, R. Limits of Detecting Text Generated by Large-Scale Language Models. In 2020 Information Theory and Applications Workshop (ITA), pp. 1–5, February 2020. doi: 10.1109/ITA50056.2020.9245012. ↗
  44. Vasilatos, C., Alam, M., Rahwan, T., Zaki, Y., and Maniatakos, M. HowkGPT: Investigating the Detection of ChatGPT-generated University Student Homework through Context-Aware Perplexity Analysis. arxiv:2305.18226[cs], June 2023. doi: 10.48550/arXiv.2305.18226. URL http://arxiv.org/abs/2305.18226. — [doi:10.48550/arxiv.2305.18226](https://doi.org/10.48550/arxiv.2305.18226) ↗
  45. Verma, V., Fleisig, E., Tomlin, N., and Klein, D. Ghostbuster: Detecting Text Ghostwritten by Large Language Models. arxiv:2305.15047[cs], May 2023. doi: 10.48550/arXiv.2305.15047. URL http://arxiv.org/abs/2305.15047. ↗
  46. Wang, Y., Mansurov, J., Ivanov, P., Su, J., Shelmanov, A., Tsvigun, A., Whitehouse, C., Afzal, O. M., Mahmoud, T., Aji, A. F., and Nakov, P. M4: Multi-generator, Multi-domain, and Multi-lingual Black-Box Machine-Generated Text Detection. arxiv:2305.14902[cs], May 2023. doi: 10.48550/arXiv.2305.14902…
  47. Wolff, M. and Wolff, S. Attacking Neural Text Detectors. arxiv:2002.11768[cs], January 2022. doi: 10.48550/arXiv.2002.11768. URL http://arxiv.org/abs/2002.11768. — [doi:10.48550/arxiv.2002.11768](https://doi.org/10.48550/arxiv.2002.11768) ↗
  48. Yang, X., Cheng, W., Petzold, L., Wang, W. Y., and Chen, H. DNA-GPT: Divergent N-Gram Analysis for Training-Free Detection of GPT-Generated Text. arxiv:2305.17359[cs], May 2023a. doi: 10.48550/arXiv.2305.17359. URL http://arxiv.org/abs/2305.17359. — [doi:10.48550/arxiv.2305.17359](https://doi.org/10.48550/arxiv.2305.17359) ↗
  49. Yang, X., Cheng, W., Wu, Y., Petzold, L., Wang, W. Y., and Chen, H. Dna-gpt: Divergent n-gram analysis for training-free detection of gpt-generated text, 2023b. find
  50. Yu, X., Qi, Y., Chen, K., Chen, G., Yang, X., Zhu, P., Zhang, W., and Yu, N. GPT Paternity Test: GPT Generated Text Detection with GPT Genetic Inheritance. arxiv:2305.12519[cs], May 2023. doi: 10.48550/arXiv.2305.12519. URL http://arxiv.org/abs/2305.12519. — [doi:10.48550/arxiv.2305.12519](https://doi.org/10.48550/arxiv.2305.12519) ↗
  51. Zellers, R., Holtzman, A., Rashkin, H., Bisk, Y., Farhadi, A., Roesner, F., and Choi, Y. Defending Against Neural Fake News. In Advances in Neural Information Processing Systems, volume 32. Curran Associates, Inc., 2019. URL https://proceedings.neurips.cc/paper/2019/hash/ 3e9f0fc9b2f89e043bc6233994… ↗
  52. Zhan, H., He, X., Xu, Q., Wu, Y., and Stenetorp, P. G3Detector: General GPT-Generated Text Detector. arxiv:2305.12680[cs], May 2023. doi: 10.48550/arXiv.2305.12680. URL http://arxiv.org/abs/2305.12680. — [doi:10.48550/arxiv.2305.12680](https://doi.org/10.48550/arxiv.2305.12680) ↗

Updated

↑ back to top

This page in Markdown → https://md.research.elamur.ai/free/en/papers/hans-2024-spotting-llms-binoculars-zero-shot/