Elamur Research⌕
☰
home / papers / 2020 / Automatic Detection of Machine Generated Text: A Critical Survey

Automatic Detection of Machine Generated Text: A Critical Survey

Summary

Modern text generative models (TGMs) can write fluent, coherent English. Adversaries can misuse them to produce fake news, fake product reviews, and spam that people struggle to tell apart from genuine writing. The authors argue that this calls for automatic detectors, which treat the problem as classifying a text as machine-written or human-written. They present what they describe as the first survey of this fast-growing English-language literature.

The survey covers:

  • background on how TGMs are trained and how text is decoded from them (greedy, beam, top-k, nucleus sampling);
  • the social impact of TGMs;
  • properties of large models such as GPT-2, GROVER, CTRL and GPT-3.

Existing detectors are sorted into four families: classifiers trained from scratch, zero-shot classifiers, fine-tuned neural language models, and human–machine collaboration. The authors also run an error analysis of the state-of-the-art RoBERTa detector on product reviews, inspecting its false positives from both linguistic and computational angles. Finally, they outline research directions, giving newcomers a structured entry point into the field.

Key points

  • The paper names five properties of an ideal detector: accurate, data-efficient, generalizable across TGM settings, interpretable, and robust to adversarial input.
  • The TGMs compared include:
    • GPT-2 (WebText, 40GB / 1.5B params)
    • GROVER (RealNews, 120GB / 1.5B)
    • CTRL (140GB / 1.6B)
    • GPT-3 (Common-Crawl, 570GB / 175B)
  • Cited studies show that humans identify TGM-generated news, reviews, and comments only at roughly chance level.
  • Some human reviews were wrongly flagged as machine-written by the RoBERTa detector. Seven of these contained typos or grammatical errors, and three showed incoherent discourse.
  • Simple attacks (Wolff, 2020) break the RoBERTa detector, cutting its recall on machine text from 97.44% to:
    • 0.26% with homoglyph substitution;
    • 22.68% with misspellings.
  • Proposed future directions include using auxiliary signals, checking veracity against external sources, cross-domain generalization, interpretability, and adversarial robustness.

Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.

Fields: Natural language processing

Cited by

93 papers of our library cite it (of the 140884 whose bibliographies we read); 12 of them have a page here, 81 not yet.

Cite

The paper itself, not this page. BibTeX from the library's record of the work. DOI 10.18653/v1/2020.coling-main.208

@inproceedings{jawahar2020automatic,
  title = {{Automatic Detection of Machine Generated Text: A Critical Survey}},
  author = {Jawahar, Ganesh and Abdul-Mageed, Muhammad and V.S},
  booktitle = {Proceedings of the 28th International Conference on Computational Linguistics},
  year = {2020},
  doi = {10.18653/v1/2020.coling-main.208},
  url = {https://doi.org/10.18653/v1/2020.coling-main.208}
}

References

50 references, 16 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

  1. David Ifeoluwa Adelani, Haotian Mai, Fuming Fang, Huy H. Nguyen, Junichi Yamagishi, and Isao Echizen. 2020. Generating Sentiment-Preserving Fake Online Reviews Using Neural Language Models and Their Human-and Machine-Based Detection. In Proceedings of the 34th International Conference on Advanced In… find
  2. Amazon. 2019. Amazon Customer Reviews Dataset. https://s3.amazonaws.com/ amazon-reviews-pds/readme.html. Accessed on 09/05/2020. ↗
  3. Anton Bakhtin, Sam Gross, Myle Ott, Yuntian Deng, Marc’Aurelio Ranzato, and Arthur Szlam. 2019. Real or Fake? Learning to Discriminate Machine from Human Generated Text. CoRR, abs/1906.03351. — [arXiv:1906.03351](https://arxiv.org/abs/1906.03351) find
  4. Anton Bakhtin, Yuntian Deng, Sam Gross, Myle Ott, Marc’Aurelio Ranzato, and Arthur Szlam. 2020. Energy-Based Models for Text. CoRR, abs/2004.10188. — [arXiv:2004.10188](https://arxiv.org/abs/2004.10188) find
  5. Piotr Bojanowski, Edouard Grave, Armand Joulin, and Tomas Mikolov. 2017. Enriching Word Vectors with Subword Information. Transactions of the Association for Computational Linguistics, pages 135–146. find
  6. Tom B. Brown, Benjamin Mann, Nick Ryder, Melanie Subbiah, Jared Kaplan, Prafulla Dhariwal, Arvind Neelakantan, Pranav Shyam, Girish Sastry, Amanda Askell, Sandhini Agarwal, Ariel Herbert-Voss, Gretchen Krueger, Tom Henighan, Rewon Child, Aditya Ramesh, Daniel M. Ziegler, Jeffrey Wu, Clemens Winter,… find
  7. Kevin Clark, Minh-Thang Luong, Quoc V. Le, and Christopher D. Manning. 2020. Electra: Pre-training text encoders as discriminators rather than generators. In International Conference on Learning Representations. find
  8. Alexis Conneau and Guillaume Lample. 2019. Cross-lingual Language Model Pretraining. In Advances in Neural Information Processing Systems 32, pages 7059–7069. find
  9. Kate Crawford. 2017. The trouble with bias. find
  10. Sumanth Dathathri, Andrea Madotto, Janice Lan, Jane Hung, Eric Frank, Piero Molino, Jason Yosinski, and Rosanne Liu. 2020. Plug and Play Language Models: A Simple Approach to Controlled Text Generation. In International Conference on Learning Representations. find our card
  11. Jacob Devlin, Ming-Wei Chang, Kenton Lee, and Kristina Toutanova. 2019. BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding. In Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies,… find
  12. Liam Dugan, Daphne Ippolito, Arun Kirubarajan, and Chris Callison-Burch. 2020. RoFT: A Tool for Evaluating Human Detection of Machine-Generated Text. CoRR, abs/2010.03070. find
  13. Jeffrey L. Elman. 1990. Finding structure in time. Cognitive Science, 14(2):179 – 211. find
  14. Tiziano Fagni, Fabrizio Falchi, Margherita Gambini, Antonio Martella, and Maurizio Tesconi. 2020. TweepFake: about Detecting Deepfake Tweets. CoRR, abs/2008.00036. find
  15. Angela Fan, Mike Lewis, and Yann Dauphin. 2018. Hierarchical Neural Story Generation. In Proceedings of the 56th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 889–898. find
  16. Sebastian Gehrmann, Hendrik Strobelt, and Alexander Rush. 2019. GLTR: Statistical Detection and Visualization of Generated Text. In Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics: System Demonstrations, pages 111–116. find our card
  17. Barbara J. Grosz, Aravind K. Joshi, and Scott Weinstein. 1995. Centering: A Framework for Modeling the Local Coherence of Discourse. Computational Linguistics, 21(2):203–225. find our card
  18. Sepp Hochreiter and Jürgen Schmidhuber. 1997. Long short-term memory. Neural Comput., 9(8):1735–1780. find
  19. Ari Holtzman, Jan Buys, Li Du, Maxwell Forbes, and Yejin Choi. 2020. The Curious Case of Neural Text Degeneration. In International Conference on Learning Representations. find
  20. Dirk Hovy. 2016. The Enemy in Your Own Camp: How Well Can We Detect Statistically-Generated Fake Reviews – An Adversarial Study. In Proceedings of the 54th Annual Meeting of the Association for Computational Linguistics (Volume 2: Short Papers), pages 351–356. find
  21. Daphne Ippolito, Daniel Duckworth, Chris Callison-Burch, and Douglas Eck. 2020. Automatic Detection of Generated Text is Easiest when Humans are Fooled. In Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics, pages 1808–1822. find our card
  22. Nitish Shirish Keskar, Bryan McCann, Lav R. Varshney, Caiming Xiong, and Richard Socher. 2019. CTRL: A Conditional Transformer Language Model for Controllable Generation. CoRR, abs/1909.05858. find our card
  23. Guanxiong Liu, Tzu-Ming Harry Hsu, Matthew B. A. McDermott, Willie Boag, Wei-Hung Weng, Peter Szolovits, and Marzyeh Ghassemi. 2019a. Clinically Accurate Chest X-Ray Report Generation. In Proceedings of the Machine Learning for Healthcare Conference, MLHC, volume 106, pages 249–269. find
  24. Yinhan Liu, Myle Ott, Naman Goyal, Jingfei Du, Mandar Joshi, Danqi Chen, Omer Levy, Mike Lewis, Luke Zettlemoyer, and Veselin Stoyanov. 2019b. RoBERTa: A Robustly Optimized BERT Pretraining Approach. CoRR, abs/1907.11692. find
  25. Luca Massarelli, Fabio Petroni, Aleksandra Piktus, Myle Ott, Tim Rocktäschel, Vassilis Plachouras, Fabrizio Silvestri, and Sebastian Riedel. 2020. How Decoding Strategies Affect the Verifiability of Generated Text. CoRR, abs/1911.03587. find
  26. Moin Nadeem, Anna Bethke, and Siva Reddy. 2020. StereoSet: Measuring stereotypical bias in pretrained language models. CoRR, abs/2004.09456. find our card
  27. Nathan Ng, Kyra Yee, Alexei Baevski, Myle Ott, Michael Auli, and Sergey Edunov. 2019. Facebook FAIR’s WMT19 news translation task submission. In Proceedings of the Fourth Conference on Machine Translation (Volume 2: Shared Task Papers, Day 1), pages 314–319. find our card
  28. Alec Radford, Karthik Narasimhan, Tim Salimans, and Ilya Sutskever. 2018. Improving Language Understanding by Generative Pre-Training. https://s3-us-west-2.amazonaws.com/openai-assets/ research-covers/language-unsupervised/language_understanding_paper.pdf. ↗
  29. Alec Radford, Jeffrey Wu, Rewon Child, David Luan, Dario Amodei, and Ilya Sutskever. 2019. Language Models are Unsupervised Multitask Learners. https://d4mucfpksywv.cloudfront.net/better-language-models/language_models_are_unsupervised_multitask_ learners.pdf. ↗
  30. Colin Raffel, Noam Shazeer, Adam Roberts, Katherine Lee, Sharan Narang, Michael Matena, Yanqi Zhou, Wei Li, and Peter J. Liu. 2020. Exploring the limits of transfer learning with a unified text-to-text transformer. Journal of Machine Learning Research, 21(140):1–67. find
  31. Tal Schuster, Roei Schuster, Darsh J. Shah, and Regina Barzilay. 2020. The Limitations of Stylometry for Detecting Machine-Generated Fake News. Computational Linguistics. find
  32. Irene Solaiman, Miles Brundage, Jack Clark, Amanda Askell, Ariel Herbert-Voss, Jeff Wu, Alec Radford, and Jasmine Wang. 2019. Release Strategies and the Social Impacts of Language Models. CoRR, abs/1908.09203. find our card
  33. Kaitao Song, Xu Tan, Tao Qin, Jianfeng Lu, and Tie-Yan Liu. 2019. MASS: Masked Sequence to Sequence Pre-training for Language Generation. In International Conference on Machine Learning, pages 5926–5936. find our card
  34. Emma Strubell, Ananya Ganesh, and Andrew McCallum. 2019. Energy and Policy Considerations for Deep Learning in NLP. In Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics, pages 3645–3650. find our card
  35. Tony Sun, Andrew Gaut, Shirlyn Tang, Yuxin Huang, Mai ElSherief, Jieyu Zhao, Diba Mirza, Elizabeth Belding, Kai-Wei Chang, and William Yang Wang. 2019. Mitigating Gender Bias in Natural Language Processing: Literature Review. In Proceedings of the 57th Annual Meeting of the Association for Computati… find our card
  36. TabNine. 2020. TabNine. Autocompletion with deep learning. https://tabnine.com/blog/deep. Accessed on 04/22/2020. ↗
  37. Reuben Tan, Bryan A. Plummer, and Kate Saenko. 2020. Detecting Cross-Modal Inconsistency to Defend Against Neural Fake News. CoRR, abs/2009.07698. find
  38. Yi Tay, Dara Bahri, Che Zheng, Clifford Brunk, Donald Metzler, and Andrew Tomkins. 2020. Reverse Engineering Configurations of Neural Text Generation Models. In Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics, pages 275–279. find
  39. James Thorne and Andreas Vlachos. 2018. Automated Fact Checking: Task Formulations, Methods and Future Directions. In Proceedings of the 27th International Conference on Computational Linguistics, pages 3346– 3359. find our card
  40. Adaku Uchendu, Thai Le, Kai Shu, and Dongwon Lee. 2020. Authorship Attribution for Neural Text Generation. In Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing. find our card
  41. Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N. Gomez, Lukasz Kaiser, and Illia Polosukhin. 2017. Attention is All you Need. In Advances in Neural Information Processing Systems, pages 6000–6010. find
  42. Soroush Vosoughi, Deb Roy, and Sinan Aral. 2018. The spread of true and false news online. Science, 359(6380):1146–1151. find
  43. Eric Wallace, Shi Feng, Nikhil Kandpal, Matt Gardner, and Sameer Singh. 2019. Universal Adversarial Triggers for Attacking and Analyzing NLP. In Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Proce… find
  44. Max Weiss. 2019. Deepfake Bot Submissions to Federal Public Comment Websites Cannot Be Distinguished from Human Submissions. In Technology Science. find
  45. Sean Welleck, Ilia Kulikov, Stephen Roller, Emily Dinan, Kyunghyun Cho, and Jason Weston. 2020. Neural Text Generation With Unlikelihood Training. In International Conference on Learning Representations. find our card
  46. Max Wolff. 2020. Attacking Neural Text Detectors. CoRR, abs/2002.11768. — [arXiv:2002.11768](https://arxiv.org/abs/2002.11768) find
  47. Zhilin Yang, Zihang Dai, Yiming Yang, Jaime Carbonell, Russ R Salakhutdinov, and Quoc V Le. 2019. XL-Net: Generalized Autoregressive Pretraining for Language Understanding. In Advances in Neural Information Processing Systems 32, pages 5753–5763. find
  48. Rowan Zellers, Ari Holtzman, Hannah Rashkin, Yonatan Bisk, Ali Farhadi, Franziska Roesner, and Yejin Choi. 2019. Defending Against Neural Fake News. In Advances in Neural Information Processing Systems, pages 9054–9065. find our card
  49. Yizhe Zhang, Siqi Sun, Michel Galley, Yen-Chun Chen, Chris Brockett, Xiang Gao, Jianfeng Gao, Jingjing Liu, and Bill Dolan. 2020. DIALOGPT : Large-Scale Generative Pre-training for Conversational Response Generation. In Proceedings of the 58th Annual Meeting of the Association for Computational Ling… find our card
  50. Wanjun Zhong, Duyu Tang, Zenan Xu, Ruize Wang, Nan Duan, Ming Zhou, Jiahai Wang, and Jian Yin. 2020. Neural Deepfake Detection with Factual Structure of Text. CoRR, abs/2010.07475. find

↑ back to top

This page in Markdown → https://md.research.elamur.ai/free/en/papers/jawahar-2020-automatic-detection-machine-generated-text/