Elamur Research
☰
home / papers / 2019 / Defending Against Neural Fake News

Defending Against Neural Fake News

Written for the Research library (claude-opus-5-5) from the paper text, in our own words: the license "arXiv — non-exclusive licence to distribute" does not allow redistributing the text itself. Original — arxiv.org/pdf/1905.12616

Summary

Large text generators could let adversaries produce large amounts of fabricated news written to look like it comes from real outlets. The authors take the threat-modeling approach used in computer security. They set up disinformation as a contest between two players. One is an adversary who generates fake articles. The other is a verifier that sees plenty of genuine news but only a small sample of one adversary's fakes. To study this threat, they build Grover, a Transformer language model that writes a complete article: domain, date, authors, headline and body. Each field has its own start and end tokens, so any subset of fields can be given as context and any missing field can be generated. Human raters judged Grover's rewritten propaganda to be more trustworthy than disinformation written by people. For detection, Grover used as a classifier reaches 92% accuracy. The best other detectors, deep pretrained models such as BERT and GPT-2, reach 73%. The authors trace this gap to artifacts left by exposure bias and by truncated sampling. They argue that publicly releasing strong generators is essential for defense.

Key points

  • Detection setup: the verifier is trained on 5,000 Grover generations plus unlimited real news. Grover as a discriminator scores 92%, compared with 73% for the strongest non-Grover baseline.
  • The model comes in three sizes (Grover-Base, Grover-Large, Grover-Mega) and is trained on RealNews, a news corpus collected from Common Crawl. Collecting the data cost about $10k in AWS credits. Training Grover-Mega took about two weeks on TPU v3 cores and cost about $25k.
  • With nucleus (top-p) sampling, fakes are hardest to detect at a threshold of roughly p=0.94–0.98. BERT's worst-case threshold is lower than Grover's. The authors read this as a sign that left-to-right models are better at spotting where the probability tail was truncated.
  • Fields are generated in a fixed order (domain → date → authors → headline → body). Grover can also regenerate an earlier field, for example rewriting a headline after the body exists.
  • The authors released the models to researchers. They recommend that platforms screen articles with an ensemble of generator-based detectors and keep humans in the loop.

Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.

Fields: Other

Cited by

381 papers of our library cite it (of the 140673 whose bibliographies we read); 24 of them have a page here, 357 not yet.

Cite

The paper itself, not this page. BibTeX from the library's record of the work. DOI 10.48550/arxiv.1905.12616 · arXiv 1905.12616

@misc{zellers2019defending,
  title = {{Defending Against Neural Fake News}},
  author = {Zellers, Rowan and Holtzman, Ari and Rashkin, Hannah and Bisk, Yonatan and Farhadi, Ali and Roesner, Franziska and Choi, Yejin},
  year = {2019},
  doi = {10.48550/arxiv.1905.12616},
  eprint = {1905.12616},
  archivePrefix = {arXiv},
  url = {https://doi.org/10.48550/arxiv.1905.12616}
}

References

48 references, 8 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

  1. Yoshua Bengio, Réjean Ducharme, Pascal Vincent, and Christian Jauvin. A neural probabilistic language model. Journal of machine learning research, 3(Feb):1137–1155, 2003. find
  2. Samantha Bradshaw and Philip Howard. Troops, trolls and troublemakers: A global inventory of organized social media manipulation. Technical report, Oxford Internet Institute, 2017. find
  3. Massimo Caccia, Lucas Caccia, William Fedus, Hugo Larochelle, Joelle Pineau, and Laurent Charlin. Language gans falling short. arXiv preprint arXiv:1811.02549, 2018. — [arXiv:1811.02549](https://arxiv.org/abs/1811.02549) find
  4. Danqi Chen, Adam Fisch, Jason Weston, and Antoine Bordes. Reading wikipedia to answer open-domain questions. In Proceedings of the 55th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 1870–1879, 2017. find our card
  5. Jacob Devlin, Ming-Wei Chang, Kenton Lee, and Kristina Toutanova. Bert: Pre-training of deep bidirectional transformers for language understanding. arXiv preprint arXiv:1810.04805, 2018. find
  6. Rachel Dicker. Avoid These Fake News Sites at All Costs. https: //www.usnews.com/news/national-news/articles/2016-11-14/avoid-these-fake-news-sites-at-all-costs, 2016. [Online; accessed 22-May-2019]. find
  7. Elizabeth Dwoskin and Tony Romm. Facebook says it has uncovered a coordinated disinformation operation ahead of the 2018 midterm elections. find
  8. The Washington Post, 2018. find
  9. Angela Fan, Mike Lewis, and Yann Dauphin. Hierarchical neural story generation. In Proceedings of the 56th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 889–898, 2018. find
  10. Robert Faris, Hal Roberts, Bruce Etling, Nikki Bourassa, Ethan Zuckerman, and Yochai Benkler. Partisanship, propaganda, and disinformation: Online media and the 2016 us presidential election. Berkman Klein Center Research Publication 2017-6., 2017. find
  11. Marjan Ghazvininejad, Omer Levy, Yinhan Liu, and Luke Zettlemoyer. Constant-time machine translation with conditional masked language models. arXiv preprint arXiv:1904.09324, 2019. — [arXiv:1904.09324](https://arxiv.org/abs/1904.09324) find
  12. Jiatao Gu, Qi Liu, and Kyunghyun Cho. Insertion-based decoding with automatically inferred generation order. arXiv preprint arXiv:1902.01370, 2019. find
  13. Xiaochuang Han and Jacob Eisenstein. Unsupervised domain adaptation of contextualized embeddings: A case study in early modern english. arXiv preprint arXiv:1904.02817, 2019. — [arXiv:1904.02817](https://arxiv.org/abs/1904.02817) find
  14. Tatsunori B Hashimoto, Hugh Zhang, and Percy Liang. Unifying human and statistical evaluation for natural language generation. arXiv preprint arXiv:1904.02792, 2019. find
  15. Brent Hecht, Lauren Wilcox, Jeffrey P. Bigham, Johannes Schöning, Ehsan Hoque, Jason Ernnst, Yonatan Bisk, Luigi De Russis, Lana Yarosh, Bushra Anjum, Danish Contractor, and Cathy Wu. It’s time to do something: Mitigating the negative impacts of computing through a change to the peer review process.… find
  16. Ari Holtzman, Jan Buys, Maxwell Forbes, and Yejin Choi. The curious case of neural text degeneration. arXiv preprint arXiv:1904.09751, 2019. find
  17. Hossein Hosseini, Baicen Xiao, Andrew Clark, and Radha Poovendran. Attacking automatic video analysis algorithms: A case study of google cloud video intelligence api. In Proceedings of the 2017 on Multimedia Privacy and Security, pages 21–32. find
  18. Zhiting Hu, Zichao Yang, Xiaodan Liang, Ruslan Salakhutdinov, and Eric P Xing. Toward controlled generation of text. In Proceedings of the 34th International Conference on Machine Learning-Volume 70, pages 1587–1596. JMLR. org, 2017. find our card
  19. Armand Joulin, Edouard Grave, Piotr Bojanowski, and Tomas Mikolov. Bag of tricks for efficient text classification. In Proceedings of the 15th Conference of the European Chapter of the Association for Computational Linguistics: Volume 2, Short Papers, volume 2, pages 427–431, 2017. find
  20. Rafal Józefowicz, Oriol Vinyals, Mike Schuster, Noam Shazeer, and Yonghui Wu. Exploring the limits of language modeling. CoRR, abs/1602.02410, 2016. find
  21. Diederik P. Kingma and Jimmy Ba. abs/1412.6980, 2014. find
  22. Adam: A method for stochastic optimization. Clare Melford and Craig Fagan. Cutting the funding of disinformation: The ad-tech solution. Technical report, The Global Disinformation Index, 2019. find
  23. Adam Mosseri. News feed fyi: Helping ensure news on facebook is from trusted sources. Facebook Newsroom, 19, 2018. find
  24. Myle Ott, Yejin Choi, Claire Cardie, and Jeffrey T Hancock. Finding deceptive opinion spam by any stretch of the imagination. In Proceedings of the 49th annual meeting of the association for computational linguistics: Human language technologies-volume 1, pages 309–319. Association for Computational… find
  25. Verónica Pérez-Rosas, Bennett Kleinberg, Alexandra Lefevre, and Rada Mihalcea. Automatic detection of fake news. In Proceedings of the 27th International Conference on Computational Linguistics, pages 3391–3401, 2018. find our card
  26. Matthew Peters, Mark Neumann, Mohit Iyyer, Matt Gardner, Christopher Clark, Kenton Lee, and Luke Zettlemoyer. Deep contextualized word representations. In Proceedings of the 2018 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, V… find
  27. Alec Radford, Karthik Narasimhan, Tim Salimans, and Ilya Sutskever. Improving language understanding by generative pre-training. Technical report, OpenAI, 2018. URL https: //blog.openai.com/language-unsupervised/. find
  28. Alec Radford, Jeffrey Wu, Rewon Child, David Luan, Dario Amodei, and Ilya Sutskever. Language models are unsupervised multitask learners. Technical report, OpenAI, 2019. find
  29. Marc’Aurelio Ranzato, Sumit Chopra, Michael Auli, and Wojciech Zaremba. Sequence level training with recurrent neural networks. In ICLR. ICLR, 2016. find our card
  30. Hannah Rashkin, Eunsol Choi, Jin Yea Jang, Svitlana Volkova, and Yejin Choi. Truth of varying shades: Analyzing language in fake news and political fact-checking. In Proceedings of the 2017 Conference on Empirical Methods in Natural Language Processing, pages 2931–2937, 2017. find
  31. Chengcheng Shao, Giovanni Luca Ciampaglia, Alessandro Flammini, and Filippo Menczer. Hoaxy: A platform for tracking online misinformation. In Proceedings of the 25th international conference companion on world wide web, pages 745–750. International World Wide Web Conferences Steering Committee, 2016… find
  32. Noam Shazeer and Mitchell Stern. Adafactor: Adaptive learning rates with sublinear memory cost. In International Conference on Machine Learning, pages 4603–4611, 2018. find
  33. Irene Solaiman, Miles Brundage, Jack Clark, Amanda Askell, Ariel Herbert-Voss, Jeff Wu, Alec Radford, and Jasmine Wang. Release strategies and the social impacts of language models. arXiv preprint arXiv:1908.09203, 2019. find our card
  34. Mitchell Stern, William Chan, Jamie Kiros, and Jakob Uszkoreit. Insertion transformer: Flexible sequence generation via insertion operations. arXiv preprint arXiv:1902.03249, 2019. — [arXiv:1902.03249](https://arxiv.org/abs/1902.03249) find
  35. Hendrik Strobelt and Sebastian Gehrmann. Catching a unicorn with gltr: A tool to detect automatically generated text. Technical report, Harvard, 2019. find
  36. Briony Swire, Ullrich KH Ecker, and Stephan Lewandowsky. The role of familiarity in correcting inaccurate information. Journal of experimental psychology: learning, memory, and cognition, 43 (12):1948, 2017. find
  37. CoRR, James Thorne, Andreas Vlachos, Christos Christodoulopoulos, and Arpit Mittal. Fever: a large-scale dataset for fact extraction and verification. In Proceedings of the 2018 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Vo… find our card
  38. Chris J Vargo, Lei Guo, and Michelle A Amazeen. The agenda-setting power of fake news: A big data analysis of the online media landscape from 2014 to 2016. New Media & Society, 20(5): 2028–2049, 2018. find
  39. Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N Gomez, Łukasz Kaiser, and Illia Polosukhin. Attention is all you need. In Proceedings of the 31st International Conference on Neural Information Processing Systems, pages 6000–6010. Curran Associates Inc., 2017. find
  40. Alex Wang, Amapreet Singh, Julian Michael, Felix Hill, Omer Levy, and Samuel R Bowman. Glue: A multi-task benchmark and analysis platform for natural language understanding. arXiv preprint arXiv:1804.07461, 2018. find our card
  41. William Yang Wang. “liar, liar pants on fire”: A new benchmark dataset for fake news detection. In Proceedings of the 55th Annual Meeting of the Association for Computational Linguistics (Volume 2: Short Papers), pages 422–426, 2017. find our card
  42. Claire Wardle. Fake news. it’s complicated. First Draft News, 16, 2017. find
  43. Claire Wardle and Hossein Derakhshan. Information disorder: Toward an interdisciplinary framework for research and policy making. Council of Europe report, DGI (2017), 9, 2017. find
  44. Rowan Zellers. Why we released grover. Technical report, 2019. URL https://thegradient. pub/why-we-released-grover/. find
  45. Rowan Zellers, Yonatan Bisk, Roy Schwartz, and Yejin Choi. Swag: A large-scale adversarial dataset for grounded commonsense inference. In Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing (EMNLP), 2018. find
  46. Rowan Zellers, Yonatan Bisk, Ali Farhadi, and Yejin Choi. From recognition to cognition: Visual commonsense reasoning. In The IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2019a. find
  47. Rowan Zellers, Ari Holtzman, Yonatan Bisk, Ali Farhadi, and Yejin Choi. Hellaswag: Can a machine really finish your sentence? In Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics, 2019b. find
  48. Rowan Zellers, Ari Holtzman, Hannah Rashkin, Yonatan Bisk, Ali Farhadi, Franziska Roesner, and Yejin Choi. Counteracting neural disinformation with grover. Technical report, 2019c. URL https://medium.com/ai2-blog/counteracting-neural-disinformation-with-grover-6cf6690d463b. ↗