Elamur Research
☰
home / papers / 2019 / GLTR: Statistical Detection and Visualization of Generated Text

GLTR: Statistical Detection and Visualization of Generated Text

Written for the Research library (claude-opus-5-5) from the paper text, in our own words: the license "arXiv — non-exclusive license to distribute" does not allow redistributing the text itself. Original — arxiv.org/pdf/1906.04043

Summary

Large language models such as GPT-2 can produce text that ordinary readers struggle to tell apart from human writing, which opens the door to fake reviews, comments and news. The authors present GLTR (Giant Language model Test Room), an open-source, publicly deployed visual tool that helps people judge whether a passage was machine-generated. Its premise is that generators usually draw from the high-probability head of a model's distribution (via temperature, top-k or beam search), while humans more often pick unexpected words. Using a detection model (GPT-2 117M or BERT), GLTR scores every token by its probability, its rank among predictions and the entropy of the predicted distribution. It then colors tokens by rank bucket and shows histograms of these scores. On 50 documents from each of six sources, rank-bucket features separated real from fake text far better than a bag-of-words baseline. In a study with 35 NLP students, the overlay raised detection accuracy from about 54% to 72% with no prior training. The work matters because it offers a cheap, explainable forensic aid that non-experts can use.

Key points

  • Three statistical tests: word probability, absolute rank of the word and predictive entropy. The default overlay colors tokens by rank: top-10 green, top-100 yellow, top-1,000 red, everything else purple. Hovering over a word shows the model's top-5 alternatives.
  • Validation data: three machine sources and three human sources, 50 articles each.
    • Machine: GPT-2 1.5B with temperature 0.7, GPT-2 1.5B with top-40 truncation, and the Washington Post Heliograf system.
    • Human: children's book corpus (CBT), NYT articles, and Nature/Science abstracts.
  • Classifier results (cross-validated AUC): top-k rank buckets reached 0.87 under GPT-2 and 0.85 under BERT. Average probability scored about 0.70–0.71, and the bag-of-words baseline scored 0.63. Human texts used words outside the top 100 predictions 2.41× more often under GPT-2 (1.67× under BERT), even in low-entropy contexts.
  • Human-subjects study (35 students): accuracy rose from 54.2% to 72.3%, an average treatment effect of +18.1% (p < 0.001). Participants rated unaided difficulty at 3.89 out of 5. Qualitatively, generated text tends to repeat entity names rather than use pronouns, and Heliograf output shows highly parallel sentence structure.
  • Resource: the demo is at gltr.io, and the code is on GitHub (HendrikStrobelt/detecting-fake-text) with an extensible PyTorch backend. In its first month the demo had about 30,000 page views.

Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.

Fields: Other

Cited by

260 papers of our library cite it (of the 140673 whose bibliographies we read); 7 of them have a page here, 253 not yet.

Cite

The paper itself, not this page. BibTeX from the library's record of the work. DOI 10.48550/arxiv.1906.04043 · arXiv 1906.04043

@misc{gehrmann2019gltr,
  title = {{GLTR: Statistical Detection and Visualization of Generated Text}},
  author = {Gehrmann, Sebastian and Strobelt, Hendrik and Rush, Alexander M.},
  year = {2019},
  doi = {10.48550/arxiv.1906.04043},
  eprint = {1906.04043},
  archivePrefix = {arXiv},
  url = {https://doi.org/10.48550/arxiv.1906.04043}
}

References

22 references, 4 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

  1. Yuki Arase and Ming Zhou. 2013. Machine translation detection from monolingual web-text. In Proceedings of the 51st Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), volume 1, pages 1597–1607. find
  2. Sameer Badaskar, Sachin Agarwal, and Shilpa Arora. 2008. Identifying real or fake articles: Towards better language modeling. In Proceedings of the Third International Joint Conference on Natural Language Processing: Volume-II. find
  3. Daria Beresneva. 2016. Computer-generated text detection using machine learning: A systematic review. In International Conference on Applications of Natural Language to Information Systems, pages 421–426. Springer. find
  4. Jan Chorowski and Navdeep Jaitly. 2016. Towards better decoding and language model integration in sequence to sequence models. arXiv preprint arXiv:1612.02695. — [arXiv:1612.02695](https://arxiv.org/abs/1612.02695) find
  5. Ido Dagan and Sean P Engelson. 1995. Selective sampling in natural language learning. In Proceedings of the IJCAI Workshop on New Approaches to Learning for Natural Language Processing, pages 41–48. find
  6. Jacob Devlin, Ming-Wei Chang, Kenton Lee, and Kristina Toutanova. 2018. Bert: Pre-training of deep bidirectional transformers for language understanding. arXiv preprint arXiv:1810.04805. find
  7. Angela Fan, Mike Lewis, and Yann Dauphin. 2018. Hierarchical neural story generation. In Proceedings of the 56th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), volume 1, pages 889–898. find
  8. Tommaso Fornaciari and Massimo Poesio. 2014. Identifying fake amazon reviews as learning from crowds. In Proceedings of the 14th Conference of the European Chapter of the Association for Computational Linguistics, pages 279–287. find
  9. EA Grechnikov, GG Gusev, AA Kustarev, and A Raigorodsky. 2009. Detection of artificial texts. RCDL2009 Proceedings. Petrozavodsk, pages 306– 308. find
  10. James V. Grimaldi. 2018. U.s. investigating fake comments on net neutrality. The Wall Street Journal. find
  11. Barbara J Grosz, Scott Weinstein, and Aravind K Joshi. 1995. Centering: A framework for modeling the local coherence of discourse. Computational linguistics, 21(2):203–225. find our card
  12. Jiatao Gu, Kyunghyun Cho, and Victor OK Li. 2017. Trainable greedy decoding for neural machine translation. arXiv preprint arXiv:1702.02429. find
  13. Tatsunori B Hashimoto, Hugh Zhang, and Percy Liang. 2019. Unifying human and statistical evaluation for natural language generation. arXiv preprint arXiv:1904.02792. find
  14. Felix Hill, Antoine Bordes, Sumit Chopra, and Jason Weston. 2015. The goldilocks principle: Reading children’s books with explicit memory representations. arXiv preprint arXiv:1511.02301. find our card
  15. Dirk Hovy. 2016. The enemy in your own camp: How well can we detect statistically-generated fake reviews–an adversarial study. In Proceedings of the 54th Annual Meeting of the Association for Computational Linguistics (Volume 2: Short Papers), volume 2, pages 351–356. find
  16. Thomas Lavergne, Tanguy Urvoy, and François Yvon. 2008. Detecting fake content with relative entropy scoring. PAN, 8:27–31. find
  17. Bryan McCann, James Bradbury, Caiming Xiong, and Richard Socher. 2017. Learned in translation: Contextualized word vectors. In Advances in Neural Information Processing Systems, pages 6294–6305. find our card
  18. Matthew E Peters, Mark Neumann, Mohit Iyyer, Matt Gardner, Christopher Clark, Kenton Lee, and Luke Zettlemoyer. 2018. Deep contextualized word representations. arXiv preprint arXiv:1802.05365. find
  19. Alec Radford, Jeff Wu, Rewon Child, David Luan, Dario Amodei, and Ilya Sutskever. 2019. Language models are unsupervised multitask learners. find
  20. Louis Shao, Stephan Gouws, Denny Britz, Anna Goldie, Brian Strope, and Ray Kurzweil. 2017. Generating high-quality and informative conversation responses with sequence-to-sequence models. arXiv preprint arXiv:1701.03185. find
  21. Kai Shu, Amy Sliva, Suhang Wang, Jiliang Tang, and Huan Liu. 2017. Fake news detection on social media: A data mining perspective. ACM SIGKDD Explorations Newsletter, 19(1):22–36. find
  22. William Yang Wang. 2017. ” liar, liar pants on fire”: A new benchmark dataset for fake news detection. arXiv preprint arXiv:1705.00648. find our card