Elamur Research
☰
home / papers / 2018 / A Stylometric Inquiry into Hyperpartisan and Fake News

A Stylometric Inquiry into Hyperpartisan and Fake News

Written for the Research library (claude-opus-5-5) from the paper text. Original — aclanthology.org/P18-1022.pdf

Summary

Potthast and colleagues ask whether writing style alone can flag problematic political news before it spreads. Fact-checking and social-spread methods only work after publication. To test this, the authors built the BuzzFeed-Webis Fake News Corpus 2016. It contains 1,627 Facebook-linked articles from nine publishers: three mainstream, three hyperpartisan left and three hyperpartisan right. The articles appeared on seven workdays shortly before the 2016 US presidential election, and professional BuzzFeed journalists rated each one for veracity. The authors trained style-based classifiers and compared them with topic-based ones, always testing on publishers absent from training. They also adapted Unmasking to measure how stylistically close different news categories are. Style reliably separates hyperpartisan from mainstream news and satire from real news. It fails, however, to detect fake news directly. A notable finding is that left-wing and right-wing outlets resemble each other in style more than either resembles the mainstream. The practical value is a fast pre-filter that tells human fact-checkers which outlets most likely carry fake news.

Key points

  • Corpus: The BuzzFeed-Webis Fake News Corpus 2016 has 1,627 articles (826 mainstream, 256 left, 545 right). Each was rated on a four-level scale, from "mostly true" to "no factual content." Of the 299 fake articles, 97% are also hyperpartisan.
  • Hyperpartisan detection: Style features separate hyperpartisan from mainstream news with F1 = 0.78.
  • Fake news detection: Style-based fake news detection reaches only F1 = 0.46, which does not beat naive baselines.
  • Satire detection: On Rubin et al.'s S-n-L News DB (360 articles), the style model reaches accuracy 0.82 and F1 = 0.81. Rubin et al.'s topic-inclusive classifier scores F1 = 0.87, which the authors argue generalizes poorly.
  • Left–right similarity: Three modeling approaches, including a novel use of Unmasking, confirm that left and right news share more style with each other than with the mainstream.

Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.

Fields: Natural language processing

Cited by

122 papers of our library cite it (of the 140673 whose bibliographies we read); 4 of them have a page here, 118 not yet.

Cite

The paper itself, not this page. BibTeX from the library's record of the work. DOI 10.18653/v1/p18-1022

@inproceedings{potthast2018stylometric,
  title = {{A Stylometric Inquiry into Hyperpartisan and Fake News}},
  author = {Potthast, Martin and Kiesel, Johannes and Reinartz, Kevin and Bevendorff, Janek and Stein, Benno},
  booktitle = {Proceedings of the 56th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers)},
  year = {2018},
  doi = {10.18653/v1/p18-1022},
  url = {https://doi.org/10.18653/v1/p18-1022}
}

References

35 references, 3 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

  1. Daron Acemoglu, Asuman Ozdaglar, and Ali ParandehGheibi. 2010. Spread of (Mis)Information in Social Networks. Games and Economic Behavior, 70(2):194–227. find
  2. Stefan Heindorf, Martin Potthast, Benno Stein, and Gregor Engels. 2016. Vandalism Detection in Wikidata. In Proceedings of the 25th ACM International Conference on Information and Knowledge Management (CIKM 16), pages 327–336. ACM. find
  3. Sadia Afroz, Michael Brennan, and Rachel Greenstadt. 2012. Detecting Hoaxes, Frauds, and Deception in Writing Style Online. In 2012 IEEE Symposium on Security and Privacy, pages 461–475. find
  4. Benjamin D. Horne and Sibel Adali. 2017. This just in: Fake news packs a lot in title, uses simpler, repetitive content in text body, more similar to satire than real news. CoRR, abs/1703.09398. — [arXiv:1703.09398](https://arxiv.org/abs/1703.09398) find
  5. Shlomo Argamon-Engelson, Moshe Koppel, and Galit Avneri. 1998. Style-based text categorization: What newspaper am i reading. In Proc. of the AAAI Workshop on Text Categorization, pages 1–4. find
  6. Moshe Koppel, Jonathan Schler, and Elisheva Bonchek-Dokow. 2007. Measuring differentiability: Unmasking pseudonymous authors. J. Mach. Learn. Res., 8:1261–1276. find
  7. Sameer Badaskar, Sachin Agarwal, and Shilpa Arora. 2008. Identifying real or fake articles: Towards better language modeling. In Third International Joint Conference on Natural Language Processing, IJCNLP 2008, Hyderabad, India, January 7-12, 2008, pages 817–822. The Association for Computer Linguis… find
  8. Sejeong Kwon, Meeyoung Cha, Kyomin Jung, Wei Chen, and Yajun Wang. 2013. Prominent Features of Rumor Propagation in Online Social Media. In Data Mining (ICDM), 2013 IEEE 13th International Conference on, pages 1103–1108. IEEE. find
  9. Peter Bourgonje, Julián Moreno Schneider, and Georg Rehm. 2017. From clickbait to fake news detection: An approach based on detecting the stance of headlines to articles. In Proceedings of the 2017 Workshop: Natural Language Processing meets Journalism, NLPmJ@EMNLP, Copenhagen, Denmark, September 7,… find
  10. Ceren Budak, Divyakant Agrawal, and Amr El Abbadi. 2011. Limiting the spread of misinformation in social networks. In Proceedings of the 20th International Conference on World Wide Web, WWW ’11, pages 665–674, New York, NY, USA. ACM. find
  11. Yimin Chen, Niall J. Conroy, and Victoria L. Rubin. 2015. News in an Online World: The Need for an "Automatic Crap Detector". In Proceedings of the 78th ASIS&T Annual Meeting: Information Science with Impact: Research in and for the Community, ASIST ’15, pages 81:1–81:4, Silver Springs, MD, USA. Ame… find
  12. Delia Mocanu, Luca Rossi, Qian Zhang, Marton Karsai, and Walter Quattrociocchi. 2015. Collective Attention in the Age of (Mis)Information. Comput. Hum. Behav., 51(PB):1198–1204. find
  13. Leon Derczynski, Kalina Bontcheva, Maria Liakata, Rob Procter, Geraldine Wong Sak Hoi, and Arkaitz Zubiaga. 2017. Semeval-2017 task 8: Rumoureval: Determining rumour veracity and support for rumours. In Proceedings of the 11th International Workshop on Semantic Evaluation, SemEval@ACL 2017, Vancouve… find
  14. Alexandru L. Ginsca, Adrian Popescu, and Mihai Lupu. 2015. Credibility in Information Retrieval. Found. Trends Inf. Retr., 9(5):355–475. find
  15. Jing Ma, Wei Gao, and Kam-Fai Wong. 2017. Detect rumors in microblog posts using propagation structure via kernel learning. In Proceedings of the 55th Annual Meeting of the Association for Computational Linguistics, ACL 2017, Vancouver, Canada, July 30 -August 4, Volume 1: Long Papers, pages 708–717… find
  16. Amr Magdy and Nayer Wanas. 2010. Web-based Statistical Fact Checking of Textual Documents. In Proceedings of the 2Nd International Workshop on Search and Mining User-generated Contents, SMUC ’10, pages 103–110, New York, NY, USA. ACM. find
  17. Giovanni Luca Ciampaglia, Prashant Shiralkar, Luis M Rocha, Johan Bollen, Filippo Menczer, and Alessandro Flammini. 2015. Computational Fact Checking from Knowledge Networks. PloS one, 10(6):e0128193. find
  18. Oren Etzioni, Michele Banko, Stephen Soderland, and Daniel S. Weld. 2008. Open Information Extraction from the Web. Commun. ACM, 51(12):68–74. find our card
  19. Yunfei Long, Qin Lu, Rong Xiang, Minglei Li, and Chu-Ren Huang. 2017. Fake news detection through multi-perspective speaker profiles. In Proceedings of the Eighth International Joint Conference on Natural Language Processing, IJCNLP 2017, Taipei, Taiwan, November 27 - December 1, 2017, Volume 2: Sho… find
  20. Nam P. Nguyen, Guanhua Yan, My T. Thai, and Stephan Eidenbenz. 2012. Containment of Misinformation Spread in Online Social Networks. In Proceedings of the 4th Annual ACM Web Science Conference, WebSci ’12, pages 213–222, New York, NY, USA. ACM. find
  21. Verónica Pérez-Rosas, Bennett Kleinberg, Alexandra Lefevre, and Rada Mihalcea. 2017. Automatic detection of fake news. CoRR, abs/1708.07104. find our card
  22. Hannah Rashkin, Eunsol Choi, Jin Yea Jang, Svitlana Volkova, and Yejin Choi. 2017. Truth of varying shades: Analyzing language in fake news and political fact-checking. In Proceedings of the 2017 Conference on Empirical Methods in Natural Language Processing, EMNLP 2017, Copenhagen, Denmark, Septemb… find
  23. Victoria Rubin, Niall Conroy, and Yimin Chen. 2015. Towards News Verification: Deception Detection Methods for News Discourse. In Proceedings of the Hawaii International Conference on System Sciences (HICSS48) Symposium on Rapid Screening Technologies, Deception Detection and Credibility Assessment… find
  24. Victoria Rubin, Niall Conroy, Yimin Chen, and Sarah Cornwell. 2016. Fake News or Truth? Using Satirical Cues to Detect Potentially Misleading News. In Proceedings of the Second Workshop on Computational Approaches to Deception Detection, pages 7–17, San Diego, California. Association for Computation… find
  25. Baoxu Shi and Tim Weninger. 2016. Fact Checking in Heterogeneous Information Networks. In Proceedings of the 25th International Conference Companion on World Wide Web, WWW ’16 Companion, pages 101–102, Republic and Canton of Geneva, Switzerland. International World Wide Web Conferences Steering Comm… find
  26. An empirical study on uncertainty identification in social media context. In Proceedings of the 51st Annual Meeting of the Association for Computational Linguistics (Volume 2: Short Papers), pages 58–62, Sofia, Bulgaria. Association for Computational Linguistics. find
  27. You Wu, Pankaj K. Agarwal, Chengkai Li, Jun Yang, and Cong Yu. 2014. Toward Computational Fact-checking. Proc. VLDB Endow., 7(7):589–600. find
  28. Fan Yang, Arjun Mukherjee, and Eduard Constantin Dragut. 2017. Satirical news detection and analysis using attention mechanism and linguistic features. In Proceedings of the 2017 Conference on Empirical Methods in Natural Language Processing, EMNLP 2017, Copenhagen, Denmark, September 9-11, 2017, pa… find
  29. Craig Silverman, Lauren Strapagiel, Hamza Shaban, Ellie Hall, and Jeremy Singer-Vine. 2016. Hyperpartisan Facebook Pages are Publishing False and Misleading Information at an Alarming Rate. https://www.buzzfeed.com/craigsilverman/partisan-fb-pages-analysis. BuzzFeed. ↗
  30. Philip J. Stone, Dexter C. Dunphy, and Marshall S. Smith. 1966. The General Inquirer: A Computer Approach to Content Analysis. MIT press. find
  31. Marcella Tambuscio, Giancarlo Ruffo, Alessandro Flammini, and Filippo Menczer. 2015. Fact-checking Effect on Viral Hoaxes: A Model of Misinformation Spread in Social Networks. In Proceedings of the 24th International Conference on World Wide Web, WWW ’15 Companion, pages 977–982, New York, NY, USA.… find
  32. Udo Undeutsch. 1967. Beurteilung der glaubhaftigkeit von aussagen. Handbuch der Psychologie, 11:26–181. find
  33. Svitlana Volkova, Kyle Shaffer, Jin Yea Jang, and Nathan Oken Hodas. 2017. Separating facts from fiction: Linguistic models to classify suspicious and trusted news posts on twitter. In Proceedings of the 55th Annual Meeting of the Association for Computational Linguistics, ACL 2017, Vancouver, Canad… find
  34. William Yang Wang. 2017. "liar, liar pants on fire": A new benchmark dataset for fake news detection. In Proceedings of the 55th Annual Meeting of the Association for Computational Linguistics, ACL 2017, Vancouver, Canada, July 30 - August 4, Volume 2: Short Papers, pages 422–426. find our card
  35. Zhongyu Wei, Junwen Chen, Wei Gao, Binyang Li, Lanjun Zhou, Yulan He, and Kam-Fai Wong. 2013. find