Elamur Research⌕
☰
home / papers / 2018 / DeClarE: Debunking Fake News and False Claims using Evidence-Aware Deep Learning

DeClarE: Debunking Fake News and False Claims using Evidence-Aware Deep Learning

Summary

DeClarE tackles automatic credibility assessment of free-text claims, such as rumors, political statements, news headlines or tweets. Earlier neural fact-checkers looked only at the claim text and gave no justification. Evidence-based systems, by contrast, depended on hand-built features and subjectivity lexicons. DeClarE is an end-to-end neural model that pulls in web articles related to a claim. It encodes each article with a biLSTM and uses claim-conditioned attention to weight article words by their relevance to the averaged claim embedding. It then combines this focused article vector with learned embeddings for the claim's origin and the article's source. Dense layers produce a credibility label or score for each article, and these per-article outputs are averaged into a verdict for the claim. The model was evaluated on four datasets, Snopes, PolitiFact, NewsTrust and SemEval, covering both classification and regression. It outperformed strong baselines without any feature engineering. Attention weights and source embeddings also offer readable explanations, which matters for fact-checking tools that users need to trust.

Key points

  • Architecture: shared word embeddings for claim and article, a biLSTM article encoder, claim-specific softmax attention (tanh worked best), and source embeddings for claims and articles, followed by two dense layers. Article-level scores are averaged per claim.
  • Datasets: Snopes (from Popat et al., 2017), PolitiFact, NewsTrust (from Mukherjee and Weikum, 2015) and the SemEval rumor task, scored with macro F1 and RMSE.
  • On NewsTrust regression, DeClarE (Full) lowered MSE by about 17% compared with the strongest baselines, LSTM-text and Distant Supervision. It also beat CCRF+SVR and CNN-text.
  • On SemEval it surpassed the top task systems, NileTMRG (closed variant) and IITP (open variant). Ablations show DeClarE (Plain), which uses only the biLSTM, performs notably worse than the full model.
  • PCA plots show learned source embeddings separating fake outlets (e.g., nationalreport, empirenews) from mainstream ones (nytimes, cnn) and grouping politicians by ideology. Attention highlights evidence phrases that explain each verdict.

Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.

Fields: Natural language processing

Cited by

73 papers of our library cite it (of the 140884 whose bibliographies we read); 6 of them have a page here, 67 not yet.

Cite

The paper itself, not this page. BibTeX from the library's record of the work. DOI 10.18653/v1/d18-1003

@inproceedings{popat2018declare,
  title = {{DeClarE: Debunking Fake News and False Claims using Evidence-Aware Deep Learning}},
  author = {Popat, Kashyap and Mukherjee, Subhabrata and Yates, Andrew and Weikum, Gerhard},
  booktitle = {Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing},
  year = {2018},
  doi = {10.18653/v1/d18-1003},
  url = {https://doi.org/10.18653/v1/d18-1003}
}

References

47 references, 6 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

  1. guage features and the stance of articles to assess the credibility of the natural language claims. However, their model heavily relies on hand-crafted language features. Rashkin et al. (2017); Wang (2017) propose neural network based approaches for determining the credibility of a textual claim, bu… find
  2. Vydiswaran et al. (2011) proposes an iterative algorithm which jointly learns the veracity of textual claims and trustworthiness of the sources. These approaches do not consider \#\# Related Work Our work is closely related to the following areas: Credibility analysis of Web claims: Our work builds up… find
  3. Wiebe and Riloff (2005); Lin et al. (2011); Recasens et al. (2013) study the problem of detecting bias in language, but do not consider credibility. Truth discovery: Prior approaches for truth discovery (Yin et al., 2008; Dong et al., 2009, 2015; Li et al., 2011, 2014, 2015; Pasternack and Roth, 201… find
  4. Nakashole and Mitchell (2014) proposed a method to extract conflicting values from the Web in the form of Subject-Predicate-Object (SPO) triplets and uses language objectivity analysis to determine the true value. Like the other truth discovery approaches, however, this approach is mainly suitable f… find
  5. Kumar et al. (2016) studies the problem of detecting hoax articles on Wikipedia. All these rely on domain-specific and community-specific features like retweets, likes, upvotes, etc. find
  6. Carlos Castillo, Marcelo Mendoza, and Barbara Poblete. 2011. Information credibility on twitter. In Proceedings of the 20th International Conference on World Wide Web, WWW ’11, pages 675–684, New York, NY, USA. ACM. find
  7. Leon Derczynski, Kalina Bontcheva, Maria Liakata, Rob Procter, Geraldine Wong Sak Hoi, and Arkaitz Zubiaga. 2017. Semeval-2017 task 8: Rumoureval: Determining rumour veracity and support for rumours. In Proceedings of the 11th International Workshop on Semantic Evaluation, SemEval@ACL 2017, Vancouve… find our card
  8. Xin Luna Dong, Laure Berti-Equille, and Divesh Srivastava. 2009. Integrating conflicting data: The role of source dependence. Proc. VLDB Endow., 2(1):550–561. find
  9. Xin Luna Dong, Evgeniy Gabrilovich, Kevin Murphy, Van Dang, Wilko Horn, Camillo Lugaresi, Shaohua Sun, and Wei Zhang. 2015. Knowledge-based trust: Estimating the trustworthiness of web sources. Proc. VLDB Endow., 8(9):938–949. find
  10. Omar Enayet and Samhaa R. El-Beltagy. 2017. Niletmrg at semeval-2017 task 8: Determining rumour and veracity support for rumours on twitter. In Proceedings of the 11th International Workshop on Semantic Evaluation, SemEval@ACL 2017, Vancouver, Canada, August 3-4, 2017, pages 470–474. find
  11. Jing Gao, Qi Li, Bo Zhao, Wei Fan, and Jiawei Han. 2015. Truth discovery and crowdsourcing aggregation: A unified perspective. PVLDB, 8(12):2048– 2049. find
  12. Alex Graves, Santiago Fernández, and Jürgen Schmidhuber. 2005. Bidirectional lstm networks for improved phoneme classification and recognition. In Proceedings of the 15th International Conference on Artificial Neural Networks: Formal Models and Their Applications - Volume Part II, ICANN’05, pages… find
  13. In this work, we propose a completely automated end-to-end neural network model, DeClarE, for evidence-aware credibility assessment of natural language claims without requiring hand-crafted features or lexicons. DeClarE captures signals from external evidence articles and models joint interactions b… find
  14. Diederik P. Kingma and Jimmy Ba. 2014. A method for stochastic optimization. abs/1412.6980. find
  15. Adam: CoRR, Srijan Kumar, Robert West, and Jure Leskovec. 2016. Disinformation on the web: Impact, characteristics, and detection of wikipedia hoaxes. In Proceedings of the 25th International Conference on World Wide Web, WWW ’16, pages 591–602, Republic and Credibility of user statements in health… find
  16. Canton of Geneva, Switzerland. International World Wide Web Conferences Steering Committee. find
  17. Qi Li, Yaliang Li, Jing Gao, Lu Su, Bo Zhao, Murat Demirbas, Wei Fan, and Jiawei Han. 2014. A confidence-aware approach for truth discovery on long-tail data. Proc. VLDB Endow., 8(4):425–436. find
  18. Ndapandula Nakashole and Tom M. Mitchell. 2014. Language-aware truth assessment of fact candidates. In Proceedings of the 52nd Annual Meeting of the Association for Computational Linguistics, ACL 2014, June 22-27, 2014, Baltimore, MD, USA, Volume 1: Long Papers, pages 1009–1019. find
  19. Xian Li, Weiyi Meng, and Clement Yu. 2011. T-verifier: Verifying truthfulness of fact statements. In Proceedings of the 2011 IEEE 27th International Conference on Data Engineering, ICDE ’11, pages 63–74, Washington, DC, USA. IEEE Computer Society. find
  20. Jeff Pasternack and Dan Roth. 2011. Making better informed trust decisions with generalized fact-finding. In IJCAI 2011, Proceedings of the 22nd International Joint Conference on Artificial Intelligence, Barcelona, Catalonia, Spain, July 16-22, 2011, pages 2324–2329. find
  21. Yaliang Li, Jing Gao, Chuishi Meng, Qi Li, Lu Su, Bo Zhao, Wei Fan, and Jiawei Han. 2016. A survey on truth discovery. SIGKDD Explor. Newsl., 17(2):1–16. find
  22. Yaliang Li, Qi Li, Jing Gao, Lu Su, Bo Zhao, Wei Fan, and Jiawei Han. 2015. On the discovery of evolving truth. In Proceedings of the 21th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, KDD ’15, pages 675–684, New York, NY, USA. ACM. find
  23. Jeff Pasternack and Dan Roth. 2013. Latent credibility analysis. In Proceedings of the 22Nd International Conference on World Wide Web, WWW ’13, pages 1009–1020, New York, NY, USA. ACM. find
  24. Jeffrey Pennington, Richard Socher, and Christopher D. Manning. 2014. Glove: Global vectors for word representation. In Empirical Methods in Natural Language Processing, EMNLP ’14. find
  25. Chenghua Lin, Yulan He, and Richard Everson. 2011. Sentence subjectivity detection with weakly-supervised learning. In Proceedings of 5th International Joint Conference on Natural Language Processing, pages 1153–1161. Asian Federation of Natural Language Processing. find
  26. Kashyap Popat, Subhabrata Mukherjee, Jannik Strötgen, and Gerhard Weikum. 2016. Credibility assessment of textual claims on the web. In Proceedings of the 25th ACM International on Conference on Information and Knowledge Management, CIKM ’16, pages 2173–2178, New York, NY, USA. ACM. find
  27. Shanshan Lyu, Wentao Ouyang, Huawei Shen, and Xueqi Cheng. 2017. Truth discovery by claim and source embedding. In Proceedings of the 2017 ACM on Conference on Information and Knowledge Management, CIKM ’17, pages 2183–2186, New York, NY, USA. ACM. find
  28. Kashyap Popat, Subhabrata Mukherjee, Jannik Strötgen, and Gerhard Weikum. 2017. Where the truth lies: Explaining the credibility of emerging claims on the web and social media. In Proceedings of the 26th International Conference on World Wide Web Companion, WWW ’17 Companion. find
  29. Fenglong Ma, Yaliang Li, Qi Li, Minghui Qiu, Jing Gao, Shi Zhi, Lu Su, Bo Zhao, Heng Ji, and Jiawei Han. 2015. Faitcrowd: Fine grained truth discovery for crowdsourced data aggregation. In Proceedings of the 21th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, KDD ’15, pa… find
  30. Vahed Qazvinian, Emily Rosengren, Dragomir R. Radev, and Qiaozhu Mei. 2011. Rumor has it: Identifying misinformation in microblogs. In Proceedings of the Conference on Empirical Methods in Natural Language Processing, EMNLP ’11, pages 1589–1599, Stroudsburg, PA, USA. Association for Computational Li… find our card
  31. Jing Ma, Wei Gao, Prasenjit Mitra, Sejeong Kwon, Bernard J. Jansen, Kam-Fai Wong, and Meeyoung Cha. 2016. Detecting rumors from microblogs with recurrent neural networks. In Proceedings of the Twenty-Fifth International Joint Conference on Artificial Intelligence, IJCAI’16, pages 3818–3824. AAAI Pre… find
  32. Hannah Rashkin, Eunsol Choi, Jin Yea Jang, Svitlana Volkova, and Yejin Choi. 2017. Truth of varying shades: Analyzing language in fake news and political fact-checking. In Proceedings of the 2017 Conference on Empirical Methods in Natural Language Processing, EMNLP ’17. find our card
  33. Subhabrata Mukherjee and Gerhard Weikum. 2015. Leveraging joint interactions for credibility analysis in news communities. In Proceedings of the 24th ACM International on Conference on Information and Knowledge Management, CIKM ’15. find
  34. Marta Recasens, Cristian Danescu-Niculescu-Mizil, and Dan Jurafsky. 2013. Linguistic models for analyzing and detecting biased language. In Proceedings of the 51st Annual Meeting of the Association Subhabrata Mukherjee, Gerhard Weikum, and Cristian Danescu-Niculescu-Mizil. 2014. People on drugs: Fan… find our card
  35. Natali Ruchansky, Sungyong Seo, and Yan Liu. 2017. Csi: A hybrid deep model for fake news detection. In Proceedings of the 2017 ACM on Conference on Information and Knowledge Management, CIKM ’17, pages 797–806, New York, NY, USA. ACM. find
  36. Xiaoxin Yin, Jiawei Han, and Philip S. Yu. 2008. Truth discovery with multiple conflicting information providers on the web. IEEE Trans. on Knowl. and Data Eng., 20(6):796–808. find
  37. Mehdi Samadi, Partha Talukdar, Manuela Veloso, and Manuel Blum. 2016. Claimeval: Integrated and flexible framework for claim evaluation using credibility of sources. In Proceedings of the Thirtieth AAAI Conference on Artificial Intelligence, AAAI’16, pages 222–228. AAAI Press. find
  38. Zhe Zhao, Paul Resnick, and Qiaozhu Mei. 2015. Enquiring minds: Early detection of rumors in social media from enquiry posts. In Proceedings of the 24th International Conference on World Wide Web, WWW ’15, pages 1395–1405, Republic and Canton of Geneva, Switzerland. International World Wide Web Conf… find
  39. Vikram Singh, Sunny Narayan, Md. Shad Akhtar, Asif Ekbal, and Pushpak Bhattacharyya. 2017. IITP at semeval-2017 task 8 : A supervised approach for rumour evaluation. In Proceedings of the 11th International Workshop on Semantic Evaluation, SemEval@ACL 2017, Vancouver, Canada, August 3-4, 2017, pages… find
  40. Shi Zhi, Bo Zhao, Wenzhu Tong, Jing Gao, Dian Yu, Heng Ji, and Jiawei Han. 2015. Modeling truth existence in truth discovery. In Proceedings of the 21th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, KDD ’15, pages 1543–1552, New York, NY, USA. ACM. find
  41. Svitlana Volkova, Kyle Shaffer, Jin Yea Jang, and Nathan Hodas. 2017. Separating facts from fiction: Linguistic models to classify suspicious and trusted news posts on twitter. In Proceedings of the 55th Annual Meeting of the Association for Computational Linguistics (Volume 2: Short Papers), pages… find
  42. V.G. Vinod Vydiswaran, ChengXiang Zhai, and Dan Roth. 2011. Content-driven trust propagation framework. In Proceedings of the 17th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, KDD ’11, pages 974–982, New York, NY, USA. ACM. find
  43. William Yang Wang. 2017. ”liar, liar pants on fire”: A new benchmark dataset for fake news detection. In Proceedings of the 55th Annual Meeting of the Association for Computational Linguistics, ACL 2017, Vancouver, Canada, July 30 - August 4, Volume 2: Short Papers, pages 422–426. find our card
  44. Janyce Wiebe and Ellen Riloff. 2005. Creating subjective and objective sentence classifiers from unannotated texts. In Proceedings of the 6th International Conference on Computational Linguistics and Intelligent Text Processing, CICLing’05, pages 486–497, Berlin, Heidelberg. Springer-Verlag. find
  45. John Wieting, Mohit Bansal, Kevin Gimpel, and Karen Livescu. 2015. Towards universal paraphrastic sentence embeddings. In Proceedings of the International Conference on Learning Representations (ICLR). find our card
  46. Qiongkai Xu and Hai Zhao. 2012. Using deep linguistic features for finding deceptive opinion spam. In Proceedings of COLING 2012: Posters, pages 1341–1350. find
  47. The COLING 2012 Organizing Committee. find

↑ back to top

This page in Markdown → https://md.research.elamur.ai/free/en/papers/popat-2018-declare-debunking-fake-news-false/