Elamur Research⌕
☰
home / papers / 2018 / Automatic Stance Detection Using End-to-End Memory Networks

Automatic Stance Detection Using End-to-End Memory Networks

Summary

This paper addresses stance detection as a step toward automatic fact checking. Given a claim and a news document, the system labels the document as agree, disagree, discuss, or unrelated, and it also points to the passages that justify the label. The authors build an end-to-end memory network in which each document is split into paragraphs, and each paragraph serves as one unit of evidence. Paragraphs and the claim are each encoded by both an LSTM and a CNN. The two kinds of encodings enter the memory separately instead of being chained together. A new inference component uses learned similarity matrices, together with TF.IDF similarity, to measure how relevant each paragraph is to the claim. These scores reweight the memory before an MLP with softmax predicts the stance. The same scores also rank paragraphs and CNN n-grams as evidence snippets. On the FakeNewsChallenge benchmark, this feature-light model comes close to the state of the art. The similarity-based extension gives sizable gains over a plain memory network. The work is useful because it both classifies stance and surfaces evidence that can assist human fact checkers.

Key points

  • The architecture adds one element to the classic five-part memory network {M, I, G, O, R}, giving six components: {M, I, F, G, O, R}. The added inference component F uses similarity matrices M (for LSTM encodings) and M′ (for CNN encodings). All parameters are learned jointly, including the encoders, both matrices, and the final MLP.
  • A Maxout layer pools across CNN feature maps so that n-grams can be tracked for later snippet extraction. The output vector combines two parts: the mean of the reweighted paragraph vectors, and the max and mean of the TF.IDF, LSTM and CNN claim–paragraph similarities.
  • For agree/disagree cases, snippet extraction was evaluated without supervision, because FakeNewsChallenge provides no gold evidence. Precision at ranks 1–5 was:
    • 0.25–0.35 for fixed 5-grams;
    • 0.38–0.42 after merging adjacent n-grams with similar scores;
    • 0.55–0.62 at sentence level.
  • The authors contrast their model with the 2017 FNC winner (Baird et al., 2017), an ensemble of gradient-boosted trees using rich hand-crafted features plus a deep CNN. The exact FNC classification scores fall in the part of the text that was not provided.

Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.

Fields: Linguistics & languages

Cited by

30 papers of our library cite it (of the 140673 whose bibliographies we read); 6 of them have a page here, 24 not yet.

Cite

The paper itself, not this page. BibTeX from the library's record of the work. DOI 10.18653/v1/n18-1070 · arXiv 1804.07581

@inproceedings{mohtarami2018automatic,
  title = {{Automatic Stance Detection Using End-to-End Memory Networks}},
  author = {Mohtarami, Mitra and Baly, Ramy and Glass, James and Nakov, Preslav and Màrquez, Lluís and Moschitti, Alessandro},
  booktitle = {Proceedings of the 2018 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long Papers)},
  year = {2018},
  doi = {10.18653/v1/n18-1070},
  eprint = {1804.07581},
  archivePrefix = {arXiv},
  url = {https://doi.org/10.18653/v1/n18-1070}
}

References

26 references, 5 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

  1. Sean Baird, Doug Sibley, and Yuxi Pan. 2017. Ta-los targets disinformation with fake news challenge victory. https://blog.talosintelligence.com/2017/06/ talos-fake-news-challenge.html. ↗
  2. Ramy Baly, Mitra Mohtarami, James Glass, Lluís Màrquez, Alessandro Moschitti, and Preslav Nakov. 2018. Integrating stance detection and fact checking in a unified corpus. In Proceedings of HLT-NAACL. New Orleans, LA, USA. find our card
  3. Hannah Rashkin, Eunsol Choi, Jin Yea Jang, Svitlana Volkova, and Yejin Choi. 2017. Truth of varying shades: Analyzing language in fake news and political fact-checking. In Proceedings of EMNLP. Copenhagen, Denmark, pages 2931–2937. find
  4. Jeff Donahue, Lisa Anne Hendricks, Sergio Guadar-rama, Marcus Rohrbach, Subhashini Venugopalan, Trevor Darrell, and Kate Saenko. 2015. Long-term recurrent convolutional networks for visual recognition and description. In Proceedings of CVPR. Boston, MA, USA, pages 2625–2634. find our card
  5. Tara N Sainath, Oriol Vinyals, Andrew Senior, and Haşim Sak. 2015. Convolutional, long short-term memory, fully connected deep neural networks. In Proceedings of ICASSP. Brisbane, Australia, pages 4580–4584. find
  6. Pepa Gencheva, Preslav Nakov, Lluís Màrquez, Alberto Barrón-Cedeño, and Ivan Koychev. 2017. A context-aware approach for detecting worth-checking claims in political debates. In Proceedings of RANLP. Varna, Bulgaria, pages 267–276. find
  7. Prashant Shiralkar, Alessandro Flammini, Filippo Menczer, and Giovanni Luca Ciampaglia. 2017. Finding streams in knowledge graphs to support fact checking. In Proceedings of ICDM. New Orleans, LA, USA, pages 859–864. find
  8. Ian J. Goodfellow, David Warde-Farley, Mehdi Mirza, Aaron Courville, and Yoshua Bengio. 2013. Maxout networks. In Proceedings of ICML. Atlanta, GA, USA, pages 1319–1327. find
  9. Karen Spärck Jones. 2004. IDF term weighting and IR research lessons. Journal of documentation 60(5):521–523. find
  10. Naeemul Hassan, Chengkai Li, and Mark Tremayne. 2015. Detecting check-worthy factual claims in presidential debates. In Proceedings CIKM. Melbourne, Australia, pages 1835–1838. find
  11. Sainbayar Sukhbaatar, Arthur Szlam, Jason Weston, and Rob Fergus. 2015. End-to-end memory networks. In Proceedings of NIPS. Montreal, Canada, pages 2440–2448. find
  12. Israa Jaradat, Pepa Gencheva, Alberto Barrón-Cedeño, Lluís Màrquez, and Preslav Nakov. 2018. Claim-Rank: Detecting check-worthy claims in Arabic and English. In Proceedings of HLT-NAACL. New Orleans, LA, USA. find
  13. Ming Tan, Cicero dos Santos, Bing Xiang, and Bowen Zhou. 2016. Improved representation learning for question answer matching. In Proceedings of ACL. Berlin, Germany, pages 464–473. find
  14. Georgi Karadzhov, Preslav Nakov, Lluís Màrquez, Alberto Barrón-Cedeño, and Ivan Koychev. 2017. Fully automated fact checking using external sources. In Proceedings of RANLP. Varna, Bulgaria, pages 344–353. find
  15. James Thorne, Andreas Vlachos, Christos Christodoulopoulos, and Arpit Mittal. 2018. FEVER: A large-scale dataset for fact extraction and VERification. In Proceedings of HLT-NAACL. New Orleans, LA, USA. find our card
  16. Todor Mihaylov, Georgi Georgiev, and Preslav Nakov. 2015. Finding opinion manipulation trolls in news community forums. In Proceedings of CoNLL. Beijing, China, pages 310–314. find
  17. Andreas Vlachos and Sebastian Riedel. 2014. Fact checking: Task definition and dataset construction. In Proceedings of the ACL 2014 Workshop on Language Technologies and Computational Social Science. Baltimore, MD, USA, pages 18–22. find our card
  18. Todor Mihaylov and Preslav Nakov. 2016. Hunting for troll comments in news community forums. In Proceedings of ACL. Berlin, Germany. find
  19. Soroush Vosoughi, Deb Roy, and Sinan Aral. 2018. The spread of true and false news online. Science 359(6380):1146–1151. find
  20. Tsvetomila Mihaylova, Preslav Nakov, Lluis Marquez, Alberto Barron-Cedeno, Mitra Mohtarami, Georgi Karadzhov, and James Glass. 2018. Fact checking in community forums. In Proceedings of AAAI. New Orleans, LA, USA. find
  21. Guido Zarrella and Amy Marsh. 2016. MITRE at SemEval-2016 Task 6: Transfer learning for stance detection. In Proceedings of SemEval. San Diego, CA, USA, pages 458–463. find our card
  22. Saif Mohammad, Svetlana Kiritchenko, Parinaz Sobhani, Xiao-Dan Zhu, and Colin Cherry. 2016. SemEval-2016 task 6: Detecting stance in tweets. In Proceedings of SemEval. Berlin, Germany, pages 31–41. find
  23. Arkaitz Zubiaga, Elena Kochkina, Maria Liakata, Rob Procter, and Michal Lukasik. 2016. Stance classification in rumours as a sequential task exploiting the tree structure of social media conversations. In Proceedings of COLING. Osaka, Japan, pages 2438– 2448. find
  24. Jeffrey Pennington, Richard Socher, and Christopher Manning. 2014. Glove: Global Vectors for Word Representation. In Proceedings of EMNLP. Doha, Qatar, pages 1532–1543. find
  25. Zhen Zuo, Bing Shuai, Gang Wang, Xiao Liu, Xingxing Wang, Bing Wang, and Yushi Chen. 2015. Convolutional recurrent neural networks: Learning spatial dependencies for image representation. In Proceedings of CVPR. Boston, MA, USA, pages 18–26. find
  26. Kashyap Popat, Subhabrata Mukherjee, Jannik Strötgen, and Gerhard Weikum. 2017. Where the truth lies: Explaining the credibility of emerging claims on the web and social media. In Proceedings of WWW. Perth, Australia, pages 1003–1012. find

↑ back to top

This page in Markdown → https://md.research.elamur.ai/free/papers/mohtarami-2018-automatic-stance-detection-end-end/