Elamur Research
☰
home / papers / 2016 / Stance Detection with Bidirectional Conditional Encoding

Stance Detection with Bidirectional Conditional Encoding

Written for the Research library (claude-opus-5-5) from the paper text. Original — aclanthology.org/D16-1084.pdf

Summary

The paper looks at stance detection on tweets, meaning whether a tweet is for, against or neutral toward a target. The hard part is that the test target may not appear in the tweet, and there is no labelled training data for it. The authors use the SemEval 2016 Task 6 Twitter corpus. In Task B the unseen target is Donald Trump, and the models learn only from tweets about five other targets. The method is conditional encoding: one LSTM reads the target, and its final state starts a second LSTM that reads the tweet, so the tweet representation depends on the target. A bidirectional version (BiCond) reads both sequences in each direction. Word embeddings are initialised from word2vec trained on a large collection of crawled, unlabelled tweets. Without any Trump labels, BiCond reaches 0.4901 macro-F1, second only to a system that used automatically labelled Trump tweets. When the authors add their own regex-based weak labels, BiCond reaches 0.5803 and becomes the new state of the art. This matters because labelled data rarely exists for every target of interest.

Key points

  • The model architecture runs from independent target and tweet LSTMs, to conditional encoding, to bidirectional conditional encoding (BiCond). Each step improves on the one before and beats the SVM, BoWV and tweet-only baselines.
  • The labelled training set is small, with 5,628 Task A tweets in total. To compensate, a skip-gram word2vec model (100 dimensions) is pre-trained on 395,212 tweets collected through keyword search, and this pre-training clearly helps.
  • With no Trump labels, BiCond scores 0.4901 macro-F1 over the FAVOR and AGAINST classes, compared with 0.2972 for the majority baseline and 0.2843 for SVM-ngrams-comb.
  • For weak supervision, the authors label the unlabelled Trump tweets with hashtag and phrase patterns such as #makeamericagreatagain (positive) and #dumptrump (negative). BiCond trained on these labels scores 0.5803, ahead of pkudblab (0.5628) and LitisMind (0.4466).
  • The authors released their automatic labelling method and dataset publicly in their code repository.

Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.

Fields: Linguistics & languages

Cited by

138 papers of our library cite it (of the 140673 whose bibliographies we read); 2 of them have a page here, 136 not yet.

Cite

The paper itself, not this page. BibTeX from the library's record of the work. DOI 10.18653/v1/d16-1084 · arXiv 1606.05464

@inproceedings{augenstein2016stance,
  title = {{Stance Detection with Bidirectional Conditional Encoding}},
  author = {Augenstein, Isabelle and Rocktäschel, Tim and Vlachos, Andreas and Bontcheva, Kalina},
  booktitle = {Proceedings of the 2016 Conference on Empirical Methods in Natural Language Processing},
  year = {2016},
  doi = {10.18653/v1/d16-1084},
  eprint = {1606.05464},
  archivePrefix = {arXiv},
  url = {https://doi.org/10.18653/v1/d16-1084}
}

References

24 references, 0 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

  1. Abdulaziz Alghunaim, Mitra Mohtarami, Scott Cyphers, and Jim Glass. 2015. A Vector Space Approach for Aspect Based Sentiment Analysis. In Proceedings of the 1st Workshop on Vector Space Modeling for Natural Language Processing, pages 116–122, Denver, Colorado, June. Association for Computational Lin… find
  2. Isabelle Augenstein, Andreas Vlachos, and Kalina Bontcheva. 2016. USFD: Any-Target Stance Detection on Twitter with Autoencoders. In Proceedings of the International Workshop on Semantic Evaluation, SemEval ’16, San Diego, California. find
  3. Samuel R. Bowman, Gabor Angeli, Christopher Potts, and Christopher D. Manning. 2015. A large annotated corpus for learning natural language inference. In Proceedings of the 2015 Conference on Empirical Methods in Natural Language Processing, pages 632–642, Lisbon, Portugal, September. Association fo… find
  4. Marcelo Dias and Karin Becker. 2016. INF-UFRGS-OPINION-MINING at SemEval-2016 Task 6: Automatic Generation of a Training Corpus for Unsupervised Identification of Stance in Tweets. In Proceedings of the International Workshop on Semantic Evaluation, SemEval ’16, San Diego, California, June. Ben Eisn… find
  5. Adam Faulkner. 2014. Automated Classification of Stance in Student Essays: An Approach Using Stance Target Information and the Wikipedia Link-Based Measure. In William Eberle and Chutima Boonthum-Denecke, editors, FLAIRS Conference. find
  6. AAAI Press. William Ferreira and Andreas Vlachos. 2016. Emergent: a novel data-set for stance classification. In Proceedings of the 2016 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, pages 1163– 1168, San Diego, California, Ju… find
  7. Kazi Saidul Hasan and Vincent Ng. 2013. Stance Classification of Ideological Debates: Data, Models, Features, and Constraints. In IJCNLP, pages 1348–1356. Asian Federation of Natural Language Processing / ACL. find
  8. Sepp Hochreiter and Jürgen Schmidhuber. 1997. Long short-term memory. Neural computation, 9(8):1735– 1780. find
  9. Diederik P. Kingma and Jimmy Ba. 2014. Adam: A Method for Stochastic Optimization. CoRR, abs/1412.6980. find
  10. Xiaomo Liu, Armineh Nourbakhsh, Quanzhi Li, Rui Fang, and Sameena Shah. 2015. Real-time Rumor Debunking on Twitter. In Proceedings of the 24th ACM International on Conference on Information and Knowledge Management, CIKM ’15, pages 1867–1870, New York, NY, USA. find
  11. ACM. Marcelo Mendoza, Barbara Poblete, and Carlos Castillo. 2010. Twitter Under Crisis: Can We Trust What We RT? In Proceedings of the First Workshop on Social Media Analytics (SOMA’2010), pages 71–79, New York, NY, USA. find
  12. ACM. Tomas Mikolov, Kai Chen, Greg Corrado, and Jeffrey Dean. 2013. Efficient estimation of word representations in vector space. arXiv preprint arXiv:1301.3781. find
  13. Margaret Mitchell, Jacqui Aguilar, Theresa Wilson, and Benjamin Van Durme. 2013. Open Domain Targeted Sentiment. In Proceedings of the 2013 Conference on Empirical Methods in Natural Language Processing, pages 1643–1654, Seattle, Washington, USA, October. Association for Computational Linguistics. find
  14. Saif M. Mohammad, Svetlana Kiritchenko, Parinaz Sobhani, Xiaodan Zhu, and Colin Cherry. 2016. SemEval-2016 Task 6: Detecting stance in tweets. In Proceedings of the International Workshop on Semantic Evaluation, SemEval ’16, San Diego, California, June. find
  15. Bo Pang and Lillian Lee. 2008. Opinion mining and sentiment analysis. Foundations and trends in information retrieval, 2(1-2):1–135. find
  16. F. Pedregosa, G. Varoquaux, A. Gramfort, V. Michel, B. Thirion, O. Grisel, M. Blondel, P. Prettenhofer, R. Weiss, V. Dubourg, J. Vanderplas, A. Passos, D. Cournapeau, M. Brucher, M. Perrot, and E. Duchesnay. 2011. Scikit-learn: Machine Learning in Python. Journal of Machine Learning Research, 12:282… find
  17. Vahed Qazvinian, Emily Rosengren, Dragomir R. Radev, and Qiaozhu Mei. 2011. Rumor Has It: Identifying Misinformation in Microblogs. In Proceedings of the Conference on Empirical Methods in Natural Language Processing, EMNLP ’11, pages 1589–1599. Tim Rocktäschel, Edward Grefenstette, Karl Moritz Her… find
  18. Richard Socher, Alex Perelygin, Jean Wu, Jason Chuang, Christopher D. Manning, Andrew Ng, and Christopher Potts. 2013. Recursive Deep Models for Semantic Compositionality Over a Sentiment Treebank. In Proceedings of the 2013 Conference on Empirical Methods in Natural Language Processing, pages 1631–… find
  19. Duy-Tin Vo and Yue Zhang. 2015. Target-Dependent Twitter Sentiment Classification with Rich Automatic Features. In Qiang Yang and Michael Wooldridge, editors, IJCAI, pages 1347–1353. find
  20. AAAI Press. Marilyn Walker, Pranav Anand, Rob Abbott, and Ricky Grant. 2012. Stance Classification using Dialogic Properties of Persuasion. In Proceedings of the 2012 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, pages 592–596… find
  21. Wan Wei, Xiao Zhang, Xuqin Liu, Wei Chen, and Tengjiao Wang. 2016. pkudblab at SemEval-2016 Task 6: A Specific Convolutional Neural Network System for Effective Stance Detection. In Proceedings of the International Workshop on Semantic Evaluation, SemEval ’16, San Diego, California, June. find
  22. Guido Zarrella and Amy Marsh. 2016. MITRE at SemEval-2016 Task 6: Transfer Learning for Stance Detection. In Proceedings of the International Workshop on Semantic Evaluation, SemEval ’16, San Diego, California, June. find
  23. Meishan Zhang, Yue Zhang, and Duy Tin Vo. 2015. Neural Networks for Open Domain Targeted Sentiment. In Proceedings of the 2015 Conference on Empirical Methods in Natural Language Processing, pages 612–621, Lisbon, Portugal, September. Association for Computational Linguistics. find
  24. Meishan Zhang, Yue Zhang, and Duy-Tin Vo. 2016. Gated Neural Networks for Targeted Sentiment Analysis. In Proceedings of the Thirtieth AAAI Conference on Artificial Intelligence, Phoenix, Arizona, USA, February. Association for the Advancement of Artificial Intelligence. find