Elamur Research⌕
☰
home / papers / 2021 / A Few Topical Tweets are Enough for Effective User Stance Detection

A Few Topical Tweets are Enough for Effective User Stance Detection

Summary

The paper addresses user-level stance detection on Twitter for users who rarely post about a target, often with only one or two relevant tweets. Unsupervised clustering already labels prolific users almost perfectly, but it fails on these quieter users. The authors use eight polarized US-centric topics, including gun control, climate change, the Kavanaugh nomination and vaccination. They build the training data automatically: the most active users are clustered, and balanced sets of 500 users per stance are sampled. For testing, they manually annotate about 100 low-activity users per topic.

They test two strategies. The first fine-tunes BERT on each topic to classify users from their few topical tweets, and it beats SVM and fastText baselines. The second retrieves each test user's broader timeline, even when it is off-topic, and places the user in the clusters of already-labeled users. This exploits homophily, since like-minded users tend to retweet the same accounts. Timeline expansion with unsupervised clustering gives the best results. Fine-tuned BERT is the recommended option when timelines cannot be obtained. Together, the two methods make it possible to label both vocal and non-vocal users largely automatically.

Key points

  • Training labels: They come from the unsupervised method of Darwish et al. (2020), applied to the top 5,000 users with at least 10 tweets. Similarity is based on retweeted accounts, users are projected with UMAP and then clustered with mean shift. This method's accuracy was estimated at about 98%.
  • BERT without expansion: 89.6% accuracy and 91.3% macro F, significantly above the SVM and fastText baselines.
  • Timeline expansion with unsupervised clustering: 95.6% accuracy and 93.1% macro F. Expanding timelines also improved some supervised configurations.
  • BERT errors (129 misclassified tweets): They fall into four types: unexplained, vague, quoting the opposing side, and sarcasm. A LIME analysis showed that BERT also relied on retweeted accounts and on correlations between topics.
  • Clustering errors (15 misclassified users, 7 from Kavanaugh): The main cause was too few retweets of political or topical accounts. Other causes were non-US accounts and users departing from their group's typical stance.

Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.

Fields: Linguistics & languages

Cited by

11 papers of our library cite it (of the 140673 whose bibliographies we read); 1 of them have a page here, 10 not yet.

Cite

The paper itself, not this page. BibTeX from the library's record of the work. DOI 10.18653/v1/2021.eacl-main.227

@inproceedings{samih2021few,
  title = {{A Few Topical Tweets are Enough for Effective User Stance Detection}},
  author = {Samih, Younes and Darwish, Kareem},
  booktitle = {Proceedings of the 16th Conference of the European Chapter of the Association for Computational Linguistics: Main Volume},
  year = {2021},
  doi = {10.18653/v1/2021.eacl-main.227},
  url = {https://doi.org/10.18653/v1/2021.eacl-main.227}
}

References

39 references, 3 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

  1. Abeer Aldayel and Walid Magdy. 2019. Your stance is exposed! analysing possible factors for stance detection on social media. Proc. ACM Hum.-Comput. Interact., 3(CSCW). find
  2. Pablo Barberá. 2015. Birds of the same feather tweet together: Bayesian ideal point estimation using Twitter data. Political Analysis, 23(1):76–91. find
  3. Pablo Barberá and Gonzalo Rivero. 2014. Understanding the political representativeness of Twitter users. Social Science Computer Review, page 0894439314558836. find
  4. Steven Bird, Ewan Klein, and Edward Loper. 2009. Natural language processing with Python: analyzing text with the natural language toolkit. ” O’Reilly Media, Inc.”. find
  5. Javier Borge-Holthoefer, Walid Magdy, Kareem Darwish, and Ingmar Weber. 2015. Content and network dynamics behind Egyptian political polarization on Twitter. In Proceedings of the 18th ACM Conference on Computer Supported Cooperative Work & Social Computing, pages 700–711. find
  6. Raviv Cohen and Derek Ruths. 2013. Classifying political orientation on Twitter: It’s not easy! In ICWSM 2013. find
  7. Elanor Colleoni, Alessandro Rozza, and Adam Arvids-son. 2014. Echo chamber or public sphere? Predicting political orientation and measuring political homophily in Twitter using big data. Journal of Communication, 64(2):317–332. find
  8. Michael D Conover, Bruno Gonçalves, Jacob Ratkiewicz, Alessandro Flammini, and Filippo Menczer. 2011. Predicting the political alignment of Twitter users. In Proceedings of the 2011 IEEE Third International Conference on Privacy, Security, Risk and Trust (PASSAT) and 2011 IEEE Third Inernational Co… find
  9. Kareem Darwish, Walid Magdy, Afshin Rahimi, Timothy Baldwin, Norah Abokhodair, et al. 2018. Predicting online islamophobic behavior after\# parisat-tacks. The Journal of Web Science, 4(3):34–52. find
  10. Kareem Darwish, Peter Stefanov, Michaël Aupetit, and Preslav Nakov. 2020. Unsupervised user stance detection on twitter. In Proceedings of the International AAAI Conference on Web and Social Media, volume 14, pages 141–152. find
  11. Jacob Devlin, Ming-Wei Chang, Kenton Lee, and Kristina Toutanova. 2019. BERT: Pre-training of deep bidirectional transformers for language understanding. In Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies,… find
  12. James H Fowler, Michael T Heaney, David W Nickerson, John F Padgett, and Betsy Sinclair. 2011. Causality in political networks. American Politics Research, 39(2):437–480. find
  13. Keinosuke Fukunaga and Larry Hostetler. 1975. The estimation of the gradient of a density function, with applications in pattern recognition. IEEE Transactions on information theory, 21(1):32–40. find
  14. Itai Himelboim, Stephen McCreery, and Marc Smith. 2013. Birds of a feather tweet together: Integrating network and content analyses to examine cross-ideology exposure on Twitter. Journal of Computer-Mediated Communication, 18(2):40–60. find
  15. Jeremy Howard and Sebastian Ruder. 2018. Universal language model fine-tuning for text classification. In Proceedings of the 56th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), Melbourne, Australia. Association for Computational Linguistics. find
  16. T. Joachims. 2002. Learning to Classify Text Using Support Vector Machines – Methods, Theory, and Algorithms. Kluwer/Springer. find
  17. Armand Joulin, Edouard Grave, Piotr Bojanowski, and Tomas Mikolov. 2016. Bag of tricks for efficient text classification. arXiv preprint arXiv:1607.01759. find
  18. Guillaume Lample and Alexis Conneau. 2019. Cross-lingual language model pretraining. arXiv preprint arXiv:1901.07291. find
  19. Zhenzhong Lan, Mingda Chen, Sebastian Goodman, Kevin Gimpel, Piyush Sharma, and Radu Soricut. 2019. Albert: A lite bert for self-supervised learning of language representations. arXiv preprint arXiv:1909.11942. find
  20. Yinhan Liu, Myle Ott, Naman Goyal, Jingfei Du, Mandar Joshi, Danqi Chen, Omer Levy, Mike Lewis, Luke Zettlemoyer, and Veselin Stoyanov. 2019. Roberta: A robustly optimized bert pretraining approach. arXiv preprint arXiv:1907.11692. find
  21. Walid Magdy, Kareem Darwish, Norah Abokhodair, Afshin Rahimi, and Timothy Baldwin. 2016a. \#isi-sisnotislam or \#deportallmuslims?: Predicting unspoken views. In Proceedings of the 8th ACM Conference on Web Science, pages 95–106. ACM. find
  22. Walid Magdy, Kareem Darwish, and Ingmar Weber. 2016b. \#failedrevolutions: Using Twitter to study the antecedents of ISIS support. First Monday, 21(2). find
  23. Aibek Makazhanov, Davood Rafiei, and Muhammad Waqar. 2014. Predicting political preference of Twitter users. Social Network Analysis and Mining, 4(1):1–15. find
  24. Leland McInnes and John Healy. 2018. Umap: Uniform manifold approximation and projection for dimension reduction. arXiv preprint arXiv:1802.03426. find our card
  25. Tomas Mikolov, Kai Chen, Greg Corrado, and Jeffrey Dean. 2013. Efficient estimation of word representations in vector space. arXiv preprint arXiv:1301.3781. find
  26. Saif Mohammad, Svetlana Kiritchenko, Parinaz Sobhani, Xiaodan Zhu, and Colin Cherry. 2016a. Semeval-2016 task 6: Detecting stance in tweets. In Proceedings of the 10th International Workshop on Semantic Evaluation (SemEval-2016), pages 31–41. find
  27. Saif M Mohammad, Svetlana Kiritchenko, Parinaz Sobhani, Xiaodan Zhu, and Colin Cherry. 2016b. Semeval-2016 task 6: Detecting stance in tweets. In SemEval 2016, volume 16. find
  28. Marco Pennacchiotti and Ana-Maria Popescu. 2011. Democrats, Republicans and Starbucks afficionados: user classification in Twitter. In Proceedings of ACM SIGKDD, pages 430–438. find
  29. Jeffrey Pennington, Richard Socher, and Christopher Manning. 2014. Glove: Global vectors for word representation. In Proceedings of the 2014 conference on empirical methods in natural language processing (EMNLP), pages 1532–1543. find
  30. Matt Thomas, Bo Pang, and Lillian Lee. 2006. Get out the vote: Determining support or opposition from congressional floor-debate transcripts. In Proceedings of the 2006 Conference on Empirical Methods in Natural Language Processing, pages 327–335, Sydney, Australia. Association for Computational Lin… find
  31. Ingmar Weber, Venkata R Kiran Garimella, and Alaa Batayneh. 2013. Secular vs. islamist polarization in Egypt on Twitter. In ASONAM 2013, pages 290– 297. find
  32. Felix Ming Fai Wong, Chee Wei Tan, Soumya Sen, and Mung Chiang. 2013. Quantifying political leaning from tweets and retweets. In ICWSM 2013, pages 640–649. find
  33. Guido Zarrella and Amy Marsh. 2016. Mitre at semeval-2016 task 6: Transfer learning for stance detection. arXiv preprint arXiv:1606.03784. find our card
  34. Matthew E. Peters, Mark Neumann, Mohit Iyyer, Matt Gardner, Christopher Clark, Kenton Lee, and Luke Zettlemoyer. 2018. Deep contextualized word representations. ArXiv, abs/1802.05365. find
  35. Kashyap Popat, Subhabrata Mukherjee, Andrew Yates, and Gerhard Weikum. 2019. STANCY: Stance classification based on consistency cues. In Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing (E… find
  36. Alec Radford, Karthik Narasimhan, Tim Salimans, and Ilya Sutskever. 2018. Improving language understanding by generative pre-training. URL https://s3-us-west-2.amazonaws. com/openai-assets/researchcovers/languageunsupervised/language understanding paper. pdf. ↗
  37. Marco Tulio Ribeiro, Sameer Singh, and Carlos Guestrin. 2016. ”why should I trust you?”: Explaining the predictions of any classifier. In Proceedings of the 22nd ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, San Francisco, CA, USA, August 13-17, 2016, pages 1135–1144. find our card
  38. Peter Stefanov, Kareem Darwish, Atanas Atanasov, and Preslav Nakov. 2019. Predicting the topical stance of media and popular twitter users. arXiv preprint arXiv:1907.01260. — [arXiv:1907.01260](https://arxiv.org/abs/1907.01260) find
  39. Peter Stefanov, Kareem Darwish, Atanas Atanasov, and Preslav Nakov. 2020. Predicting the topical stance and political leaning of media using tweets. The 58th Annual Meeting of the Association for Computational Linguistics (ACL’2020). find

↑ back to top

This page in Markdown → https://md.research.elamur.ai/free/papers/samih-2021-few-topical-tweets-are-enough/