Elamur Research⌕
☰
home / papers / 2021 / P-Stance: A Large Dataset for Stance Detection in Political Domain

P-Stance: A Large Dataset for Stance Detection in Political Domain

Paper · 2021 · Yingjie Li, Tiberiu Sosea, Aditya Sawant et al.

Summary

Stance detection asks whether an author supports or opposes a given target. Progress has been held back by small benchmarks. Their tweets are also short, and the stance often shows through obvious keywords or direct mentions of the target. The authors build P-Stance, an English Twitter corpus from the 2020 U.S. presidential race that covers three politicians: Donald Trump, Joe Biden and Bernie Sanders. They gathered about 2.8 million tweets through the Twitter streaming API using pro, anti and neutral seed hashtags. After length filtering and removing duplicates and retweets, they manually labeled 21,574 tweets. Among the models they test, fine-tuned BERTweet does best with 80.53% macro-F1. Uncertainty-aware Self-Training (UST) on a large unlabeled companion set, P-Stance-EXT, raises this further. The corpus also supports cross-target transfer and a new cross-topic setting, where a model trained on 2016 data about a politician is tested on 2020 tweets about the same person. Overall, it gives political opinion mining a bigger and harder benchmark with less reliance on surface cues.

Key points

  • P-Stance has 21,574 labeled tweets, more than three times the size of SemEval-2016 (4,870). Tweets were kept only if they had 10–128 words.
  • Annotator agreement (Krippendorff's alpha) averages 0.60 for the 3-class setup and 0.81 for the 2-class setup.
  • BERTweet scores 82.48 / 81.02 / 78.09 F1 on Trump, Biden and Sanders, beating BERT and other baselines.
  • UST-ALL reaches 82.42 macro-F1, 1.89 points above BERTweet. With only 100 labeled examples, UST is within 1.6 points of the full-data model on Biden.
  • In cross-topic tests (2016 → 2020), BERTweet scores 73.58 on Trump and 66.48 on Sanders. Dataset and code: github.com/chuchun8/PStance.

Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.

 Institutions

🎓University of Illinois ChicagoChicago · United States38 works · 19,844 citations🎓University of OttawaOttawa · Canada9 works · 2,393 citations

 Fields, topics and concepts

📚 Shelves 4The field of science this work belongs to (OpenAlex fields).Computer Science · 9,085 worksArtificial Intelligence · 8,888 worksArts and Humanities · 9,296 worksLanguage and Linguistics · 9,206 works

🧭 Topics 1Research subjects and languages the work is about.Sentiment & emotion · 609 works

Appears in

Articles of ours that cite this paper.

AgglomerationsOttawa

Cite

Cite as Yingjie Li, Tiberiu Sosea, Aditya Sawant, Ajith Jayaraman Nair, Diana Inkpen and Cornelia Caragea. 2021. P-Stance: A Large Dataset for Stance Detection in Political Domain. Findings of the Association for Computational Linguistics: ACL-IJCNLP 2021.

The paper itself, not this page. BibTeX from the library's record of the work. DOI 10.18653/v1/2021.findings-acl.208

@inproceedings{li2021p,
  title = {{P-Stance: A Large Dataset for Stance Detection in Political Domain}},
  author = {Yingjie Li, Tiberiu Sosea, Aditya Sawant, Ajith Jayaraman Nair, Diana Inkpen and Cornelia Caragea},
  booktitle = {Findings of the Association for Computational Linguistics: ACL-IJCNLP 2021},
  year = {2021},
  doi = {10.18653/v1/2021.findings-acl.208},
  url = {https://doi.org/10.18653/v1/2021.findings-acl.208}
}

References

4 references, 3 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

  1. Abeer ALDayel and Walid Magdy. 2021. Stance detection on social media: State of the art and trends. Information Processing & Management, 58(4):102597. find
  2. Isabelle Augenstein, Tim Rocktäschel, Andreas Vlachos, and Kalina Bontcheva. 2016. Stance detection with bidirectional conditional encoding. In Proceedings of the 2016 Conference on Empirical Methods in Natural Language Processing, pages 876–885.
  3. Roy Bar-Haim, Indrajit Bhattacharya, Francesco Din-uzzo, Amrita Saha, and Noam Slonim. 2017. Stance classification of context-dependent claims. In Proceedings of the 15th Conference of the European Chapter of the Association for Computational Linguistics: Volume 1, Long Papers, pages 251–261.
  4. Costanza Conforti, Jakob Berndt, Mohammad Taher Pilehvar, Chryssi Giannitsarou, Flavio Toxvaerd, and Nigel Collier. 2020a. STANDER: An expert-annotated dataset for news stance detection and evidence retrieval. In Findings of the Association for Computational Linguistics: EMNLP 2020, pages 4086–4101.

Updated

This page in Markdown → https://md.research.elamur.ai/free/en/papers/li-2021-p-stance-large-dataset-stance/