Elamur Research⌕
☰
home / papers / 2021 / P-Stance: A Large Dataset for Stance Detection in Political Domain

P-Stance: A Large Dataset for Stance Detection in Political Domain

Summary

Stance detection asks whether an author supports or opposes a given target. Progress has been held back by small benchmarks. Their tweets are also short, and the stance often shows through obvious keywords or direct mentions of the target. The authors build P-Stance, an English Twitter corpus from the 2020 U.S. presidential race that covers three politicians: Donald Trump, Joe Biden and Bernie Sanders. They gathered about 2.8 million tweets through the Twitter streaming API using pro, anti and neutral seed hashtags. After length filtering and removing duplicates and retweets, they manually labeled 21,574 tweets. Among the models they test, fine-tuned BERTweet does best with 80.53% macro-F1. Uncertainty-aware Self-Training (UST) on a large unlabeled companion set, P-Stance-EXT, raises this further. The corpus also supports cross-target transfer and a new cross-topic setting, where a model trained on 2016 data about a politician is tested on 2020 tweets about the same person. Overall, it gives political opinion mining a bigger and harder benchmark with less reliance on surface cues.

Key points

  • P-Stance has 21,574 labeled tweets, more than three times the size of SemEval-2016 (4,870). Tweets were kept only if they had 10–128 words.
  • Annotator agreement (Krippendorff's alpha) averages 0.60 for the 3-class setup and 0.81 for the 2-class setup.
  • BERTweet scores 82.48 / 81.02 / 78.09 F1 on Trump, Biden and Sanders, beating BERT and other baselines.
  • UST-ALL reaches 82.42 macro-F1, 1.89 points above BERTweet. With only 100 labeled examples, UST is within 1.6 points of the full-data model on Biden.
  • In cross-topic tests (2016 → 2020), BERTweet scores 73.58 on Trump and 66.48 on Sanders. Dataset and code: github.com/chuchun8/PStance.

Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.

Fields: Linguistics & languages

Cited by

78 papers of our library cite it (of the 140673 whose bibliographies we read); 10 of them have a page here, 68 not yet.

Cite

The paper itself, not this page. BibTeX from the library's record of the work. DOI 10.18653/v1/2021.findings-acl.208

@inproceedings{li2021p,
  title = {{P-Stance: A Large Dataset for Stance Detection in Political Domain}},
  author = {Li, Yingjie and Sosea, Tiberiu and Sawant, Aditya and Nair, Ajith Jayaraman and Inkpen, Diana and Caragea, Cornelia},
  booktitle = {Findings of the Association for Computational Linguistics: ACL-IJCNLP 2021},
  year = {2021},
  doi = {10.18653/v1/2021.findings-acl.208},
  url = {https://doi.org/10.18653/v1/2021.findings-acl.208}
}

References

4 references, 2 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

  1. Abeer ALDayel and Walid Magdy. 2021. Stance detection on social media: State of the art and trends. Information Processing & Management, 58(4):102597. find
  2. Isabelle Augenstein, Tim Rocktäschel, Andreas Vlachos, and Kalina Bontcheva. 2016. Stance detection with bidirectional conditional encoding. In Proceedings of the 2016 Conference on Empirical Methods in Natural Language Processing, pages 876–885. find our card
  3. Roy Bar-Haim, Indrajit Bhattacharya, Francesco Din-uzzo, Amrita Saha, and Noam Slonim. 2017. Stance classification of context-dependent claims. In Proceedings of the 15th Conference of the European Chapter of the Association for Computational Linguistics: Volume 1, Long Papers, pages 251–261. find
  4. Costanza Conforti, Jakob Berndt, Mohammad Taher Pilehvar, Chryssi Giannitsarou, Flavio Toxvaerd, and Nigel Collier. 2020a. STANDER: An expert-annotated dataset for news stance detection and evidence retrieval. In Findings of the Association for Computational Linguistics: EMNLP 2020, pages 4086–4101. find our card

This page in Markdown → https://md.research.elamur.ai/free/papers/li-2021-p-stance-large-dataset-stance/