Elamur Research⌕
☰
home / papers / 2022 / ExaASC: A General Target-Based Stance Detection Corpus in Arabic Language

ExaASC: A General Target-Based Stance Detection Corpus in Arabic Language

Summary

The paper addresses the lack of public Arabic data for target-based stance detection, which asks whether a text supports or opposes a given target. Most earlier work fixes a small list of targets in advance. The authors instead frame the task around Twitter threads. For each argumentative source tweet, an annotator picks its single most debated target, chosen by what the replies actually discuss. Each reply is then labeled Favor, Against, or None toward that target. The result is ExaASC, an Arabic corpus covering many politicians, events, and some sports figures. On ExaASC they fine-tune an Arabic BERT model in several configurations. Instead of one classifier per target, a single model handles every target. The best variant reaches a macro F-score of 70.69, averaged over the Favor and Against classes. The authors present this as evidence that a general, open-target stance system is feasible. The resource is also notable because Arabic is low-resource for stance detection, and the corpus is publicly released.

Key points

  • About 80,000 source tweets were collected from the Twitter stream API and filtered down to controversial ones. Roughly 17,500 replies were tagged, and only 9,566 items were kept: those labeled by at least two annotators who agreed.
  • The training set has 6,826 samples covering more than 180 unique targets drawn from 360 source tweets. The validation set is 10% of training data and shares no source tweets with it. The test set has 2,740 samples spanning 20 of those targets.
  • Labels follow the SemEval-2016 guidelines with one change. None now covers both neutral replies and replies irrelevant to the target, for example spam.
  • Annotators were selected with a test of 100 samples verified by at least three experts. Further calibration sessions corrected their mistakes.
  • Six BERT variants were compared: sequence output, average pooling, and LSTM on top, each with and without the target as input. BERT-seq-target scored best (F avg 70.69), and adding the target helped every architecture.

Interesting ideas

  • I1. The stance target is not drawn from a fixed list; it is read off the conversation, as the entity in the source post that the replies argue about. This makes stance a relation that depends on discourse context. — For Mova (frames): when analysing replies or dialogue turns, link evaluative statements to the main entity of the preceding turn via coreference. This yields explicit (holder, target, polarity) frames rather than topic-specific classifiers.
  • I2. Annotators are first screened on a small expert-verified gold set, and items with a single label or with disagreement are discarded. Quality is bought by trading volume for consensus. — For Mova (learn): qualify LLM teacher tiers or witness parsers on a gold sample before trusting them, then keep only silver items on which several witnesses agree.
  • I3. The validation split shares no source tweets with training, so the model cannot score well by memorising thread-specific context. — For Mova (learn): split evaluation by document, such as a whole fable or test paper, not by sentence, so that scores reflect generalisation rather than leakage.
  • I4. One model serves all targets, and the target itself is fed as input, which consistently beat target-blind variants. Conditioning on the target is what lets a single system generalise. — For Mova (answer): build question answering about attitudes ("does X approve of Y?") as one target-conditioned reader over the world model, not one rule set per topic.
  • I5. The None class merges neutral and off-topic replies, and the headline score averages only Favor and Against. This keeps the metric focused on the classes that matter. — For Mova (answer): when testing stance or sentiment readers, report F on the polar classes separately so a large null class cannot inflate the score.

Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.

Fields: Linguistics & languages

Cited by

8 papers of our library cite it (of the 140884 whose bibliographies we read); 1 of them have a page here, 7 not yet.

Cite

The paper itself, not this page. BibTeX from the library's record of the work. DOI 10.1109/iccke54056.2021.9721486 · arXiv 2204.13979

@misc{jaziriyan2022exaasc,
  title = {{ExaASC: A General Target-Based Stance Detection Corpus in Arabic Language}},
  author = {Jaziriyan, Mohammad Mehdi and Akbari, Ahmad and Karbasi, Hamed},
  year = {2022},
  doi = {10.1109/iccke54056.2021.9721486},
  eprint = {2204.13979},
  archivePrefix = {arXiv},
  url = {https://doi.org/10.1109/iccke54056.2021.9721486}
}

References

20 references, 8 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

  1. S. Mohammad, S. Kiritchenko, P. Sobhani, X. Zhu, and C. Cherry, “SemEval-2016 task 6: Detecting stance in tweets,” in Proceedings of the 10th International Workshop on Semantic Evaluation (SemEval-2016), San Diego, California, Jun. 2016, pp. 31–41. find our card
  2. L. Derczynski, K. Bontcheva, M. Liakata, R. Procter, G. Wong Sak Hoi, and A. Zubiaga, “SemEval-2017 task 8: RumourEval: Determining rumour veracity and support for rumours,” in Proceedings of the 11th International Workshop on Semantic Evaluation (SemEval-2017), Vancouver, Canada, Aug. 2017, pp. 69–76. find our card
  3. J. Devlin, M.-W. Chang, K. Lee, and K. Toutanova, “BERT: Pre-training of deep bidirectional transformers for language understanding,” in Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics, Minneapolis, Minnesota, Jun. 2019, pp. 4171–4186. find
  4. A. Radford, K. Narasimhan, T. Salimans, and I. Sutskever, “Improving language understanding by generative pre-training,” 2018. find
  5. M. E. Peters, M. Neumann, M. Iyyer, M. Gardner, C. Clark, K. Lee, and L. Zettlemoyer, “Deep contextualized word representations,” in Proceedings of the 2018 Conference of the North American Chapter of the Association for Computational Linguistics, New Orleans, Louisiana, Jun. 2018, pp. 2227–2237. find
  6. S. Mohammad, S. Kiritchenko, P. Sobhani, X. Zhu, and C. Cherry, “A dataset for detecting stance in tweets,” in Proceedings of the Tenth International Conference on Language Resources and Evaluation (LREC’16). Slovenia: Portorož, 2016. find
  7. G. Zarrella and A. Marsh, “MITRE at SemEval-2016 task 6: Transfer learning for stance detection,” in Proceedings of the 10th International Workshop on Semantic Evaluation (SemEval-2016), San Diego, California, Jun. 2016, pp. 458–463. find our card
  8. W. Wei, X. Zhang, X. Liu, W. Chen, and T. Wang, “pkudblab at SemEval-2016 task 6 : A specific convolutional neural network system for effective stance detection,” in Proceedings of the 10th International Workshop on Semantic Evaluation (SemEval-2016), San Diego, California, Jun. 2016, pp. 384–388. find our card
  9. P. Sobhani, S. Mohammad, and S. Kiritchenko, “Detecting stance in tweets and analyzing its interaction with sentiment,” in *SEMth Joint Conference on Lexical and Computational Semantics. pp. 159-169: Proceedings, 2016, pp. 2016–5. find
  10. S. M. Mohammad, P. Sobhani, and S. Kiritchenko, “Stance and sentiment in tweets,” ACM Transactions on Internet Technology, vol. 17, no. 3, p. 6, 2017. find
  11. P. Sobhani, D. Inkpen, and X. Zhu, “A dataset for multi-target stance detection,” in Proceedings of the 15th Conference of the European Chapter of the Association for Computational Linguistics: Volume 2, Short Papers, Valencia, Spain, Apr. 2017, pp. 551–557. 6 www.exalab.co find our card
  12. P. Krejzl, B. Hourová, and J. Steinberger, “Stance detection in online discussions,” vol. 1, 2017. find
  13. G. Gorrell, E. Kochkina, M. Liakata, A. Aker, A. Zubiaga, K. Bontcheva, and L. Derczynski, “SemEval-2019 task 7: RumourEval, determining rumour veracity and support for rumours,” in Proceedings of the 13th International Workshop on Semantic Evaluation, Minneapolis, Minnesota, USA, Jun. 2019, pp. 845–854. find our card
  14. M. Lai, A. T. Cignarella, D. I. H. Farías, C. Bosco, V. Patti, and P. Rosso, “Multilingual stance detection in social media political debates,” Computer Speech & Language, vol. 63, p. 101075, 2020. find
  15. Y. Li and C. Caragea, “Multi-task stance detection with sentiment and stance lexicons,” in on Empirical Methods in Natural Language Processing and 9th International Joint Conference on Natural Language Processing, Proceedings of the Conference, pp, E. ijcnlp Conference, Ed. 6299-6305, 2019, pp. 2019–2019. find our card
  16. Z. Wang, Q. Sun, S. Li, Q. Zhu, and G. Zhou, “Neural stance detection with hierarchical linguistic representations,” IEEE/ACM Transactions on Audio Speech and Language Processing, vol. 28, pp. 635–645, 2020. find
  17. K. Darwish, W. Magdy, and T. Zanouda, “Improved stance prediction in a user similarity feature space,” in Proceedings of the 2017 IEEE/ACM International Conference on Advances in Social Networks Analysis and Mining, 7, 2017, pp. 145–148. find
  18. S. Ghosh, P. Singhania, S. Singh, K. Rudra, and S. Ghosh, “Stance detection in web and social media: A comparative study,” in Experimental IR Meets Multilinguality, Multimodality, and Interaction. Cham: Springer International Publishing, 2019, pp. 75–87. find
  19. A. Abdelali, S. Hassan, H. Mubarak, K. Darwish, and Y. Samih, “Pre-training bert on arabic tweets: Practical considerations,” vol. 2, 2021. find our card
  20. D. P. Kingma and J. Ba, “Adam: A method for stochastic optimization,” in 3rd International Conference on Learning Representations, ICLR 2015, , CA, USA, May, 2015, Conference Track Proceedings, 2015, pp. 7–9. find

↑ back to top

This page in Markdown → https://md.research.elamur.ai/free/en/papers/jaziriyan-2022-exaasc-general-target-based-stance/