Elamur Research⌕
☰
home / papers / 2022 / Pythoneers at WANLP 2022 Shared Task: Monolingual AraBERT for Arabic Propaganda Detection and Span Extraction

Pythoneers at WANLP 2022 Shared Task: Monolingual AraBERT for Arabic Propaganda Detection and Span Extraction

Paper · 2022 · Joseph Attieh and Fadi Hassan

Summary

The authors address propaganda detection in Arabic tweets for the WANLP 2022 shared task, which had two sub-tasks. The first asks which persuasion techniques, out of roughly twenty fallacy-based categories, appear in a tweet. The second asks where in the text each technique occurs. The organizers released only about 500 training tweets, and many tweets carry several techniques at once. Both systems build on bert-large-arabertv02-twitter, a large AraBERT model given extra pretraining on dialectal tweets.

For sub-task 1, a single shared encoder feeds a separate yes/no classifier for every technique, so the multi-label problem becomes a set of jointly trained binary tasks. Rare classes are handled by balanced oversampling. For sub-task 2, AraBERT is topped with a CRF layer that tags tokens in BIO format.

The systems placed 3rd on sub-task 1 (micro-F1 0.602) and 1st on sub-task 2 (micro-F1 0.396). This shows that careful fine-tuning of a domain-matched monolingual encoder can compete strongly even with very little data and no augmentation.

Key points

  • The training data has 504 tweets carrying 880 labels, plus two dev splits (dev, dev_test) of about 50 tweets each. Both sub-tasks use the same tweets, and over half the tweets contain more than one technique or none.
  • Nine techniques, led by Loaded Language and Name calling/Labeling, account for about 85% of training labels. Every other technique has fewer than 10 examples, and Straw Man is missing from all splits.
  • System 1 settings:
    • Each head is a dropout layer (0.1) followed by a linear layer over the pooled AraBERT output.
    • Each batch optimizes a single head's cross-entropy loss.
    • Adam is used with learning rate 1e-5.
    • Batch size is 8 for two epochs, then 1 for two more.
    • Each head gets 2000 positive and 2000 negative tweets, sampled with replacement.
  • System 2 settings: AraBERT + CRF trained with negative log-likelihood, batch size 32, 13 epochs. In preliminary tests, BIO beat the IO and BIOUL tagging schemes.
  • Preprocessing for sub-task 1 strips non-Arabic words, emojis and URLs before applying the AraBERT tokenizer.

Interesting ideas

  • I1. Splitting a multi-label problem into one independent binary head per label lets each label fire on its own, while the shared encoder still captures how labels tend to co-occur.
  • I2. Training only one head's loss per batch keeps the gradients of unrelated objectives from mixing inside a single update.
  • I3. Oversampling with replacement to an equal number of positives and negatives for each head gives rare labels the same training weight as frequent ones.
  • I4. The authors compared several span encodings (IO, BIO, BIOUL) before choosing one, treating the encoding itself as a tunable design decision.
  • I5. The authors explicitly point out a label that never appears in any split, a reminder that some reported per-label scores carry no real evidence.

For Mova

How these ideas could serve Mova, our small open language model — its files, stages and tests:

  • I1 (frames): Speech-act, irony and implicature cues could be detected by separate binary heads over a shared sentence representation, so a clause can carry several pragmatic labels without them competing.
  • I2 (learn): When several auxiliary objectives share one encoder, rotating them batch by batch is a simple way to stop one loss from swamping the others.
  • I3 (learn): Silver data for rare frames or verb classes could be rebalanced per detector in the same way before training, so infrequent categories are not drowned out.
  • I4 (frames): For tagging semantic-role argument spans, the span scheme should be benchmarked rather than assumed, since it measurably affects how well a CRF-style tagger works.
  • I5 (learn): Evaluation reports could flag every category with zero or near-zero support, so that apparent perfect scores on empty classes are not taken as real performance.

Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.

 Institutions

🎓Aalto UniversityEspoo · Finland13 works · 648 citations

 Fields, topics and concepts

📚 Shelves 6The field of science this work belongs to (OpenAlex fields).Computer Science · 16,667 worksArtificial Intelligence · 16,194 worksArts and Humanities · 16,039 worksLanguage and Linguistics · 15,891 worksSocial Sciences · 1,772 worksSociology and Political Science · 577 works

🧭 Topics 1Research subjects and languages the work is about.Semitic languages · 2,908 works

Cite

Cite as Joseph Attieh and Fadi Hassan. 2022. Pythoneers at WANLP 2022 Shared Task: Monolingual AraBERT for Arabic Propaganda Detection and Span Extraction. Proceedings of the Seventh Arabic Natural Language Processing Workshop (WANLP).

The paper itself, not this page. BibTeX from the library's record of the work. DOI 10.18653/v1/2022.wanlp-1.64

@inproceedings{attieh2022pythoneers,
  title = {{Pythoneers at WANLP 2022 Shared Task: Monolingual AraBERT for Arabic Propaganda Detection and Span Extraction}},
  author = {Joseph Attieh and Fadi Hassan},
  booktitle = {Proceedings of the Seventh Arabic Natural Language Processing Workshop (WANLP)},
  year = {2022},
  doi = {10.18653/v1/2022.wanlp-1.64},
  url = {https://doi.org/10.18653/v1/2022.wanlp-1.64}
}

References

25 references, 8 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

  1. Firoj Alam, Hamdy Mubarak, Wajdi Zaghouani, Preslav Nakov, and Giovanni Da San Martino. 2022. Overview of the WANLP 2022 shared task on propaganda detection in Arabic. In Proceedings of the Seventh Arabic Natural Language Processing Workshop, Abu Dhabi, UAE. Association for Computational Linguistics.
  2. Wissam Antoun, Fady Baly, and Hazem Hajj. 2020. Arabert: Transformer-based model for arabic language understanding. In LREC 2020 Workshop Language Resources and Evaluation Conference 11–16 May 2020, page 9. ↗
  3. Pepa Atanasova, Lluís Màrquez i Villodre, Alberto Barrón-Cedeño, T. Elsayed, Reem Suwaileh, Wajdi Zaghouani, Spas Kyuchukov, Giovanni Da San Martino, and Preslav Nakov. 2018. Overview of the clef-2018 checkthat! lab on automatic identification and verification of political claims. task 1: Check-worthiness. ArXiv, abs/1808.05542. ↗
  4. Pepa Atanasova, Preslav Nakov, Georgi Karadzhov, Mitra Mohtarami, and Giovanni Da San Martino. 2019. Overview of the clef-2019 checkthat! lab: Automatic identification and verification of claims. task 1: Check-worthiness. In CLEF. find
  5. Alberto Barrón-Cedeño, T. Elsayed, Reem Suwaileh, Lluís Màrquez i Villodre, Pepa Atanasova, Wajdi Zaghouani, Spas Kyuchukov, Giovanni Da San Martino, and Preslav Nakov. 2018. Overview of the clef-2018 checkthat! lab on automatic identification and verification of political claims. task 2: Factuality. In CLEF. find
  6. Giovanni Da San Martino, Alberto Barron-Cedeno, and Preslav Nakov. 2019a. Findings of the nlp4if-2019 shared task on fine-grained propaganda detection. In Proceedings of the second workshop on natural language processing for internet freedom: censorship, disinformation, and propaganda, pages 162–170. ↗
  7. Giovanni Da San Martino, Seunghak Yu, Alberto Barrón-Cedeno, Rostislav Petrov, and Preslav Nakov. 2019b. Fine-grained analysis of propaganda in news article. In Proceedings of the 2019 conference on empirical methods in natural language processing and the 9th international joint conference on natural language processing (EMNLP-IJCNLP), pages 5636–5646. find
  8. Giovanni Da San Martino, Seunghak Yu, Alberto Barrón-Cedeño, Rostislav Petrov, and Preslav Nakov. 2019c. Fine-grained analysis of propaganda in news article. In Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing (EMNLP-IJCNLP), pages 5636–5646, Hong Kong, China. Association for Computational Linguistics. find
  9. Jacob Devlin, Ming-Wei Chang, Kenton Lee, and Kristina Toutanova. 2019. BERT: Pre-training of deep bidirectional transformers for language understanding. In Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long and Short Papers), pages 4171–4186, Minneapolis, Minnesota. Association for Computational Linguistics. ↗
  10. Ivan Habernal and Iryna Gurevych. 2017. Argumentation Mining in User-Generated Web Discourse. Computational Linguistics, 43(1):125–179. ↗
  11. Maram Hasanain, Fatima Haouari, Reem Suwaileh, Zien Sheikh Ali, Bayan Hamdan, Tamer Elsayed, Alberto Barrón-Cedeño, Giovanni Da San Martino, and Preslav Nakov. 2020. Overview of checkthat! 2020i arabic: Automatic identification and verification of claims in social media. In CLEF. find
  12. Maram Hasanain, Reem Suwaileh, T. Elsayed, Alberto Barrón-Cedeño, and Preslav Nakov. 2019. Overview of the clef-2019 checkthat! lab: Automatic identification and verification of claims. task 2: Evidence and factuality. In CLEF. ↗
  13. Diederik P. Kingma and Jimmy Ba. 2015. Adam: A method for stochastic optimization. In 3rd International Conference on Learning Representations, ICLR 2015, San Diego, CA, USA, May 7-9, 2015, Conference Track Proceedings. ↗
  14. John D. Lafferty, Andrew McCallum, and Fernando C. N. Pereira. 2001. Conditional random fields: Probabilistic models for segmenting and labeling sequence data. In Proceedings of the Eighteenth International Conference on Machine Learning, ICML ’01, pages 282–289, San Francisco, CA, USA. Morgan Kaufmann Publishers Inc. find
  15. Giovanni Da San Martino, Stefano Cresci, Alberto Barrón-Cedeño, Seunghak Yu, Roberto Di Pietro, and Preslav Nakov. 2020. A survey on computational propaganda detection. arXiv preprint arXiv:2007.08024. ↗
  16. Clyde Raymond Miller. 1939. How to detect and analyze propaganda. Town Hall, Incorporated. find
  17. Preslav Nakov, Alberto Barrón-Cedeño, Giovanni Da San Martino, Firoj Alam, Rubén Míguez, Tommaso Caselli, Mucahid Kutlu, Wajdi Zaghouani, Chengkai Li, Shaden Shaar, Hamdy Mubarak, Alex Nikolov, and Yavuz Selim Kartal. 2022a. Overview of the clef-2022 checkthat! lab task 1 on identifying relevant claims in tweets. In CLEF. find
  18. Preslav Nakov, Giovanni Da San Martino, Firoj Alam, Shaden Shaar, Hamdy Mubarak, and Nikolay Babulkov. 2022b. Overview of the clef-2022 checkthat! lab task 2 on detecting previously fact-checked claims. In CLEF. find
  19. Preslav Nakov, Giovanni Da San Martino, Tamer Elsayed, Alberto Barrón-Cedeño, Rubén Míguez, Shaden Shaar, Firoj Alam, Fatima Haouari, Maram Hasanain, Watheq Mansour, Bayan Hamdan, Zien Sheikh Ali, Nikolay Babulkov, Alex Nikolov, Gautam Kishore Shahi, Julia Maria Struß, Thomas Mandl, Mucahid Kutlu, and Yavuz Selim Kartal. 2021. Overview of the clef-2021 checkthat! lab on detecting check-worthy claims, previously fact-checked claims, and fake news. In CLEF. find
  20. Hannah Rashkin, Eunsol Choi, Jin Yea Jang, Svitlana Volkova, and Yejin Choi. 2017. Truth of varying shades: Analyzing language in fake news and political fact-checking. In Proceedings of the 2017 Conference on Empirical Methods in Natural Language Processing, pages 2931–2937, Copenhagen, Denmark. Association for Computational Linguistics.
  21. Shaden Shaar, Fatima Haouari, Watheq Mansour, Maram Hasanain, Nikolay Babulkov, Firoj Alam, Giovanni Da San Martino, Tamer Elsayed, and Preslav Nakov. 2021a. Overview of the clef-2021 checkthat! lab task 2 on detecting previously fact-checked claims in tweets and political debates. In CLEF. ↗
  22. Shaden Shaar, Maram Hasanain, Bayan Hamdan, Zien Sheikh Ali, Fatima Haouari, Alex Nikolov, Mucahid Kutlu, Yavuz Selim Kartal, Firoj Alam, Giovanni Da San Martino, Alberto Barrón-Cedeño, Rubén Míguez, Javier Beltrán, Tamer Elsayed, and Preslav Nakov. 2021b. Overview of the clef-2021 checkthat! lab task 1 on check-worthiness estimation in tweets and political debates. In CLEF. find
  23. Shaden Shaar, Alex Nikolov, Nikolay Babulkov, Firoj Alam, Alberto Barrón-Cedeño, Tamer Elsayed, Maram Hasanain, Reem Suwaileh, Fatima Haouari, Giovanni Da San Martino, and Preslav Nakov. 2020. Overview of checkthat! 2020 english: Automatic identification and verification of claims in social media. In CLEF. find
  24. Bruce Lannes Smith. 2021. Propaganda. find
  25. Anthony Weston. 2018. A rulebook for arguments. Hackett Publishing. find

Updated

↑ back to top

This page in Markdown → https://md.research.elamur.ai/free/en/papers/attieh-2022-pythoneers-wanlp-2022-shared-task/