Elamur Research⌕
☰
home / papers / 2019 / Assessing Arabic Weblog Credibility via Deep Co-learning

Assessing Arabic Weblog Credibility via Deep Co-learning

Paper · 2019 · Chadi Helwe, Shady Elbassuoni, Ayman Al Zaatari et al.

Summary

The paper tackles automatic credibility assessment of Arabic blog posts when labeled data is very scarce. The only public resource is the crowdsourced corpus of Al Zaatari et al. (2016), with 268 posts. Each post is marked credible, fairly credible or not credible, and the posts cover topics such as regional wars and elections.

To work around this, the authors adapt Blum and Mitchell's co-training to neural networks and call the result deep co-learning. Two CNN classifiers look at the text differently: one uses CBOW word embeddings and the other uses character-level embeddings. Both start from the small labeled set. Each round, each model labels a random sample of unlabeled posts, and its most confident predictions are added to the other model's training data. A round's changes are kept only if an averaged ensemble of the two models scores higher F1 on a validation set.

On the test split, co-learning reaches F1 0.63. It beats a TF-IDF SVM, the single word and char CNNs, and their plain ensemble. This shows that semi-supervision can make deep models usable even with only a few hundred labeled examples.

Key points

  • Data: the Al Zaatari et al. (2016) Arabic blog credibility corpus (268 posts, crowdsourced three-way labels), plus a collected pool of unlabeled blog posts.
  • Models: a Word-CNN on CBOW embeddings and a Char-CNN on character embeddings. Both use dropout 0.2 and batch normalization between hidden layers, and are trained for 500 epochs with batch size 16 using Adagrad.
  • Co-learning setup: 50 iterations, each taking about 24 hours of retraining. Each round samples 1000 unlabeled items, and the top-50 scored items go to the opposite model.
  • Test F1 scores:
    • SVM TF-IDF (C=5): 0.57
    • Word-CNN: 0.52
    • Char-CNN: 0.54
    • Ensemble CNN: 0.50
    • Deep co-learning: 0.63
  • With so little data, the simple SVM beat every fully supervised deep model. Naively ensembling the two CNNs made results worse, not better.

Interesting ideas

  • I1. Two learners that see the data through different views (word tokens vs. characters) label unlabeled text for each other, so each one's errors are less likely to be repeated by its partner.
  • I2. Each augmentation step is a tentative proposal: machine-labeled additions are committed only if a held-out score improves, and otherwise discarded.
  • I3. Only a small top-k slice of the most confident automatic labels is transferred per round, rather than the whole sample. This keeps label noise low while the pool grows gradually.
  • I4. The evaluation includes a plain TF-IDF SVM and a naive ensemble, and these exposed that small-data deep models and simple averaging can underperform.
  • I5. Character-level input is used because it tolerates misspellings, emoticons and subword patterns that word embeddings miss.

For Mova

How these ideas could serve Mova, our small open language model — its files, stages and tests:

  • I1 (learn): Witness parsers with genuinely different inputs (the perceptron parser vs. gparse) could cross-supply silver trees, with each parser retrained only on the other's most confident parses.
  • I2 (learn): Self-induced repair rules or new silver batches could be accepted only when dev LAS or answer accuracy rises, giving a built-in guard against drift.
  • I3 (learn): Silver data agreed by witnesses could be admitted in small, confidence-ranked increments instead of in bulk.
  • I4 (learn): Every new learned component should be compared against a simple baseline and a trivial combination, as a negative control, before its gains are believed.
  • I5 (tokens): A character-based feature channel in the tagger or lemmatizer could handle noisy or out-of-vocabulary web tokens that a word lexicon misses.

Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.

 Fields, topics and concepts

📚 Shelves 5The field of science this work belongs to (OpenAlex fields).Computer Science · 16,667 worksArtificial Intelligence · 16,194 worksSocial Sciences · 1,772 worksInformation Systems · 1,437 worksSociology and Political Science · 577 works

🧭 Topics 1Research subjects and languages the work is about.Semitic languages · 2,908 works

Cite

Cite as Chadi Helwe, Shady Elbassuoni, Ayman Al Zaatari and Wassim El-Hajj. 2019. Assessing Arabic Weblog Credibility via Deep Co-learning. Proceedings of the Fourth Arabic Natural Language Processing Workshop.

The paper itself, not this page. BibTeX from the library's record of the work. DOI 10.18653/v1/w19-4614

@inproceedings{helwe2019assessing,
  title = {{Assessing Arabic Weblog Credibility via Deep Co-learning}},
  author = {Chadi Helwe, Shady Elbassuoni, Ayman {Al Zaatari} and Wassim El-Hajj},
  booktitle = {Proceedings of the Fourth Arabic Natural Language Processing Workshop},
  year = {2019},
  doi = {10.18653/v1/w19-4614},
  url = {https://doi.org/10.18653/v1/w19-4614}
}

References

18 references, 4 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

  1. Ayman Al Zaatari, Rim El Ballouli, Shady Elbassuoni, Wassim El-Hajj, Hazem M. Hajj, Khaled B Shaban, Nizar Habash, and Emad Yahya. 2016. Arabic corpora for credibility analysis. In LREC.
  2. Avrim Blum and Tom Mitchell. 1998. Combining labeled and unlabeled data with co-training. In Proceedings of the eleventh annual conference on Computational learning theory, pages 92–100. ACM. ↗
  3. Carlos Castillo, Marcelo Mendoza, and Barbara Poblete. 2011. Information credibility on twitter. In Proceedings of the 20th international conference on World wide web, pages 675–684. ACM. find
  4. John Duchi, Elad Hazan, and Yoram Singer. 2011. Adaptive subgradient methods for online learning and stochastic optimization. Journal of Machine Learning Research, 12(Jul):2121–2159. find
  5. Rim El Ballouli, Wassim El-Hajj, Ahmad Ghandour, Shady Elbassuoni, Hazem Hajj, and Khaled Shaban. 2017. Cat: Credibility analysis of arabic content on twitter. In Proceedings of the Third Arabic Natural Language Processing Workshop, pages 62–71. Knowledge Discovery & Data Mining, pages 849– 857. ACM.
  6. Aditi Gupta and Ponnurangam Kumaraguru. 2012. Credibility ranking of tweets during high impact events. In Proceedings of the 1st workshop on privacy and security in online social media, page 2. ACM. find
  7. Jiawei Zhang, Limeng Cui, Yanjie Fu, and Fisher B Gouza. 2018. Fake news detection with deep diffusive network model. arXiv preprint arXiv:1805.08751. ↗
  8. Aditi Gupta, Ponnurangam Kumaraguru, Carlos Castillo, and Patrick Meier. 2014. Tweetcred: Real-time credibility assessment of content on twitter. In International Conference on Social Informatics, pages 228–243. Springer. find
  9. Pranam Kolari, Tim Finin, Anupam Joshi, et al. 2006a. Svms for the blogosphere: Blog identification and splog detection. In AAAI spring symposium on computational approaches to analysing weblogs. find
  10. Pranam Kolari, Akshay Java, Tim Finin, Tim Oates, Anupam Joshi, et al. 2006b. Detecting spam blogs: A machine learning approach. In Proceedings of the national conference on artificial intelligence, volume 21, page 1351. ↗
  11. Menlo Park, CA; Cambridge, MA; London; AAAI Press; MIT Press; 1999. find
  12. Yu-Ru Lin, Hari Sundaram, Yun Chi, Junichi Tatemura, and Belle L Tseng. 2007. Splog detection using self-similarity analysis on blog temporal dynamics. In Proceedings of the 3rd international workshop on Adversarial information retrieval on the web, pages 1–8. ACM. find
  13. Jing Ma, Wei Gao, Prasenjit Mitra, Sejeong Kwon, Bernard J Jansen, Kam-Fai Wong, and Meeyoung Cha. 2016. Detecting rumors from microblogs with recurrent neural networks. In IJCAI, pages 3818– 3824. find
  14. Victoria Rubin, Niall Conroy, Yimin Chen, and Sarah Cornwell. 2016. Fake news or truth? using satirical cues to detect potentially misleading news. In Proceedings of the Second Workshop on Computational Approaches to Deception Detection, pages 7–17. find
  15. Natali Ruchansky, Sungyong Seo, and Yan Liu. 2017. Csi: A hybrid deep model for fake news detection. In Proceedings of the 2017 ACM on Conference on Information and Knowledge Management, pages 797–806. ACM. ↗
  16. Franco Salvetti and Nicolas Nicolov. 2006. Weblog classification for fast splog filtering: A url language model segmentation approach. In Proceedings of the Human Language Technology Conference of the NAACL, Companion Volume: Short Papers, pages 137–140. Association for Computational Linguistics. find
  17. William Yang Wang. 2017. ” liar, liar pants on fire”: A new benchmark dataset for fake news detection. arXiv preprint arXiv:1705.00648. ↗
  18. Yaqing Wang, Fenglong Ma, Zhiwei Jin, Ye Yuan, Guangxu Xun, Kishlay Jha, Lu Su, and Jing Gao. 2018. Eann: Event adversarial neural networks for multi-modal fake news detection. In Proceedings of the 24th ACM SIGKDD International Conference on find

Updated

↑ back to top

This page in Markdown → https://md.research.elamur.ai/free/en/papers/helwe-2019-assessing-arabic-weblog-credibility-deep/