Elamur Research⌕
☰
home / papers / 2022 / Domain Adaptation for Arabic Crisis Response

Domain Adaptation for Arabic Crisis Response

Paper · 2022 · Reem Alrashdi and Simon O'Keefe

Summary

Classifiers that sort crisis tweets into useful and useless ones usually need human-labelled examples. During a new disaster those labels do not exist yet. Models trained on earlier events transfer poorly, because each crisis differs in place, time and how people react online. Arabic makes this harder: labelled crisis data is scarce, supporting resources are weak, and tweets are informal and dialectal.

The authors adapt a framework they first built for English. It labels tweets from the new event automatically, without any annotation:

  • Seed keywords come from K-means clusters of tweets from past events.
  • The keywords are ranked by a statistical score.
  • The list is expanded with synonyms from the Arabic dictionary Almaany.
  • Tweets are labelled by simple keyword rules.

These pseudo-labelled target tweets are combined with labelled source data to train the target model (DS-TM).

Tested on seven Arabic events from 2018–2020 (flood, explosion, virus, storm), the method beats self-training. It also helps when the keywords come from a different crisis type, which matters for low-resource languages.

Key points

  • Clustering and labelling the clusters. The number of K-means clusters is set with the silhouette score, because the elbow method gave unclear results. Each cluster gets a topic profile from the three tweets nearest its centre. Clusters about damage, casualties, or help and caution advice count as related and informative. Clusters about advertisements, political opinions or emotional support count as not informative.
  • Building the keyword list. Keywords are stemmed to roots with an ISRI stemmer. Stop words, hashtags, place names and Twitter tokens such as "RT" are removed with NLTK. A KW score, inspired by Chen et al. (2017), favours words frequent in related tweets and penalises words that also occur in unrelated ones.
  • Labelling rule. A tweet is marked positive if it contains a bigram of expanded keywords, and negative if it contains none of them.
  • Main gains in F1. DS-TM gained 3.7% absolute F1 on average when the keyword set matched the target's crisis type. It gained 5.5% on average with keyword sets from other crisis types. Unlike self-training (SelfL), it always improved over the baseline.
  • What drives transfer, and its limits. Transfer quality follows keyword overlap. The Dragon Storm data shares 6 top keywords and 55% of expanded keywords with the Floods set, but only 2 and 16% with the Explosion set. All adapted models remain far below the in-domain upper bound (LT).

Interesting ideas

  • I1. Label automatically only where evidence is strong on both sides: two keyword hits for positive, zero hits for negative. This keeps the pseudo-labels cleaner than a single-cue rule would.
  • I2. Self-training copies the source model's own mistakes into the new data. Labelling the target from an external resource avoids that feedback loop, which applies to any bootstrapping setup.
  • I3. Instead of annotating every item, a human labels whole clusters after reading the three examples nearest each centroid. This turns large unlabelled pools into usable signal at little cost.
  • I4. A small seed vocabulary can be widened with dictionary synonyms after reducing words to roots. This lets the system cover morphological and lexical variants it never saw in the seed data.
  • I5. Shared vocabulary between source and target explained when transfer worked, and a cue that signals relevance in one event can mislead in another. In the paper, terrorism words marked related tweets in the Explosion source but appeared in unrelated Dragon Storm tweets.

For Mova

How these ideas could serve Mova, our small open language model — its files, stages and tests:

  • I1 (learn): Silver trees could be accepted only when several witness parsers agree strongly, with clear disagreements kept as negatives and the ambiguous middle left out.
  • I2 (learn): The labels for retraining a parser should come from witnesses or teachers that are independent of the model being trained, not from its own earlier outputs.
  • I3 (learn): Parser errors or failed frame extractions could be clustered, and each cluster inspected through its three central examples to write one self-induced repair rule per cluster.
  • I4 (world): Frame-trigger lists could be expanded from lemmas through WordNet synonyms, so new events are recognised even when the wording differs from the seeds.
  • I5 (learn): Before evaluating, the lexical and construction overlap between training treebanks and a test domain could be measured to predict where accuracy will drop, with deliberately mismatched domains used as negative controls.

Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.

 Institutions

🎓University of YorkYork · United Kingdom18 works · 6,570 citations🎓University of Ha'ilHa'il · Saudi Arabia2 works · 3 citations

 Fields, topics and concepts

📚 Shelves 4The field of science this work belongs to (OpenAlex fields).Computer Science · 18,614 worksArtificial Intelligence · 17,965 worksSocial Sciences · 1,954 worksCommunication · 104 works

🧭 Topics 1Research subjects and languages the work is about.Semitic languages · 3,066 works

Cite

Cite as Reem Alrashdi and Simon O'Keefe. 2022. Domain Adaptation for Arabic Crisis Response. Proceedings of the Seventh Arabic Natural Language Processing Workshop (WANLP).

The paper itself, not this page. BibTeX from the library's record of the work. DOI 10.18653/v1/2022.wanlp-1.23

@inproceedings{alrashdi2022domain,
  title = {{Domain Adaptation for Arabic Crisis Response}},
  author = {Reem Alrashdi and Simon O'Keefe},
  booktitle = {Proceedings of the Seventh Arabic Natural Language Processing Workshop (WANLP)},
  year = {2022},
  doi = {10.18653/v1/2022.wanlp-1.23},
  url = {https://doi.org/10.18653/v1/2022.wanlp-1.23}
}

References

32 references, 5 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

  1. Diab Abuaiadah, Dileep Rajendran, and Mustafa Jarrar. 2017. Clustering arabic tweets for sentiment analysis. In 2017 IEEE/ACS 14th International Conference on Computer Systems and Applications (AICCSA), pages 449–456. IEEE. find
  2. Ghadah Adel and Yuping Wang. 2020. Detecting and classifying humanitarian crisis in arabic tweets. In 2020 3rd International Conference on Artificial Intelligence and Big Data (ICAIBD), pages 269–274. IEEE. find
  3. Lamia Al-Horaibi, M Badruddin Khan, and L Al-Horaibi Muhammad Badruddin Khan. 2017. Sentiment analysis of arabic tweets using semantic resources. Int. J. Comput. Inf. Sci, 13(1). find
  4. Rawan N Al-Matham and Hend S Al-Khalifa. 2021. Synoextractor: a novel pipeline for arabic synonym extraction using word2vec word embeddings. ↗
  5. Firoj Alam, Shafiq Joty, and Muhammad Imran. 2018. Domain adaptation with adversarial training and graph embeddings. arXiv preprint arXiv:1805.05151. ↗
  6. Alaa Alharbi and Mark Lee. 2019. Crisis detection from arabic tweets. In Proceedings of the 3rd workshop on arabic corpus linguistics, pages 72–79. dialects. In 11th Conference of the European Chapter of the Association for Computational Linguistics, pages 369–376.
  7. Kenneth Church and Patrick Hanks. 1990. Word association norms, mutual information, and lexicography. Computational linguistics, 16(1):22–29. ↗
  8. EDDY MUNTINA DHARMA, FORD LUMBAN GAOL, HARCO LESLIE HENDRIC SPITS WARNARS, and BENFANO SOEWITO. 2022. The accuracy comparison among word2vec, glove, and fasttext towards convolution neural network (cnn) text classification. Journal of Theoretical and Applied Information Technology, 100(2). find
  9. Yasmeen George, Shanika Karunasekera, Aaron Harwood, and Kwan Hui Lim. 2021. Real-time spatiotemporal event detection on geotagged social media. Journal of Big Data, 8(1):1–28. find
  10. Maria Habib, Mohammad Faris, Alaa Alomari, and Hossam Faris. 2021. Altibbivec: A word embedding model for medical and health applications in the arabic language. IEEE Access, 9:133875–133888. ↗
  11. Karen Sparck Jones. 1972. A statistical interpretation of term specificity and its application in retrieval. Journal of documentation. ↗
  12. James Lane. 2019. The 10 most spoken languages in the world. Babbel Magazine, 6. find
  13. Alaa Alharbi and Mark Lee. 2021. Kawarith: an arabic twitter corpus for crisis events. In Proceedings of the Sixth Arabic Natural Language Processing Workshop, pages 42–52.
  14. Hongmin Li. 2021. Domain adaptation approaches for classifying social media crisis data. Kansas State University. find
  15. Alaa Alharbi and Mark Lee. 2022. Classifying arabic crisis tweets using data selection and pre-trained language models. In Proceedinsg of the 5th Workshop on Open-Source Arabic Corpora and Processing Tools with Shared Tasks on Qur’an QA and Fine-Grained Hate Speech Detection, pages 71–78.
  16. Reem Alrashdi and Simon O’Keefe. 2020. Automatic labeling of tweets for crisis response using distant supervision. In Companion Proceedings of the Web Conference 2020, pages 418–425. find
  17. Reem ALRashdi and Simon O’Keefe. 2019. Robust domain adaptation approach for tweet classification for crisis response. In International Conference Europe Middle East & North Africa Information Systems and Technologies to Support Learning, pages 124–134. Springer. find
  18. Yubo Chen, Shulin Liu, Xiang Zhang, Kang Liu, and Jun Zhao. 2017. Automatically labeled data generation for large scale event extraction. In Proceedings of the 55th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 409–419.
  19. David Chiang, Mona Diab, Nizar Habash, Owen Rambow, and Safiullah Shareef. 2006. Parsing arabic Hongmin Li, Doina Caragea, and Cornelia Caragea. 2021. Combining self-training with deep learning for disaster tweet classification. In The 18th International Conference on Information Systems for Crisis Response and Management (ISCRAM 2021). find
  20. Hongmin Li, Doina Caragea, Cornelia Caragea, and Nic Herndon. 2018a. Disaster response aided by tweet classification with a domain adaptation approach. Journal of Contingencies and Crisis Management, 26(1):16–27. ↗
  21. Hongmin Li, Oleksandra Sopova, Doina Caragea, and Cornelia Caragea. 2018b. Domain adaptation for crisis data using correlation alignment and self-training. International Journal of Information Systems for Crisis Response and Management (IJISCRAM), 10(4):1– 20. ↗
  22. Dat Tien Nguyen, Kamela Ali Al Mannai, Shafiq Joty, Hassan Sajjad, Muhammad Imran, and Prasenjit Mitra. 2017. Robust classification of crisis-related data on social networks using convolutional neural networks. In Eleventh international AAAI conference on web and social media. find
  23. Radwa MK Saeed, Sherine Rady, and Tarek F Gharib. 2022. An ensemble approach for spam detection in arabic opinion texts. Journal of King Saud University-Computer and Information Sciences, 34(1):1407–1416. find
  24. Takeshi Sakaki, Makoto Okazaki, and Yutaka Matsuo. 2010. Earthquake shakes twitter users: real-time event detection by social sensors. In Proceedings of the 19th international conference on World wide web, pages 851–860. find
  25. Arun Kumar Sangaiah, Ahmed E Fakhry, Mohamed Abdel-Basset, and Ibrahim El-henawy. 2019. Arabic text clustering using improved clustering algorithms with dimensionality reduction. Cluster Computing, 22(2):4535–4549. ↗
  26. Muhammed Ali Sit, Caglar Koylu, and Ibrahim Demir. 2019. Identifying disaster-related tweets and their semantic, spatial and temporal context using deep learning, natural language processing and spatial analysis: a case study of hurricane irma. International Journal of Digital Earth. ↗
  27. Sakirin Tam, Rachid Ben Said, and Ö Özgür Tanriöver. 2021. A convbilstm deep learning model-based approach for twitter sentiment classification. IEEE Access, 9:41283–41293. ↗
  28. Ibtissam Touahri and Azzeddine Mazroui. 2021. Deep analysis of an arabic sentiment classification system based on lexical resource expansion and custom approaches building. International Journal of Speech Technology, 24(1):109–126. ↗
  29. Zeynep Tufekci and Christopher Wilson. 2012. Social media and the decision to participate in political protest: Observations from tahrir square. Journal of communication, 62(2):363–379. ↗
  30. Sarah Elizabeth Vieweg. 2012. Situational awareness in mass emergency: A behavioral and linguistic analysis of microblogged communications. Ph.D. thesis, University of Colorado at Boulder. find
  31. Si Si Mar Win and Than Nwe Aung. 2018. Automated text annotation for social media data during natural disasters. Ph.D. thesis, MERAL Portal. ↗
  32. Ying Zeng, Yansong Feng, Rong Ma, Zheng Wang, Rui Yan, Chongde Shi, and Dongyan Zhao. 2018. Scale up event extraction learning via automatic training data generation. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 32. find

Updated

↑ back to top

This page in Markdown → https://md.research.elamur.ai/free/en/papers/alrashdi-2022-domain-adaptation-arabic-crisis-response/