Elamur Research⌕
☰
home / papers / 2020 / FloDusTA: Saudi Tweets Dataset for Flood, Dust Storm, and Traffic Accident Events

FloDusTA: Saudi Tweets Dataset for Flood, Dust Storm, and Traffic Accident Events

Summary

Arabic event detection on Twitter has had few public datasets to train and test on. This paper addresses that gap with FloDusTA, a collection of Saudi tweets in Modern Standard Arabic and Saudi dialect about three hazards that matter locally: floods, dust storms and traffic accidents. The authors gathered about 3.6 million keyword-matched tweets over seven months of 2018. They removed non-Arabic posts, retweets and duplicates, then kept tweets tied to Saudi Arabia using location, hashtag, country and time-zone signals. Three native-speaker volunteers labelled a random sample of 9,000 tweets in two stages. One stage records the temporal status of the report and the other records the event type. Agreement measured with Fleiss kappa reached the "almost perfect" range. A baseline SVM with 10-fold cross-validation scored an F1 of 0.881 using TF-IDF features. The dataset matters because it is described as the first public Arabic resource for these event types, and dust storms had not been targeted in earlier work.

Key points

  • Raw collection from the Twitter Streaming API (March–September 2018) gave 3,644,838 tweets. After removing non-Arabic, retweet and duplicate tweets, 894,277 remained.
  • Saudi-location filtering left 85,894 flood, 95,024 dust storm and 39,270 traffic accident tweets. Tweets shorter than three words were also dropped.
  • The annotation sample had 9,000 tweets (3,000 per event). Each tweet received two labels:
    • a period label (PL): immediate, historical, warning or irrelevant;
    • an event label (EL): flood, dust storm or traffic accident.
  • The resulting dataset used four classes: one for each of the three hazard types plus a non-event class.
  • The SVM baseline with term-frequency features reached P/R/F1 of 0.866/0.857/0.858. With TF-IDF features it reached 0.887/0.881/0.881.
  • The data is released as FloDusTA on GitHub.

Interesting ideas

  • I1. The paper labels when or how real an event report is separately from what kind of event it is, distinguishing live reports, past recollections, forecasts/warnings and unrelated uses. This separation could suit any event-extraction task.
  • I2. An "irrelevant" class captures tweets where a hazard word is used figuratively, such as a "flood of goals". This makes explicit that a keyword match does not mean an event took place.
  • I3. No single location field was reliable, so the authors combined several weak, independent signals: free-text user location, hashtags, profile country and time zone.
  • I4. Very short items were removed before annotation because they confuse annotators. The paper also reports concrete disagreement cases, such as one tweet labelled flood by two annotators and dust storm by the third.
  • I5. The paper compares two simple feature representations, term frequency and TF-IDF, to give a clear lower bound for later systems.

For Mova

How these ideas could serve Mova, our small open language model — its files, stages and tests:

  • I1 (world): the world of a text could tag every event with a reality/time status (happening now, already past, predicted or warned, not real) so that a warning is never treated as a fact.
  • I2 (frames): figurative uses of event nouns could be flagged as metaphor in prag so that no flood frame is instantiated, and such sentences make good test items.
  • I3 (learn): this mirrors accepting silver data only when several witness parsers agree, and treating each signal as one witness with its own noise.
  • I4 (learn): silver or teacher-labelled data could be filtered by length and ambiguity, and disagreement cases kept as a curated set for testing repair rules.
  • I5 (learn): a TF-IDF classifier could serve as a negative-control baseline, showing how much the explainable tree→frames pipeline adds over bag-of-words on event-type questions.

Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.

Fields: Linguistics & languages

Topics Semitic languages

 Institutions

🎓Umm al-Qura UniversityMakkah al Mukarramah · Saudi Arabia7 works · 87 citations

Cite

Cite as Btool Hamoui, Mourad Mars and Khaled Almotairi. 2020. FloDusTA: Saudi Tweets Dataset for Flood, Dust Storm, and Traffic Accident Events. Proceedings of the Twelfth Language Resources and Evaluation Conference.

The paper itself, not this page. BibTeX from the library's record of the work.

@inproceedings{hamoui2020flodusta,
  title = {{FloDusTA: Saudi Tweets Dataset for Flood, Dust Storm, and Traffic Accident Events}},
  author = {Btool Hamoui, Mourad Mars and Khaled Almotairi},
  booktitle = {Proceedings of the Twelfth Language Resources and Evaluation Conference},
  year = {2020},
  url = {https://aclanthology.org/2020.lrec-1.174/}
}

References

16 references, 0 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

  1. Arthur, R., Boulton, C. A., Shotton, H., and Williams, H. T. (2018). Social sensing of floods in the UK. PloS one, 13(1), e0189327. find
  2. Albugami, S., Palmer, S., Cinnamon, J., and Meersmans, J. (2019). Spatial and Temporal Variations in the Incidence of Dust Storms in Saudi Arabia Revealed from In Situ Observations. Geosciences, 9(4), 162. find
  3. Alhelbawy, A., Massimo, P., and Kruschwitz, U. (2016). Towards a corpus of violence acts in arabic social media. In Proceedings of the Tenth International Conference on Language Resources and Evaluation (LREC 2016) (pp. 1627-1631). find
  4. Almerekhi, H., Hasanain, M., & Elsayed, T. (2016). Evetar: A new test collection for event detection in arabic tweets. In Proceedings of the 39th International ACM SIGIR conference on Research and Development in Information Retrieval (pp. 689-692). ACM. find
  5. Alsaedi, N., & Burnap, P. (2015). Arabic event detection in social media. In International Conference on Intelligent Text Processing and Computational Linguistics (pp. 384-401). Springer, Cham. find
  6. Al-Twairesh, N., Al-Khalifa, H., Al-Salman, A., and Al-Ohali, Y. (2017). AraSenTi-Tweet: A Corpus for Arabic Sentiment Analysis of Saudi Tweets. Procedia Computer Science, 117, 63–72. find
  7. Fleiss, J. L. (1971). Measuring nominal scale agreement among many raters. Psychological Bulletin, 76(5), 378– 382. find
  8. Gu, Y., Qian, Z. (S., and Chen, F. (2016). From Twitter to detector: Real-time traffic incident detection using social media data. Transportation Research Part C: Emerging Technologies, 67, 321–342. find
  9. Hsu, C.-W., and Lin, C.-J. (2002). A comparison of methods for multiclass support vector machines. IEEE Transactions on Neural Networks, 13(2), 415–425. find
  10. Karagulian, F., Temimi, M., Ghebreyesus, D., Weston, M., Kondapalli, N. K., Valappil, V. K., … Abdooli, A. A. (2019). Analysis of a severe dust storm and its impact on air quality conditions using WRF-Chem modeling, satellite imagery, and ground observations. Air Quality, Atmosphere & Health, 12(4), 453–470. find
  11. Landis, J. R., and Koch, G. G. (1977). The Measurement of Observer Agreement for Categorical Data. Biometrics, 33(1), 159-174. find
  12. Mansuri, F., Al-Zalabani, A., Zalat, M., and Qabshawi, R. (2015). Road safety and road traffic accidents in Saudi Arabia. A systematic review of existing evidence. Saudi Medical Journal, 36(4), 418–424. find
  13. Mubarak, H., and Darwish, K. (2014). Using Twitter to Collect a Multi-Dialectal Corpus of Arabic. Proceedings of the EMNLP 2014 Workshop on Arabic Natural Language Processing (ANLP) (pp. 1-7). find
  14. Sakaki, T., Okazaki, M., and Matsuo, Y. (2010, April). Earthquake shakes Twitter users: real-time event detection by social sensors. In Proceedings of the 19th international conference on World wide web (pp. 851-860). ACM. find
  15. Schulz, A., Hadjakos, A., Paulheim, H., Nachtwey, J., and Mühlhäuser, M. (2013, June). A multi-indicator approach for geolocalization of tweets. In Seventh international AAAI conference on weblogs and social media. find
  16. Youssef, A. M., Sefry, S. A., Pradhan, B., and Alfadail, E. A. (2015). Analysis on causes of flash flood in Jeddah city (Kingdom of Saudi Arabia) of 2009 and 2011 using multi-sensor remote sensing data and GIS. Geomatics, Natural Hazards and Risk, 7(3), 1018–1042. find

↑ back to top

This page in Markdown → https://md.research.elamur.ai/free/en/papers/hamoui-2020-flodusta-saudi-tweets-dataset-flood/