Elamur Research⌕
☰
home / papers / 2020 / Supervised Event Coding from Text Written in Arabic: Introducing Hadath

Supervised Event Coding from Text Written in Arabic: Introducing Hadath

Paper · 2020 · Javier Osorio, Alejandro Reyes, Alejandro Beltrán et al.

Summary

Most computerized event data in conflict research is built from English-language news. This can distort the picture for places where English is not spoken locally. The authors present Hadath, a supervised event-coding protocol for Modern Standard Arabic, and test it on the war in Afghanistan from 2008 to 2018.

The pipeline has two stages:

  • Filtering. They collected roughly 100k candidate articles from Arabic newspapers on Nexis Uni. Several classifiers were compared for separating relevant stories from noise, and logistic regression was selected.
  • Coding. Hadath applies shallow parsing driven by dictionaries of actors, actions and locations. It identifies the actors involved, the action taken, its target, and the time and place of each event.

The resulting dataset is geo-referenced at the district level with daily resolution. Its event concept is broader than violence and also covers governance provision and traditional dispute resolution.

Arabic was chosen for practical reasons: it has more text, tools and coders than Dari or Pashto. The authors see it as a stepping stone toward other languages written in Arabic script.

Key points

  • Source and corpus. Nexis Uni offers 17 Arabic newspapers with more than 2.5 million articles from 2008–2018. A targeted query produced 100,857 stories, which three research assistants downloaded by hand.

  • Dating. The date-extractor Python package found each article's publication date. The date was written into the filename so Hadath could assign event days and group stories by year.

  • Classifier comparison (5-fold CV, average F1):

    ModelF1
    CNN, kernel 30.923
    CNN, kernel 50.922
    Random Forest0.924
    Naive Bayes0.835
    XGBoost0.919
    SVM0.94
    Logistic Regression0.94

  • Tie-break. SVM and LR were retrained on the full training data with 10% held out for testing. SVM scored 0.934 F1 and LR scored 0.938, so LR was used to classify all articles.

  • Training-data fix. An added subset made up only 7% of the training data but noticeably improved results. It raised the "accept" class to 16,339 articles.

Interesting ideas

  • I1. Run a cheap relevance classifier first and apply the costly structured extraction only to documents that pass it; this saves effort and reduces false events from off-topic text.
  • I2. Coding events with explicit dictionaries of actors, actions and places keeps every extracted triple traceable to a lexicon entry rather than to opaque weights.
  • I3. Defining "events" to include cooperative and institutional acts (governance, conflict resolution), not just violence, avoids a skewed view of what happens in a text.
  • I4. Attaching the document's publication date as metadata gives every extracted event a temporal anchor.
  • I5. A small, deliberately added slice of training data (7%) improved the classifier noticeably, and a near-tie between models was settled by a final retrain with a held-out test.

For Mova

How these ideas could serve Mova, our small open language model — its files, stages and tests:

  • I1 (answer): A light filter could pick out which passages of a school text or fable matter for a question before the full tree→frames→world reasoning runs.
  • I2 (frames): Agent/patient/location slots in prag frames could be checked against curated actor and action lists, so each role assignment carries an inspectable justification.
  • I3 (world): The world model's event inventory should give equal weight to helping, ruling and settling actions so that tale and fable worlds are not dominated by conflict verbs.
  • I4 (world): A document-level reference time could anchor relative expressions ("the next day") when ordering events in the world of the text.
  • I5 (learn): Targeted additions of hard silver examples, plus a clean held-out decision step, could guide which parser or repair rule gets promoted.

Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.

 Fields, topics and concepts

📚 Shelves 6The field of science this work belongs to (OpenAlex fields).Computer Science · 16,667 worksArtificial Intelligence · 16,194 worksArts and Humanities · 16,039 worksLanguage and Linguistics · 15,891 worksSocial Sciences · 1,772 worksGeneral Social Sciences · 227 works

🧭 Topics 1Research subjects and languages the work is about.Semitic languages · 2,908 works

Cite

Cite as Javier Osorio, Alejandro Reyes, Alejandro Beltrán and Atal Ahmadzai. 2020. Supervised Event Coding from Text Written in Arabic: Introducing Hadath. Proceedings of the Workshop on Automated Extraction of Socio-political Events from News 2020.

The paper itself, not this page. BibTeX from the library's record of the work.

@inproceedings{osorio2020supervised,
  title = {{Supervised Event Coding from Text Written in Arabic: Introducing Hadath}},
  author = {Javier Osorio, Alejandro Reyes, Alejandro Beltrán and Atal Ahmadzai},
  booktitle = {Proceedings of the Workshop on Automated Extraction of Socio-political Events from News 2020},
  year = {2020},
  url = {https://aclanthology.org/2020.aespen-1.9/}
}

References

53 references, 2 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

  1. Althaus, S., Bajjalieh, J., Carter, J. F., Peyton, B., and Shalmon, D. A. (2019). Cline Center Historical Phoenix Event Data. find
  2. Arjona, A. (2016). Rebelocracy: Social Order in the Colombian Civil War. Cambridge University Press, NY. find
  3. Best, R. H., Carpino, C., and Crescenzi, M. J. C. (2013). An analysis of the TABARI coding system. Conflict Management and Peace Science, 30(4):335–348, jul. ↗
  4. Bond, D., Bond, J., Oh, C., Jenkins, C. J., and Taylor, C. L. (2003). Integrated Data for Events Analysis (IDEA): An Event Typology for Automated Events Data Development. Journal of Peace Research, 40(6):733–745. ↗
  5. Boschee, E., Lautenschlager, J., O’Brien, S., Shellman, S., Starz, J., and Ward, M. (2016). ICEWS Coded Event Data. find
  6. CFR. (2020). A timeline of the u.s. war in afghanistan. find
  7. Chojnacki, S., Ickler, C., Spies, M., and Wiesel, J. (2012). Event Data on Armed Conflict and Security: New Perspectives, Old Challenges, and Some Solutions. International Interactions, 38(4):382–401. ↗
  8. Gambhir, H. (2015). Isis in afghanistan. Backgrounder, Institute for the Study of War, 6. find
  9. Gerner, D., Schrodt, P., Yilmaz, O., and Abu-Jabr, R. (2002). The Creation of CAMEO (Conflict and Mediation Event Observations): An Event Data Framework for a Post Cold War World. In The Annual Meeting of the American Political Science Association. 01/10/2013. find
  10. Grimmer, J. and Steward, B. M. (2013). Text as data: The promise and pitfalls of automatic content analysis methods for political texts. Political Analysis, 21(3):267–297. ↗
  11. Halterman, A., Irvine, J., Grant, C., Jabr, K., and Yand, L. (2018). Creating and Automated Event Data System for Arabic Text. find
  12. Hammond, J. and Weidmann, N. B. (2014a). Using machine-coded event data for the micro-level study of political violence. Research & Politics, 1(2):2053168014539924, jul. ↗
  13. Hammond, J. and Weidmann, N. B. (2014b). Using machine-coded event data for the micro-level study of political violence. Research and Politics, 1(2):1–8. ↗
  14. Hollibaugh, G. E. (2018). The use of text as data methods in public administration: A review and an application to agency priorities. Journal of Public Administration Research and Theory, 29(3):474–490. ↗
  15. Hürriyetoğlu, A., Yörük, E., Yüret, D., Yoltar, Ç., Gürel, B., Duruşan, F., and Mutlu, O. (2019). A Task Set Proposal for Automatic Protest Information Collection Across Multiple Countries. In Leif Azzopardi, et al., editors, Advances in Information Retrieval. 41st European Conference on IR Research, ECIR 2019, Cologne, Germany, April 14–18, 2019, Proceedings, Part II. Springer. find
  16. Jhonson, S. (2011). Where Good Ideas Come From. Riverhead Books, New York. find
  17. Jones, S. (2008). The rise of afghanistan’s insurgency: State failure and jihad. International Security, 32(4):7– 40. ↗
  18. Kean, T. (2011). The 9/11 commission report: Final report of the national commission on terrorist attacks upon the United States. Government Printing Office. find
  19. Kenneth, K. and Thomas, C. (2017). Afghanistan: Post-taliban governance, security, and us policy. Congressional Research Service, available at: www. fas. org/sgp/crs/row/RL30588. pdf. find
  20. Masadykov, T., Giustozzi, A., and Page, J. M. (2010). Negotiating with the taliban: toward a solution for the afghan conflict. find
  21. Mohiuddin, S., Salam, S., Mustafa, A. M., Khan, L., Brandt, P. T., and Bhavani, T. (2016). Near Real-Time Atrocity Event Coding. IEEE Intelligence and Security Informatics (ISI) 2016. find
  22. O’Brien, S. (2012). A multi-method approach for near real time conflict and crisis early warning. In V. S. Subrahmanian, editor, Handbook of Computational Approaches to Counterterrorism, pages 401–418. Springer, NY. ↗
  23. Open Event Data Alliance. (2016). Universal Petrarch. find
  24. Osorio, J. and Reyes, A. (2017). Supervised event coding from text written in spanish: Introducing eventus id. Social Science Computer Review, 35(3):406–416. ↗
  25. Osorio, J., Mohamed, M., Pavon, V., and Brewer-Osorio, S. (2019a). Mapping violent presence of armed actors in colombia. Advances of Cartography and GIScience of the International Cartographic Association. ↗
  26. Osorio, J., Pavon, V., Salam, S., Holmes, J., Brandt, P. T., and Khan, L. (2019b). Translating CAMEO verbs for automated coding of event data. International Interactions, 45(6):1049–1064. ↗
  27. O’Brien, S. P. (2010). Crisis Early Warning and Decision Support: Contemporary Approaches and Thoughts on Future Research. International Studies Review, 12(1):87–104. ↗
  28. Piskorski, J., Tanev, H., Atkinson, M., Van Der Goot, E., and Zavarella, V. (2011). Online news event extraction for global crisis surveillance. In Transactions on computational collective intelligence V, pages 182–212. Springer. find
  29. Raleigh, C., Linke, A., Hegre, H., and Karlsen, J. (2010). Introducing acled: an armed conflict location and event dataset: special data feature. Journal of peace research, 47(5):651–660. ↗
  30. Schrodt, P. and Gerner, D. (2012). Fundamentals of Machine Coding. In Analyzing International Event Data: A Handbook of Computer-Based Techniques. find
  31. Schrodt, P., Gerner, D., and Yilmaz, O. (2004). Using Event Data to Monitor Contemporary Conflict in the Israeli-Palestine Dyad. In Annual Meeting of the International Studies Association. find
  32. Schrodt, P., Beieler, J., and Idris, M. (2014). Three’s a Charm?: Open Event Data Coding with EL:DIABLO, PETRARCH, and the Open Event Data Alliance. In International Studies Association, Toronto. find
  33. Schrodt, P. (1998). Kansas Event Data System. find
  34. Schrodt, P. (2001). Automated Coding of International Event Data Using Sparse Parsing Techniques. Chicago. Paper presented at the annual meeting of the International Studies Association. find
  35. Schrodt, P. (2006). Twenty Years of the Kansas Event Data System Project. The Political Methodologist, 14(1):2–6. find
  36. Schrodt, P. (2009). TABARI. Textual Analysis by Augmented Replacement Instructions. find
  37. Schrodt, P. a. (2012). Precedents, Progress, and Prospects in Political Event Data. International Interactions, 38(4):546–569, sep. ↗
  38. Staniland, P. (2012). States, Insurgents, and Wartime Political Orders. Perspectives on Politics, 10(2):243–264. ↗
  39. Subrahmanian, V. S. (2013). Handbook of Computational Approaches to Counterterrorism. Springer, New York. ↗
  40. Sundberg, R. and Melander, E. (2013). Introducing the ucdp georeferenced event dataset. Journal of Peace Research, 50(4):523–532. ↗
  41. Wang, W., Kennedy, R., Lazer, D., and Ramakrishnan, N. (2016). Growing pains for global monitoring of societal events. Science, 353(6307):1502–1504. ↗
  42. Wilkerson, J. and Casas, A. (2017). Large-scale computerized text analysis in political science: Opportunities and challenges. Annual Review of Political Science, 20(1):529–544. 11. ↗
  43. Language Resource References Abdelali, A., Darwish, K., Durrani, N., and Mubarak, H. (2016). Farasa: A fast and furious segmenter for arabic. In Proceedings of the 2016 Conference of the North American Chapter of the Association for Computational Linguistics: Demonstrations, pages 11–16.
  44. AbdelRahim Elmadany, H. M. and Magdy, W. (2018). Arsas: An arabic speech-act and sentiment corpus of tweets. In Hend Al-Khalifa, et al., editors, Proceedings of the Eleventh International Conference on Language Resources and Evaluation (LREC 2018), Paris, France, May. European Language Resources Association (ELRA). find
  45. Alshutayri, A. and Atwell, E. (2018). Creating an arabic dialect text corpus by exploring twitter, facebook, and online newspapers. In Hend Al-Khalifa, et al., editors, Proceedings of the Eleventh International Conference on Language Resources and Evaluation (LREC 2018), Paris, France, May. European Language Resources Association (ELRA). find
  46. Dufour, D. J. (2020). date-extractor. find
  47. Hasanain, M., Suwaileh, R., Elsayed, T., Kutlu, M., and Almerekhi, H. (2018). Evetar: building a large-scale multi-task test collection over arabic tweets. Information Retrieval Journal, 21(4):307–336. ↗
  48. Khaled Yasser, Reem Suwaileh, A. S. Y. B. M. K. and Elsayed, T. (2018). Iarabicweb16: Making a large web collection more accessible for research. In Hend Al-Khalifa, et al., editors, Proceedings of the Eleventh International Conference on Language Resources and Evaluation (LREC 2018), Paris, France, may. European Language Resources Association (ELRA). find
  49. Pedregosa, F., Varoquaux, G., Gramfort, A., Michel, V., Thirion, B., Grisel, O., Blondel, M., Prettenhofer, P., Weiss, R., Dubourg, V., et al. (2011). Scikit-learn: Machine learning in python. Journal of machine learning research, 12(Oct):2825–2830. ↗
  50. Sawsan Alqahtani, M. D. and Zaghouani, W. (2018). A large scale comprehensive lexical inventory for modern standard arabic. In Hend Al-Khalifa, et al., editors, Proceedings of the Eleventh International Conference on Language Resources and Evaluation (LREC 2018), Paris, France, May. European Language Resources Association (ELRA). find
  51. Suwaileh, R., Kutlu, M., Fathima, N., Elsayed, T., and Lease, M. (2016). Arabicweb16: A new crawl for today’s arabic web. In Proceedings of the 39th International ACM SIGIR conference on Research and Development in Information Retrieval, pages 673–676. find
  52. Zaghouani, W. and Charfi, A. (2018). Guidelines and annotation framework for arabic author profiling. In Hend Al-Khalifa, et al., editors, Proceedings of the Eleventh International Conference on Language Resources and Evaluation (LREC 2018), Paris, France, may. European Language Resources Association (ELRA). ↗
  53. Zerrouki, T. (2012). Tashaphyne, arabic light stemmer. find

Updated

↑ back to top

This page in Markdown → https://md.research.elamur.ai/free/en/papers/osorio-2020-supervised-event-coding-text-written/