Elamur Research⌕
☰
home / papers / 2024 / Event-Arguments Extraction Corpus and Modeling using BERT for Arabic

Event-Arguments Extraction Corpus and Modeling using BERT for Arabic

Paper · 2024 · Alaa Aljabari, Lina Duaibes, Mustafa Jarrar et al.

Summary

Arabic has few annotated resources for linking events to their participants, places and times. The authors address this by adding relation annotations to the existing Wojood nested NER corpus, producing Wojood Hadath. In it, each event mention is connected to agent, location and date entities in the same sentence through three relation labels. Two annotators worked in parallel and reached high agreement.

For modelling, they convert every candidate (event, entity) pair into a sentence-pair inference problem. The original sentence serves as the premise. A short templated statement asserting the relation serves as the hypothesis. A BERT model then decides whether the sentence entails that statement. This reformulation scores about 94% F1 in-domain.

A separately collected out-of-domain corpus shows that performance drops but stays reasonable, at about 84% F1. The whole pipeline is packaged as an end-to-end tool inside SinaTools, and both corpora are released publicly. The work matters because it provides a sizable Arabic benchmark and shows that entailment framing is a practical route to argument extraction.

Key points

  • Wojood Hadath extends Wojood, which has 550k tokens, 21 entity types and 2,772 event mentions, with three relations:
    • hasAgent links to PERS, ORG, OCC or NORP.
    • hasLocation links to GPE, LOC or FAC.
    • hasDate links to DATE or TIME.
  • Inter-annotator agreement reached a Cohen's kappa of 82.23% and an F1 of 87.2%.
  • Hadath NLI (about 25k premise–hypothesis pairs) was derived from the corpus.
    • The fine-tuned BERT model reaches 94.01% average F1.
    • Per relation, F1 is 84.07 for hasAgent, 89.88 for hasLocation and 96.44 for hasDate.
  • Wojood OutOfDomain (about 80k tokens) contains 481 relations and yields 1,124 NLI pairs.
    • The model reaches 83.59% average F1 on it.
    • The positive class is clearly weaker than the negative class, at 74.38 versus 92.80 F1.
  • The end-to-end EAE system in SinaTools is evaluated assuming gold named entities.
    • It reaches 93.99 F1 on the Hadath NLI test set and 74.90 F1 on Wojood OutOfDomain.
    • In ablations, template t2 was marginally the best, and the top loss combined weighted cross-entropy (positive weight 1, negative weight 0.5) with an NCE loss.

Interesting ideas

  • I1. Each candidate relation is rewritten as a plain-language claim and checked for entailment against the sentence, so a single binary classifier covers all relation types.
  • I2. The entity category decides which relation template is even attempted, which prunes impossible pairings before any model is consulted.
  • I3. The guidelines keep coreferent mentions as separate arguments, allow several agents per event, and allow one argument to be shared by multiple events. This leaves identity resolution to a later layer.
  • I4. Generalization is measured on a freshly annotated out-of-domain set that follows the same guidelines. Gold entities are assumed so that the relation step is measured in isolation.
  • I5. Down-weighting the abundant negative pairs in the loss improved out-of-domain results noticeably.

For Mova

How these ideas could serve Mova, our small open language model — its files, stages and tests:

  • I1 (frames): An entailment-style checker could act as an independent verifier of VerbNet role assignments, flagging frames whose verbalized claim the sentence does not support.
  • I2 (world): Type-to-role compatibility tables built from WordNet or Wikidata classes could filter candidate role fillers early and keep the reasoning trace short and explainable.
  • I3 (world): Store event–argument links per mention and merge them only after coreference, so each merge decision stays auditable instead of being baked into the annotation.
  • I4 (learn): Hold out a differently sourced test slice and score each stage with oracle inputs from the previous stage, to locate where errors really originate.
  • I5 (learn): When training on silver data where "no relation" pairs dominate, class-weighted losses could keep the positive class from being drowned out.

Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.

 Institutions

🎓Birzeit UniversityBirzeit · Palestine36 works · 798 citations

 Fields, topics and concepts

📚 Shelves 4The field of science this work belongs to (OpenAlex fields).Computer Science · 16,667 worksArtificial Intelligence · 16,194 worksArts and Humanities · 16,039 worksLanguage and Linguistics · 15,891 works

🧭 Topics 1Research subjects and languages the work is about.Semitic languages · 2,908 works

Cite

Cite as Alaa Aljabari, Lina Duaibes, Mustafa Jarrar and Mohammed Khalilia. 2024. Event-Arguments Extraction Corpus and Modeling using BERT for Arabic. Proceedings of the Second Arabic Natural Language Processing Conference.

The paper itself, not this page. BibTeX from the library's record of the work. DOI 10.18653/v1/2024.arabicnlp-1.26

@inproceedings{aljabari2024event,
  title = {{Event-Arguments Extraction Corpus and Modeling using BERT for Arabic}},
  author = {Alaa Aljabari, Lina Duaibes, Mustafa Jarrar and Mohammed Khalilia},
  booktitle = {Proceedings of the Second Arabic Natural Language Processing Conference},
  year = {2024},
  doi = {10.18653/v1/2024.arabicnlp-1.26},
  url = {https://doi.org/10.18653/v1/2024.arabicnlp-1.26}
}

References

35 references, 11 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

  1. Muhammad Abdul-Mageed, AbdelRahim Elmadany, and El Moatez Billah Nagoudi. 2021. ARBERT & MARBERT: Deep bidirectional transformers for Arabic. In Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Conference on Natural Language Processing (Volume 1: Long Papers), pages 7088–7105, Online. Association for Computational Linguistics. ↗
  2. Kashif Ahmad, Konstantin Pogorelov, Michael Riegler, Nicola Conci, and Pål Halvorsen. 2019. Social media and satellites: Disaster event detection, linking and summarization. Multimedia Tools and Applications, 78:2837–2875. find
  3. Moustafa Al-Hajj and Mustafa Jarrar. 2021. Arab-GlossBERT: Fine-Tuning BERT on Context-Gloss Pairs for WSD. In Proceedings of the International Conference on Recent Advances in Natural Language Processing (RANLP 2021), pages 40–48, Online. INCOMA Ltd. find
  4. Mohammad AL-Smadi and Omar Qawasmeh. 2016. Knowledge-based approach for event extraction from arabic tweets. International Journal of Advanced Computer Science and Applications, 7(6). ↗
  5. Ebtesam Alomari, Iyad Katib, and Rashid Mehmood. 2020. Iktishaf: A big data road-traffic event detection tool using twitter and spark machine learning. Mobile Networks and Applications, pages 1–16. ↗
  6. Sylvio Barbon Junior, Paolo Ceravolo, Sven Groppe, Mustafa Jarrar, Samira Maghool, Florence Sèdes, Soror Sahri, and Maurice Van Keulen. 2024. Are Large Language Models the New Interface for Data Pipelines? In Proceedings of the International Workshop on Big Data in Emergent Distributed Environments, BiDEDE ’24, New York, NY, USA. Association for Computing Machinery. ↗
  7. Pere-Lluís Huguet Cabot, Simone Tedeschi, Axel-Cyrille Ngonga Ngomo, and Roberto Navigli. 2023. Red FM :a filtered and multilingual relation extraction dataset. arXiv preprint arXiv:2306.09802. ↗
  8. Pengfei Cao, Zhuoran Jin, Yubo Chen, Kang Liu, and Jun Zhao. 2023. Zero-shot cross-lingual event argument extraction with language-oriented prefix-tuning. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 37, pages 12589– 12597. find
  9. Amina Chouigui, Oussama Ben Khiroun, and Bilel Elayeb. 2018. A tf-idf and co-occurrence based approach for events extraction from arabic news corpus. In Natural Language Processing and Information Systems: 23rd International Conference on Applications of Natural Language to Information Systems, NLDB 2018, Paris, France, June 13-15, 2018, Proceedings 23, pages 272–280. Springer. find
  10. Kareem Darwish, Nizar Habash, Mourad Abbas, Hend Al-Khalifa, Huseein T. Al-Natsheh, Houda Bouamor, Karim Bouzoubaa, Violetta Cavalli-Sforza, Samhaa R. El-Beltagy, Wassim El-Hajj, Mustafa Jarrar, and Hamdy Mubarak. 2021. A Panoramic survey of Natural Language Processing in the Arab Worlds. Commun. ACM, 64(4):72–81. find
  11. Lina Duaibes, Areej Jaber, Mustafa Jarrar, Ahmad Qadi, and Mais Qandeel. 2024. Sina at FigNews 2024: Multilingual Datasets Annotated with Bias and Propaganda. In Proceedings of the Second Arabic Natural Language Processing Conference (ArabicNLP 2024), Bangkok, Thailand. Association for Computational Linguistics.
  12. Niama El Khbir, Nadi Tomeh, and Thierry Charnois. 2022. ArabIE: Joint entity, relation and event extraction for Arabic. In Proceedings of the The Seventh Arabic Natural Language Processing Workshop (WANLP), pages 331–345, Abu Dhabi, United Arab Emirates (Hybrid). Association for Computational Linguistics.
  13. Barbara Di Eugenio and Michael Glass. 2004. The kappa statistic: A second look. Computational linguistics, 30(1):95–101. ↗
  14. Karim El Haff, Mustafa Jarrar, Tymaa Hammouda, and Fadi Zaraket. 2022. Curras + Baladi: Towards a Levantine Corpus. In Proceedings of the International Conference on Language Resources and Evaluation (LREC 2022), Marseille, France. ↗
  15. Tymaa Hammouda, Mustafa Jarrar, and Mohammed Khalilia. 2024. SinaTools: Open Source Toolkit for Arabic Natural Language Understanding. In Proceedings of the 2024 AI in Computational Linguistics (ACLING 2024), Procedia Computer Science, Dubai. ELSEVIER. find
  16. Aldo Hernandez-Suarez, Gabriel Sanchez-Perez, Karina Toscano-Medina, Hector Perez-Meana, Jose Portillo-Portillo, Victor Sanchez, and Luis Javier García Villalba. 2019. Using twitter data to monitor natural disaster social dynamics: A recurrent neural network approach with word embeddings and kernel density estimation. Sensors, 19(7):1746. ↗
  17. Emna Hkiri, Souheyl Mallat, and Mounir Zrigui. 2016. Events automatic extraction from arabic texts. International Journal of Information Retrieval Research (IJIRR), 6(1):36–51. ↗
  18. Mustafa Jarrar. 2021. The Arabic Ontology - An Arabic Wordnet with Ontologically Clean Content. Applied Ontology Journal, 16(1):1–26. ↗
  19. Mustafa Jarrar, Muhammad Abdul-Mageed, Mohammed Khalilia, Bashar Talafha, AbdelRahim Elmadany, Nagham Hamad, and Alaa’ Omar. 2023a. WojoodNER 2023: The First Arabic Named Entity Recognition Shared Task. In Proceedings of the 1st Arabic Natural Language Processing Conference (ArabicNLP), Part of the EMNLP 2023, pages 748–758. ACL.
  20. Mustafa Jarrar and Hamzeh Amayreh. 2019. An arabic-multilingual database with a lexicographic search engine. In The 24th International Conference on Applications of Natural Language to Information Systems (NLDB 2019), volume 11608 of LNCS, pages 234–246. Springer. find
  21. Mustafa Jarrar and Werner Ceusters. 2017. Classifying processes and basic formal ontology. In Proceedings of the 8th International Conference on Biomedical Ontology (ICBO 2017), volume 2137. CEUR Workshop Proceedings. find
  22. Mustafa Jarrar, Nagham Hamad, Mohammed Khalilia, Bashar Talafha, AbdelRahim Elmadany, and Muhammad Abdul-Mageed. 2024. WojoodNER 2024: The Second Arabic Named Entity Recognition Shared Task. In Proceedings of the Second Arabic Natural Language Processing Conference (ArabicNLP 2024), Bangkok, Thailand. Association for Computational Linguistics.
  23. Mustafa Jarrar, Mohammed Khalilia, and Sana Ghanem. 2022a. Wojood: Nested Arabic Named Entity Corpus and Recognition using BERT. In Proceedings of the International Conference on Language Resources and Evaluation (LREC 2022), Marseille, France.
  24. Mustafa Jarrar, Mohammed Khalilia, and Sana Ghanem. 2022b. Wojood: Nested arabic named entity corpus and recognition using bert. In Proceedings of the Thirteenth Language Resources and Evaluation Conference, pages 3626–3636.
  25. Mustafa Jarrar, Sanad Malaysha, Tymaa Hammouda, and Mohammed Khalilia. 2023b. SALMA: Arabic Sense-annotated Corpus and WSD Benchmarks. In Proceedings of the 1st Arabic Natural Language Processing Conference (ArabicNLP), Part of the EMNLP 2023, pages 359–369. ACL.
  26. Alessandro Seganti, Klaudia Firl ag, Helena Skowronska, Michał Satława, and Piotr Andruszkiewicz. 2021. Multilingual entity and relation extraction dataset and model. In Proceedings of the 16th Conference of the European Chapter of the Association for Computational Linguistics: Main Volume, pages 1946–1955. find
  27. Tomer Simon, Avishay Goldberg, and Bruria Adini. 2015. Socializing in emergencies—a review of the use of social media in emergency situations. International journal of information management, 35(5):609–619. ↗
  28. Sonit Singh. 2018. Natural language processing for information extraction. arXiv preprint arXiv:1807.02383. ↗
  29. Jiashu Xu, Mingyu Derek Ma, and Muhao Chen. 2022. Can nli provide proper indirect supervision for low-resource biomedical relation extraction? arXiv preprint arXiv:2212.10784. ↗
  30. Hongbin Ye, Ningyu Zhang, Hui Chen, and Huajun Chen. 2022. Generative knowledge graph construction: A review. CoRR, abs/2210.12714. ↗
  31. Rasmus Jørgensen, Oliver Brandt, Mareike Hartmann, Xiang Dai, Christian Igel, and Desmond Elliott. 2023. MultiFin: A dataset for multilingual financial NLP. In Findings of the Association for Computational Linguistics: EACL 2023, pages 894–909, Dubrovnik, Croatia. Association for Computational Linguistics. find
  32. Mohammed Khalilia, Sanad Malaysha, Reem Suwaileh, Mustafa Jarrar, Alaa Aljabari, Tamer Elsayed, and Imed Zitouni. 2024. ArabicNLU 2024: The First Arabic Natural Language Understanding Shared Task. In Proceedings of the Second Arabic Natural Language Processing Conference (ArabicNLP 2024), Bangkok, Thailand. Association for Computational Linguistics.
  33. Nouran Khallaf, Elin Arfon, Mo El-Haj, Jon Morris, Dawn Knight, Paul Rayson, and Tymaa Ham-moudaand Mustafa Jarrar. 2023. Open-source thesaurus development for under-resourced languages: a welsh case study. In The 4th Conference on Language, Data and Knowledge (LDK2023). find
  34. Haneen Liqreina, Mustafa Jarrar, Mohammed Khalilia, Ahmed Oumar El-Shangiti, and Muhammad Abdul-Mageed. 2023. Arabic Fine-Grained Entity Recognition. In Proceedings of the 1st Arabic Natural Language Processing Conference (ArabicNLP), Part of the EMNLP 2023, pages 310–323. ACL.
  35. Joshua Robinson, Ching-Yao Chuang, Suvrit Sra, and Stefanie Jegelka. 2020. Contrastive learning with hard negative samples. ArXiv, abs/2010.04592. ↗

Updated

↑ back to top

This page in Markdown → https://md.research.elamur.ai/free/en/papers/aljabari-2024-event-arguments-extraction-corpus-modeling/