Elamur Research⌕
☰
home / papers / 2025 / Relations: Arabic Relation Extraction Corpus and Modeling

Relations: Arabic Relation Extraction Corpus and Modeling

Paper · 2025 · Alaa Aljabari, Mohammed Khalilia and Mustafa Jarrar

Summary

Arabic relation extraction (RE) has lacked large, well-annotated, openly available data. Existing multilingual sets cover Arabic only thinly, use flat entities, or come from noisy distant supervision. The authors build Wojood Relations by adding manual relation labels to the nested-entity Wojood NER corpus. The result has about 33K sentences (~550K tokens) and roughly 15K triples over 40 relation types. Each relation is defined formally, with an allowed domain and range of entity types. Native-speaker experts reached Cohen's κ of 0.92.

On the modeling side, they propose two methods:

  • NLI-RE recasts each candidate relation as a yes/no entailment test, using a hypothesis built from a relation template.
  • GPT-Joint is a few-shot LLM setup for joint entity and relation extraction. It narrows the candidate relations first, then retrieves examples that match those relations.

Supervised models reach 92.89% F1 on RE, while the best LLM joint setup reaches 72.73% F1. A BERT-based pipeline built on NLI-RE ships as an end-to-end tool in SinaTools. The work gives Arabic RE a reusable benchmark and shows that, here, fine-tuned compact models still beat prompting large ones.

Key points

  • Wojood Relations: ~33K sentences, ~550K tokens, ~15K triples, 40 relation types, 21 entity types with nested NER.
    • It mostly covers formal MSA, such as news and historical texts.
    • Corpus, code and models are released under CC BY 4.0; the RE tool is an MIT-licensed Python library.
  • The 40 relation types were chosen for two reasons: they occur often in Arabic text, and they align with Wikidata and Schema.org.
  • NLI-RE uses ArBERTv2 and achieves the highest macro F1 among the compared models (88.58%). It does especially well on relations with few training examples.
  • Candidate relation prediction compares a fine-tuned multi-label BERT classifier with a few-shot LLM:
    • The classifier wins on micro F1 (91.31% vs 79.34%).
    • The LLM does slightly better on long-tail relations.
  • In joint extraction, zero-shot LLMs score poorly (best F1 ≈35%). Few-shot deepseek-reasoner, combined with BERT candidate prediction, reaches 72.73% F1.

Interesting ideas

  • I1. Each relation has an explicit domain and range over entity types, and entity pairs that violate these are discarded before any classifier runs. This removes ambiguity in annotation and cuts inference cost.
  • I2. Relation classification is turned into one binary entailment check per relation, using a natural-language template and a probability threshold. This lets several relations hold for the same entity pair and generalizes better to rare labels.
  • I3. Before prompting an LLM, a cheap supervised classifier shortlists the relations likely present in the sentence. Few-shot examples are then retrieved only for those relations, which keeps prompts focused.
  • I4. Each annotated triple carries a short relation-aware explanation of why the sentence supports that label. This doubles as documentation and as a check on annotator reasoning.
  • I5. The error analysis focuses on sentences where an explicit relation co-occurs with an implicit one. The example is president_of, which also implies lives_in. Most models recover only one of the two.

For Mova

How these ideas could serve Mova, our small open language model — its files, stages and tests:

  • I1 (world): Give every knowledge-graph relation typed argument slots, and reject or flag world facts whose arguments fall outside the allowed types before they are stored.
  • I2 (answer): A reader could verbalize each candidate fact as a hypothesis and test it against the text's world one by one, giving an explicit yes/no trace per fact instead of a single argmax label.
  • I3 (learn): In LLM teacher tiers, a light local model could first restrict the label set and select matching demonstrations, making teacher queries cheaper and less prone to off-schema answers.
  • I4 (learn): Silver data agreed by witness parsers could store a brief justification per accepted fact, so self-induced repair rules and audits can see why a label was kept.
  • I5 (world): Add evaluation cases that check whether implied states are also derived from a stated fact, and encode such implications as explicit inference rules in the compiled knowledge.

Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.

 Institutions

🎓Hamad bin Khalifa UniversityDoha · Qatar141 works · 14,175 citations🎓Birzeit UniversityBirzeit · Palestine36 works · 798 citations

 Fields, topics and concepts

📚 Shelves 5The field of science this work belongs to (OpenAlex fields).Computer Science · 16,667 worksArtificial Intelligence · 16,194 worksArts and Humanities · 16,039 worksLanguage and Linguistics · 15,891 worksComputer Vision and Pattern Recognition · 2,869 works

🧭 Topics 2Research subjects and languages the work is about.Semitic languages · 2,908 worksInformation extraction · 167 works

Cite

Cite as Alaa Aljabari, Mohammed Khalilia and Mustafa Jarrar. 2025. Relations: Arabic Relation Extraction Corpus and Modeling. Proceedings of the 2025 Conference on Empirical Methods in Natural Language Processing.

The paper itself, not this page. BibTeX from the library's record of the work. DOI 10.18653/v1/2025.emnlp-main.1741

@inproceedings{aljabari2025relations,
  title = {{Relations: Arabic Relation Extraction Corpus and Modeling}},
  author = {Alaa Aljabari, Mohammed Khalilia and Mustafa Jarrar},
  booktitle = {Proceedings of the 2025 Conference on Empirical Methods in Natural Language Processing},
  year = {2025},
  doi = {10.18653/v1/2025.emnlp-main.1741},
  url = {https://doi.org/10.18653/v1/2025.emnlp-main.1741}
}

References

44 references, 19 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

  1. Ahmed Abdelali, Hamdy Mubarak, Shammur Absar Chowdhury, Maram Hasanain, Basel Mousi, Sabri Boughorbel, Samir Abdaljalil, Yassine El Kheir, Daniel Izham, Fahim Dalvi, Majd Hawasly, Nizi Nazar, Yousseif Elshahawy, Ahmed Ali, Nadir Durrani, Natasa Milic-Frayling, and Firoj Alam. 2024. Larabench: Benchmarking arabic AI with large language models. In Proceedings of the 18th Conference of the European Chapter of the Association for Computational Linguistics, EACL 2024 - Volume 1: Long Papers, St. Julian’s, Malta, March 17-22, 2024, pages 487–520. Association for Computational Linguistics. ↗
  2. Muhammad Abdul-Mageed, AbdelRahim Elmadany, and El Moatez Billah Nagoudi. 2021. ARBERT & MARBERT: Deep bidirectional transformers for Arabic. In Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Conference on Natural Language Processing (Volume 1: Long Papers), pages 7088–7105, Online. Association for Computational Linguistics. ↗
  3. Diyam Akra, Tymaa Hammouda, and Mustafa Jarrar. 2025. QuranMorph: Morphologically Annotated Quranic Corpus. Technical report, Birzeit University. ↗
  4. Alaa Aljabari, Lina Duaibes, Mustafa Jarrar, and Mohammed Khalilia. 2024. Event-Arguments Extraction Corpus and Modeling using BERT for Arabic. In Proceedings of the Second Arabic Natural Language Processing Conference (ArabicNLP 2024), Bangkok, Thailand. Association for Computational Linguistics.
  5. Sylvio Barbon Junior, Paolo Ceravolo, Sven Groppe, Mustafa Jarrar, Samira Maghool, Florence Sèdes, Soror Sahri, and Maurice Van Keulen. 2024. Are Large Language Models the New Interface for Data Pipelines? In Proceedings of the International Workshop on Big Data in Emergent Distributed Environments, BiDEDE ’24, New York, NY, USA. Association for Computing Machinery. ↗
  6. Yupeng Chang, Xu Wang, Jindong Wang, Yuan Wu, Linyi Yang, Kaijie Zhu, Hao Chen, Xiaoyuan Yi, Cunxiang Wang, Yidong Wang, Wei Ye, Yue Zhang, Yi Chang, Philip S. Yu, Qiang Yang, and Xing Xie. 2024. A survey on evaluation of large language models. ACM Trans. Intell. Syst. Technol., 15(3). ↗
  7. Philippe Cudre-Mauroux, Karl Aberer, Alia Abdel-moty, Tiziana Catarci, Ernesto Damiani, Arantza Il-larramendi, Mustafa Jarrar, Robert Meersman, Erich Neuhold, Christine Parent, Kai-Uwe Sattler, Monica Scannapieco, Stefano Spaccapietra, Peter Spyns, and Guy De Tre. 2006. Viewpoints on emergent semantics. Journal on Data Semantics, 4090(6):2-s2.0-38549145879. find
  8. Jayati Deshmukh, Annervaz K M, and Shubhashis Sengupta. 2019. A sequence modeling approach for structured data extraction from unstructured text. In Proceedings of the 5th Workshop on Semantic Deep Learning (SemDeep-5), pages 57–66, Macau, China. Association for Computational Linguistics. find
  9. Kartik Detroja, C.K. Bhensdadia, and Brijesh S. Bhatt. 2023. A survey on relation extraction. Intelligent Systems with Applications, 19:200244. ↗
  10. George Doddington, Alexis Mitchell, Mark Przybocki, Lance Ramshaw, Stephanie Strassel, and Ralph Weischedel. 2004. The automatic content extraction (ACE) program – tasks, data, and evaluation. In Proceedings of the Fourth International Conference on Language Resources and Evaluation (LREC’04), Lisbon, Portugal. European Language Resources Association (ELRA). ↗
  11. AbdelRahim Elmadany, El Moatez Billah Nagoudi, and Muhammad Abdul-Mageed. 2022. Orca: A challenging benchmark for arabic language understanding. arXiv preprint arXiv:2212.10758. ↗
  12. Karim El Haff, Mustafa Jarrar, Tymaa Hammouda, and Fadi Zaraket. 2022. Curras + Baladi: Towards a Levantine Corpus. In Proceedings of the International Conference on Language Resources and Evaluation(LREC 2022), Marseille, France. ↗
  13. Tymaa Hammouda, Mustafa Jarrar, and Mohammed Khalilia. 2024. SinaTools: Open Source Toolkit for Arabic Natural Language Understanding. In Proceedings of the 2024 AI in Computational Linguistics (ACLING 2024), Procedia Computer Science, Dubai. ELSEVIER. find
  14. Pere-Lluís Huguet Cabot, Simone Tedeschi, Axel-Cyrille Ngonga Ngomo, and Roberto Navigli. 2023. RED fm : a filtered and multilingual relation extraction dataset. In Proceedings of the 61st Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 4326–4343, Toronto, Canada. Association for Computational Linguistics. find
  15. Mustafa Jarrar. 2011. Building a formal arabic ontology (invited paper). In Proceedings of the Experts Meeting on Arabic Ontologies and Semantic Networks. ALECSO, Arab League. find
  16. Mustafa Jarrar. 2021. The Arabic Ontology - An Arabic Wordnet with Ontologically Clean Content. Applied Ontology Journal, 16(1):1–26. ↗
  17. Mustafa Jarrar, Muhammad Abdul-Mageed, Mohammed Khalilia, Bashar Talafha, AbdelRahim Elmadany, Nagham Hamad, and Alaa’ Omar. 2023a. WojoodNER 2023: The First Arabic Named Entity Recognition Shared Task. In Proceedings of the 1st Arabic Natural Language Processing Conference (ArabicNLP), Part of the EMNLP 2023, pages 748– 758. ACL.
  18. Mustafa Jarrar, Diyam Akra, and Tymaa Hammouda. 2024a. ALMA: Fast Lemmatizer and POS Tagger for Arabic. In Proceedings of the 2024 AI in Computational Linguistics (ACLING 2024), Procedia Computer Science, Dubai. ELSEVIER. find
  19. Mustafa Jarrar and Anton Deik. 2015. The graph signature: A scalable query optimization index for rdf graph databases using bisimulation and trace equivalence summarization. International Journal on Semantic Web and Information Systems, 11(2):2-s2.0-84945200316. find
  20. Siyi Liu, Yang Li, Jiang Li, Shan Yang, and Yunshi Lan. 2024. Unleashing the power of large language models in zero-shot relation extraction via self-prompting. In Findings of the Association for Computational Linguistics: EMNLP 2024, pages 13147–13161. Association for Computational Linguistics. find
  21. Mustafa Jarrar and Marios D. Dikaiakos. 2010. Querying the data web -the mashql approach. IEEE Internet Computing, 14:2-s2.0-77953344860. ↗
  22. Amal Nayouf, Mustafa Jarrar, Fadi zaraket, Tymaa Hammouda, and Mohamad-Bassam Kurdy. 2023. Nâbra: Syrian Arabic Dialects with Morphological Annotations. In Proceedings of the 1st Arabic Natural Language Processing Conference (ArabicNLP), Part of the EMNLP 2023, pages 12–23. ACL.
  23. Mustafa Jarrar, Nagham Hamad, Mohammed Khalilia, Bashar Talafha, AbdelRahim Elmadany, and Muhammad Abdul-Mageed. 2024b. WojoodNER 2024: The Second Arabic Named Entity Recognition Shared Task. In Proceedings of the Second Arabic Natural Language Processing Conference (ArabicNLP 2024), Bangkok, Thailand. Association for Computational Linguistics.
  24. Mustafa Jarrar, Mohammed Khalilia, and Sana Ghanem. 2022. Wojood: Nested Arabic Named Entity Corpus and Recognition using BERT. In Proceedings of the International Conference on Language Resources and Evaluation (LREC 2022), Marseille, France.
  25. Mustafa Jarrar, Sanad Malaysha, Tymaa Hammouda, and Mohammed Khalilia. 2023b. SALMA: Arabic Sense-annotated Corpus and WSD Benchmarks. In Proceedings of the 1st Arabic Natural Language Processing Conference (ArabicNLP), Part of the EMNLP 2023, pages 359–369. ACL.
  26. Mustafa Jarrar, Fadi Zaraket, Tymaa Hammouda, Daanish Masood Alavi, and Martin Waahlisch. 2023c. Lisan: Yemeni, Irqi, Libyan, and Sudanese Arabic Dialect Copora with Morphological Annotations. In The 20th IEEE/ACS International Conference on Computer Systems and Applications (AICCSA), pages 1–7. IEEE. find
  27. Hrant Khachatrian, Lilit Nersisyan, Karen Hambardzumyan, Tigran Galstyan, Anna Hakobyan, Arsen Arakelyan, Andrey Rzhetsky, and Aram Galstyan. 2019. BioRelEx 1.0: Biological relation extraction benchmark. In Proceedings of the 18th BioNLP Workshop and Shared Task, pages 176–190, Florence, Italy. Association for Computational Linguistics. find
  28. Mohammed Khalilia, Sanad Malaysha, Reem Suwaileh, Mustafa Jarrar, Alaa Aljabari, Tamer Elsayed, and Imed Zitouni. 2024. ArabicNLU 2024: The First Arabic Natural Language Understanding Shared Task. In Proceedings of the Second Arabic Natural Language Processing Conference (ArabicNLP 2024), Bangkok, Thailand. Association for Computational Linguistics.
  29. Xiaoya Li, Fan Yin, Zijun Sun, Xiayu Li, Arianna Yuan, Duo Chai, Mingxin Zhou, and Jiwei Li. 2019. Entity-relation extraction as multi-turn question answering. In Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics, pages 1340– 1350, Florence, Italy. Association for Computational Linguistics. ↗
  30. Osama Rakan Al Mraikhat, Hadi Hamoud, and Fadi A. Zaraket. 2024. AREEj: Arabic relation extraction with evidence. In Proceedings of the Second Arabic Natural Language Processing Conference, pages 67–72, Bangkok, Thailand. Association for Computational Linguistics.
  31. Dan Roth and Wen-tau Yih. 2004. A linear programming formulation for global inference in natural language tasks. In Proceedings of the Eighth Conference on Computational Natural Language Learning (CoNLL-2004) at HLT-NAACL 2004, pages 1–8, Boston, Massachusetts, USA. Association for Computational Linguistics.
  32. Oscar Sainz, Oier Lopez de Lacalle, Gorka Labaka, Ander Barrena, and Eneko Agirre. 2021. Label verbalization and entailment for effective zero and few-shot relation extraction. In Proceedings of the 2021 Conference on Empirical Methods in Natural Language Processing, pages 1199–1212, Online and Punta Cana, Dominican Republic. Association for Computational Linguistics.
  33. Alessandro Seganti, Klaudia Firl ag, Helena Skowronska, Michał Satława, and Piotr Andruszkiewicz. 2021. Multilingual entity and relation extraction dataset and model. In Proceedings of the 16th Conference of the European Chapter of the Association for Computational Linguistics: Main Volume, pages 1946–1955, Online. Association for Computational Linguistics. find
  34. Anushka Swarup, Tianyu Pan, Ronald Wilson, Avanti Bhandarkar, and Damon Woodard. 2025. LLM4RE: A data-centric feasibility study for relation extraction. In Proceedings of the 31st International Conference on Computational Linguistics, pages 6670– 6691, Abu Dhabi, UAE. Association for Computational Linguistics. find
  35. Somin Wadhwa, Silvio Amir, and Byron Wallace. 2023. Revisiting relation extraction in the era of large language models. In Proceedings of the 61st Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 15566– 15589, Toronto, Canada. Association for Computational Linguistics. find
  36. Zhen Wan, Fei Cheng, Zhuoyuan Mao, Qianying Liu, Haiyue Song, Jiwei Li, and Sadao Kurohashi. 2023. find
  37. GPT-RE: In-context learning for relation extraction using large language models. In Proceedings of the 2023 Conference on Empirical Methods in Natural Language Processing, pages 3534–3547, Singapore. Association for Computational Linguistics.
  38. Jianyou Wang, Kaicheng Wang, Xiaoyue Wang, Weili Cao, Ramamohan Paturi, and Leon Bergen. 2024. IR2: Information regularization for information retrieval. In Proceedings of the 2024 Joint International Conference on Computational Linguistics, Language Resources and Evaluation (LREC-COLING 2024), pages 9261–9284, Torino, Italia. ELRA and ICCL. find
  39. Xiyu Wang and Nora El-Gohary. 2023. Deep learning-based relation extraction and knowledge graph-based representation of construction safety requirements. Automation in Construction, 147:104696. ↗
  40. Zhaohui Yan, Songlin Yang, Wei Liu, and Kewei Tu. 2023. Joint entity and relation extraction with span pruning and hypergraph neural networks. In Proceedings of the 2023 Conference on Empirical Methods in Natural Language Processing, pages 7512–7526, Singapore. Association for Computational Linguistics. find
  41. Hongbin Ye, Ningyu Zhang, Hui Chen, and Huajun Chen. 2022. Generative knowledge graph construction: A review. In Proceedings of the 2022 Conference on Empirical Methods in Natural Language Processing, pages 1–17, Abu Dhabi, United Arab Emirates. Association for Computational Linguistics. find
  42. Kai Zhang, Bernal Jimenez Gutierrez, and Yu Su. 2023. Aligning instruction tasks unlocks large language models as zero-shot relation extractors. In Findings of the Association for Computational Linguistics: ACL 2023, pages 794–812. Association for Computational Linguistics.
  43. Yuhao Zhang, Victor Zhong, Danqi Chen, Gabor Angeli, and Christopher D. Manning. 2017. Position-aware attention and supervised data improve slot filling. In Proceedings of the 2017 Conference on Empirical Methods in Natural Language Processing, pages 35–45, Copenhagen, Denmark. Association for Computational Linguistics. find
  44. Kang Zhao, Hua Xu, Yue Cheng, Xiaoteng Li, and Kai Gao. 2021. Representation iterative fusion based on heterogeneous graph neural network for joint entity and relation extraction. Knowledge-Based Systems, page 106888. ↗

Updated

↑ back to top

This page in Markdown → https://md.research.elamur.ai/free/en/papers/aljabari-2025-mathrmwojood-relations-arabic-relation-extraction/