Elamur Research⌕
☰
home / papers / 2025 / WojoodOntology: Ontology-Driven LLM Prompting for Unified Information Extraction Tasks

WojoodOntology: Ontology-Driven LLM Prompting for Unified Information Extraction Tasks

Paper · 2025 · Alaa Aljabari, Nagham Hamad, Mohammed Khalilia et al.

Summary

Arabic NER and relation-extraction corpora use incompatible tagsets, span conventions and relation definitions. This blocks merging datasets, cross-corpus evaluation and interoperability between tools such as SinaTools and CaMLTools. The authors build WojoodOntology, an OWL ontology of 55 entity concepts and 40 relation types. They derived it from a survey of existing Arabic IE datasets and linked it to Wikidata and Schema.org. Entity types are OWL classes and relations are object properties with domain and range constraints. equivalentClass and subClassOf axioms record how labels from different schemes relate.

The ontology is used in two ways:

  • a Python library of one-way mapping rules;
  • prompting, where the OWL ontology is placed inside an LLM (GPT-4o) prompt so the model can map labels in either direction.

They tested this by re-annotating the AQMAR corpus under Wojood guidelines. Ontology-based prompting improved out-of-domain mapping accuracy by about 15% over the rule-based method. Few-shot prompting with the ontology gave the best F1 (about 0.55). The ontology is released as open source.

Key points

  • WojoodOntology has 55 entity types and 40 relation types. It is formalized in OWL using equivalentClass, subClassOf and domain-range axioms, and aligned with Schema.org and Wikidata.
  • The paper shows concrete mismatches between schemes:
    • Wojood tags Damascus City as GPE, while ANER tags only "Damascus" as GPE.
    • Wojood marks the full span King Abdallah as PERS, while ANER and OntoNotes mark only the name.
    • Wikidata and Wojood Relations attach hasConflictWith to different pairs of entity types.
  • Evaluation is re-annotation of AQMAR into the Wojood tagset with GPT-4o:
    • zero-shot F1 was below 0.30 in both variants (0.2733 and 0.2913);
    • few-shot with seven demonstrations reached F1 0.4991 without the ontology and 0.5504 with it.
  • In zero-shot mode, the ontology helped slightly more when the original AQMAR entities were not given in the prompt.
  • Limitations: the MISC tag of ANERcorp and AQMAR is left out because its definition differs between the two corpora, and only one LLM (GPT-4o) was tested.

Interesting ideas

  • I1. Map every tagset to one shared ontology, linked by equivalence and subclass axioms, instead of writing a converter for each pair of schemes; adding a new scheme then costs one alignment.
  • I2. The subclass relation makes mapping asymmetric: fine-to-coarse can be done with simple deterministic rules, while coarse-to-fine needs sentence context and so a model.
  • I3. Put the formal schema itself into the LLM prompt, require structured JSON output with no prefixes, and let the model answer "None" when unsure; this reduces invented or arbitrary labels.
  • I4. Measure a mapping by re-annotating an out-of-domain corpus and running ablations: with and without the ontology, and with and without the source labels.
  • I5. Domain-range constraints on relations show when two resources disagree about which entity types a relation may connect.

For Mova

How these ideas could serve Mova, our small open language model — its files, stages and tests:

  • I1 (world): Our WordNet, FrameNet and Wikidata layers could each be tied to one central type hierarchy with explicit equivalent and subclass links, so a concept from any source resolves to a single node.
  • I2 (tree): When converting between UD dialects or treebank schemes, split deterministic collapsing rules from the cases that need a classifier or an LLM to decide.
  • I3 (learn): LLM teacher tiers could receive the exact role or frame inventory as a formal spec, and abstentions could be sent to other witnesses rather than forced into a label.
  • I4 (learn): Run the same knock-out tests on our teacher prompts, for example dropping the frame definitions or the original parse, to show which input actually helps.
  • I5 (frames): VerbNet selectional restrictions could serve as such constraints, flagging extracted who-did-what-to-whom frames whose argument types break them.

Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.

 Institutions

🎓Hamad bin Khalifa UniversityDoha · Qatar144 works · 14,175 citations🎓Birzeit UniversityBirzeit · Palestine38 works · 798 citations

 Fields, topics and concepts

📚 Shelves 6The field of science this work belongs to (OpenAlex fields).Computer Science · 18,614 worksArtificial Intelligence · 17,965 worksArts and Humanities · 17,450 worksLanguage and Linguistics · 17,296 worksBiochemistry, Genetics and Molecular Biology · 263 worksMolecular Biology · 260 works

🧭 Topics 4Research subjects and languages the work is about.Language models & pretraining · 3,894 worksSemitic languages · 3,066 worksInformation extraction · 175 worksKnowledge graphs & ontologies · 211 works

Cite

Cite as Alaa Aljabari, Nagham Hamad, Mohammed Khalilia and Mustafa Jarrar. 2025. WojoodOntology: Ontology-Driven LLM Prompting for Unified Information Extraction Tasks. Proceedings of The Third Arabic Natural Language Processing Conference.

The paper itself, not this page. BibTeX from the library's record of the work. DOI 10.18653/v1/2025.arabicnlp-main.14

@inproceedings{aljabari2025wojoodontology,
  title = {{WojoodOntology: Ontology-Driven LLM Prompting for Unified Information Extraction Tasks}},
  author = {Alaa Aljabari, Nagham Hamad, Mohammed Khalilia and Mustafa Jarrar},
  booktitle = {Proceedings of The Third Arabic Natural Language Processing Conference},
  year = {2025},
  doi = {10.18653/v1/2025.arabicnlp-main.14},
  url = {https://doi.org/10.18653/v1/2025.arabicnlp-main.14}
}

References

36 references, 17 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

  1. Moustafa Al-Hajj and Mustafa Jarrar. 2021. ArabGloss-BERT: Fine-Tuning BERT on Context-Gloss Pairs for WSD. In Proceedings of the International Conference on Recent Advances in Natural Language Processing (RANLP 2021), pages 40–48, Online. IN-COMA Ltd. find
  2. Alaa Aljabari, Lina Duaibes, Mustafa Jarrar, and Mohammed Khalilia. 2024. Event-Arguments Extraction Corpus and Modeling using BERT for Arabic. In Proceedings of the Second Arabic Natural Language Processing Conference (ArabicNLP 2024), Bangkok, Thailand. Association for Computational Linguistics.
  3. Alaa Aljabari, Mohammed Khalilia, and Mustafa Jarrar. 2025. Wojood Relations : Arabic Relation Extraction Corpus and Modeling. In Proceedings of the 2025 Conference on Empirical Methods in Natural Language Processing, Suzhou, China. Association for Computational Linguistics. find
  4. Sylvio Barbon Junior, Paolo Ceravolo, Sven Groppe, Mustafa Jarrar, Samira Maghool, Florence Sèdes, Soror Sahri, and Maurice Van Keulen. 2024. Are Large Language Models the New Interface for Data Pipelines? In Proceedings of the International Workshop on Big Data in Emergent Distributed Environments, BiDEDE ’24, New York, NY, USA. Association for Computing Machinery. ↗
  5. Yassine Benajiba, Paolo Rosso, and José Miguel BenedíRuiz. 2007a. Anersys: An arabic named entity recognition system based on maximum entropy. In Computational Linguistics and Intelligent Text Processing, pages 143–153, Berlin, Heidelberg. Springer Berlin Heidelberg. find
  6. Yassine Benajiba, Paolo Rosso, and José Miguel Benedíruiz. 2007b. Anersys: An arabic named entity recognition system based on maximum entropy. In Computational Linguistics and Intelligent Text Processing: 8th International Conference, CICLing 2007, Mexico City, Mexico, February 18-24, 2007. Proceedings 8, pages 143–153. Springer. find
  7. George Doddington, Alexis Mitchell, Mark Przybocki, Lance Ramshaw, Stephanie Strassel, and Ralph Weischedel. 2004. The automatic content extraction (ACE) program – tasks, data, and evaluation. In Proceedings of the Fourth International Conference on (LREC’04), Lisbon, Portugal. European Language Resources Association (ELRA).
  8. Xiaohan Feng, Xixin Wu, and Helen Meng. 2024. Ontology-grounded automatic knowledge graph construction by LLM under wikidata schema. In Proceedings of the KDD Workshop on Human-Interpretable AI, volume 3841 of CEUR Workshop Proceedings, pages 117–135. CEUR-WS.org. ↗
  9. Honghao Gui, Lin Yuan, Hongbin Ye, Ningyu Zhang, Mengshu Sun, Lei Liang, and Huajun Chen. 2024. IEPile: Unearthing large scale schema-conditioned information extraction corpus. In Proceedings of the 62nd Annual Meeting of ACL (Volume 2: Short Papers), pages 127–146, Bangkok, Thailand. Association for Computational Linguistics. find
  10. Karim El Haff, Mustafa Jarrar, Tymaa Hammouda, and Fadi Zaraket. 2022. Curras + Baladi: Towards a Levantine Corpus. In Proceedings of the International Conference on Language Resources and Evaluation(LREC 2022), Marseille, France. ↗
  11. Nagham Hamad, Mohammed Khalilia, and Mustafa Jarrar. 2025. Konooz: Multi-domain Multi-dialect Corpus for Named Entity Recognition. In Proceedings of the 63rd Annual Meeting of the Association for Computational Linguistics, pages 0–0, Vienna, Austria. Association for Computational Linguistics.
  12. Pere-Lluís Huguet Cabot, Simone Tedeschi, Axel-Cyrille Ngonga Ngomo, and Roberto Navigli. 2023. RED fm : a filtered and multilingual relation extraction dataset. In Proceedings of the 61st Annual Meeting of ACL (Volume 1: Long Papers), pages 4326–4343, Toronto, Canada. ACL. find
  13. Mustafa Jarrar. 2021. The Arabic Ontology - An Arabic Wordnet with Ontologically Clean Content. Applied Ontology Journal, 16(1):1–26. ↗
  14. Mustafa Jarrar, Muhammad Abdul-Mageed, Mohammed Khalilia, Bashar Talafha, AbdelRahim Elmadany, Nagham Hamad, and Alaa’ Omar. 2023a. WojoodNER 2023: The First Arabic Named Entity Recognition Shared Task. In Proceedings of the 1st Arabic Natural Language Processing Conference (ArabicNLP), Part of the EMNLP 2023, pages 748– 758. ACL.
  15. Mustafa Jarrar, Anton Deik, and Bilal Faraj. 2011. Ontology-based data and process governance framework -the case of e-government interoperability in palestine. In Proceedings of the IFIP International Symposium on Data-Driven Process Discovery and Analysis (SIMPDA’11), pages 83–98. find
  16. Mustafa Jarrar, Nizar Habash, Faeq Alrimawi, Diyam Akra, and Nasser Zalmout. 2017. Curras: An annotated corpus for the palestinian arabic dialect. Journal Language Resources and Evaluation, 51(3):2-s2.0-85001544989. ↗
  17. Mustafa Jarrar, Nizar Habash, Mo El-Haj, Amal Haddad Haddad, Zeina Jallad, Camille Mansour, Diana Allan, Paul Rayson, Tymaa Hammouda, and Sanad Malaysha, editors. 2025. Proceedings of the first International Workshop on Nakba Narratives as Language Resources. Association for Computational Linguistics, Abu Dhabi, UAE. find
  18. Mustafa Jarrar, Nagham Hamad, Mohammed Khalilia, Bashar Talafha, AbdelRahim Elmadany, and Muhammad Abdul-Mageed. 2024. WojoodNER 2024: The Second Arabic Named Entity Recognition Shared Task. In Proceedings of the Second Arabic Natural Language Processing Conference (ArabicNLP 2024), Bangkok, Thailand. Association for Computational Linguistics.
  19. Mustafa Jarrar and Tymaa Hasanain Hammouda. 2024. Qabas: An Open-Source Arabic Lexicographic Database. In Proceedings of the 2024 Joint International Conference on Computational Linguistics, Language Resources and Evaluation (LREC-COLING 2024), pages 13363–13370, Torino, Italy. ELRA and ICCL.
  20. Mustafa Jarrar, Mohammed Khalilia, and Sana Ghanem. 2022. Wojood: Nested Arabic Named Entity Corpus and Recognition using BERT. In Proceedings of the International Conference on Language Resources and Evaluation (LREC 2022), Marseille, France.
  21. Mustafa Jarrar, Sanad Malaysha, Tymaa Hammouda, and Mohammed Khalilia. 2023b. SALMA: Arabic Sense-annotated Corpus and WSD Benchmarks. In Proceedings of the 1st Arabic Natural Language Processing Conference (ArabicNLP), Part of the EMNLP 2023, pages 359–369. ACL.
  22. Mustafa Jarrar, Fadi Zaraket, Tymaa Hammouda, Daanish Masood Alavi, and Martin Waahlisch. 2023c. Lisan: Yemeni, Irqi, Libyan, and Sudanese Arabic Dialect Copora with Morphological Annotations. In The 20th IEEE/ACS International Conference on Computer Systems and Applications (AICCSA), pages 1–7. IEEE. find
  23. Mohammed Khalilia, Sanad Malaysha, Reem Suwaileh, Mustafa Jarrar, Alaa Aljabari, Tamer Elsayed, and Imed Zitouni. 2024. ArabicNLU 2024: The First Arabic Natural Language Understanding Shared Task. In Proceedings of the Second Arabic Natural Language Processing Conference (ArabicNLP 2024), Bangkok, Thailand. Association for Computational Linguistics.
  24. Haneen Liqreina, Mustafa Jarrar, Mohammed Khalilia, Ahmed Oumar El-Shangiti, and Muhammad Abdul-Mageed. 2023. Arabic Fine-Grained Entity Recognition. In Proceedings of the 1st Arabic Natural Language Processing Conference (ArabicNLP), Part of the EMNLP 2023, pages 310–323. ACL.
  25. Martin Marinov, Youcef Benkhedda, Goran Nenadic, and Riza Batista-Navarro. 2024. Relation extraction for constructing knowledge graphs: Enhancing the searchability of community-generated digital content (CGDC) collections. In Workshop on Deep Learning and Large Language Models for Knowledge Graphs. find
  26. Behrang Mohit, Nathan Schneider, Rishav Bhowmick, Kemal Oflazer, and Noah A Smith. 2012a. Recall-oriented learning of named entities in arabic wikipedia. In Proceedings of EACL 2012, pages 162–173.
  27. Behrang Mohit, Nathan Schneider, Rishav Bhowmick, Kemal Oflazer, and Noah A. Smith. 2012b. Recall-oriented learning of named entities in arabic wikipedia. In Proceedings of ECAL 2012, EACL ’12, page 162–173, USA. ACL.
  28. Amal Nayouf, Mustafa Jarrar, Fadi zaraket, Tymaa Hammouda, and Mohamad-Bassam Kurdy. 2023. Nâbra: Syrian Arabic Dialects with Morphological Annotations. In Proceedings of the 1st Arabic Natural Language Processing Conference (ArabicNLP), Part of the EMNLP 2023, pages 12–23. ACL.
  29. Debora Nozza, Pikakshi Manchanda, Elisabetta Fersini, Matteo Palmonari, and Enza Messina. 2021. Learn-ingtoadapt with word embeddings: Domain adaptation of named entity recognition systems. Inf. Process. Manag., 58:102537. find
  30. Ossama Obeid, Nasser Zalmout, Salam Khalifa, Dima Taji, Mai Oudah, Bashar Alhafni, Go Inoue, Fadhl Eryani, Alexander Erdmann, and Nizar Habash. 2020. CAMeL tools: An open source python toolkit for Arabic natural language processing. In Proceedings of the Twelfth Language Resources and Evaluation Conference, pages 7022–7032, Marseille, France. European Language Resources Association.
  31. Sai Teja Potu, Rachana Niranjan Murthy, Akhil Thomas, Lokesh Mishra, Natalie Prange, and Ali Riza Durmaz. 2025. Ontology-conformal recognition of materials entities using language models. Scientific Reports, 15(1):1–16. ↗
  32. Giuseppe Rizzo and Raphaël Troncy. 2012. NERD: A framework for unifying named entity recognition and disambiguation extraction tools. In Proceedings of the Demonstrations at the 13th Conference of the European Chapter of ACL, pages 73–76, Avignon, France. ACL. find
  33. Oscar Sainz, Iker García-Ferrero, Rodrigo Agerri, Oier Lopez de Lacalle, German Rigau, and Eneko Agirre. 2024. GoLLIE: Annotation guidelines improve zero-shot information-extraction. In The Twelfth International Conference on Learning Representations. ↗
  34. Alessandro Seganti, Klaudia Firl ag, Helena Skowronska, Michał Satława, and Piotr Andruszkiewicz. 2021. Multilingual entity and relation extraction dataset and model. In Proceedings of the 16th Conference of the European Chapter of the Association for Computational Linguistics: Main Volume, pages 1946–1955, Online. Association for Computational Linguistics. find
  35. Ralph M. Weischedel, Eduard H. Hovy, Mitchell P. Marcus, and Martha Palmer. 2017. Ontonotes : A large training corpus for enhanced processing. find
  36. Yuming Yang, Wantong Zhao, Caishuang Huang, Junjie Ye, Xiao Wang, Huiyuan Zheng, Yang Nan, Yuran Wang, Xueying Xu, Kaixin Huang, Yunke Zhang, Tao Gui, Qi Zhang, and Xuanjing Huang. 2025. Beyond boundaries: Learning a universal entity taxonomy across datasets and languages for open named entity recognition. In Proceedings of the 31st International Conference on Computational Linguistics, pages 10902–10923, Abu Dhabi, UAE. ACL. ↗

Updated

↑ back to top

This page in Markdown → https://md.research.elamur.ai/free/en/papers/aljabari-2025-wojoodontology-ontology-driven-llm-prompting/