Elamur Research⌕
☰
home / papers / 2021 / SERAG: Semantic Entity Retrieval from Arabic Knowledge Graphs

SERAG: Semantic Entity Retrieval from Arabic Knowledge Graphs

Paper · 2021 · Saher Esmeir

Summary

Neural entity-retrieval systems for knowledge graphs had been built and tested almost only on English. This paper adapts KEWER to Arabic and calls the result SERAG. It also creates the first non-English version of DBpedia-Entity v2 (DEv2), the standard entity-search benchmark.

To build the benchmark, English queries were translated by native Arabic speakers. The relevance judgements were carried over by mapping English entities to Arabic ones through DBpedia inter-language links. Queries that lost too many relevant entities in this mapping were dropped.

SERAG works in two stages:

  • BM25 first picks 1000 candidate entities.
  • These are then reranked with a word2vec model trained on random walks over the Arabic DBpedia graph. The final score linearly mixes the BM25 score with cosine similarity between the query and the entity embeddings.

Because the walks connect entities several hops apart, the model can capture indirect relevance that no single entity document states. SERAG beats BM25 in every query category, and the overall gain is statistically significant. The work matters because it supplies a reusable Arabic benchmark and documents why Arabic KG retrieval is harder.

Key points

  • Arabic DEv2 keeps 139 of the 467 DEv2 queries: SemSearch-ES 7, INEX-LD 23, QALD2 56, ListSearch 53. Only 3025 of 16,700 relevant entities could be mapped to Arabic. A query was kept if it had at least 10 mapped relevant entities, or if at least half of its relevant entities were mapped.
  • The source graph is DBpedia 2015-10. Its Arabic chapter has about 368K entities with abstracts, roughly 8% of the English chapter. At least 30% of Arabic entities have no English counterpart.
  • The setup is unsupervised, because the data is too small for tuning:
    • 100 walks per entity, each of length 10;
    • an entity ID is kept in a walk with probability p = 0.1, and otherwise replaced by its surface label;
    • mixing weight β = 0.9;
    • gensim default settings.
  • Overall nDCG@10 rose from 0.172 (BM25) to 0.226 (SERAG), and nDCG@100 from 0.248 to 0.303 (paired t-test, α = 0.05).
  • Bpref, a measure for incomplete judgements, also favoured SERAG: 0.17 vs 0.142 at top 10, and 0.336 vs 0.314 at top 100.

Interesting ideas

  • I1. Random walks over the graph are turned into pseudo-sentences in which each entity randomly appears either as its ID or as its textual label, so one word2vec space holds both words and graph nodes.
  • I2. Relevance is often justified by a multi-hop path in the graph rather than by any single document, as in the example of a museum linked to an Arab capital through a country.
  • I3. When gold judgements are known to be incomplete, the paper adds bpref, which only compares judged relevant items against judged non-relevant ones, so unjudged items are not counted as wrong.
  • I4. A benchmark is projected to a new language by mapping gold labels through cross-language links, with an explicit coverage threshold that removes items damaged by the mapping.
  • I5. The same orthographic normalization is applied to both the knowledge-graph text and the incoming queries; for Arabic this means unifying visually identical letters and stripping the definite article.

For Mova

How these ideas could serve Mova, our small open language model — its files, stages and tests:

  • I1 (world): This could align WordNet/Wikidata node IDs with the lemmas that name them, letting text mentions be linked to ontology entries through the same compiled resource.
  • I2 (answer): Readers could return the explicit graph path as the justification for an answer, keeping every step explainable instead of relying on one matching sentence.
  • I3 (learn): Evaluation against silver or partially annotated data could use a bpref-style measure, so that correct but unannotated outputs do not look like errors.
  • I4 (back): Test sets for translation through the meta-world could be built the same way, keeping only items whose entities survive the cross-lingual mapping well enough.
  • I5 (tokens): The tokenizer and lemmatizer could share one normalization layer with the lexicon/KG builder, covering apostrophe variants, hyphenation and Unicode look-alikes, so that lookups never fail on invisible differences.

Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.

 Institutions

🏢BloombergNew York · United States7 works · 1,559 citations

 Fields, topics and concepts

📚 Shelves 5The field of science this work belongs to (OpenAlex fields).Computer Science · 18,614 worksArtificial Intelligence · 17,965 worksArts and Humanities · 17,450 worksLanguage and Linguistics · 17,296 worksInformation Systems · 1,730 works

🧭 Topics 3Research subjects and languages the work is about.Semitic languages · 3,066 worksInformation retrieval & search · 552 worksKnowledge graphs & ontologies · 211 works

Cite

Cite as Saher Esmeir. 2021. SERAG: Semantic Entity Retrieval from Arabic Knowledge Graphs. Proceedings of the Sixth Arabic Natural Language Processing Workshop.

The paper itself, not this page. BibTeX from the library's record of the work.

@inproceedings{esmeir2021serag,
  title = {{SERAG: Semantic Entity Retrieval from Arabic Knowledge Graphs}},
  author = {Saher Esmeir},
  booktitle = {Proceedings of the Sixth Arabic Natural Language Processing Workshop},
  year = {2021},
  doi = {10.5281/zenodo.4560652},
  url = {https://doi.org/10.5281/zenodo.4560652}
}

References

33 references, 9 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

  1. Alaa W. Abu Taha. 2015. An ontology-based arabic question answering system submitted. Master’s thesis, Faculty of Information Technology, Islamic University of Gaza. find
  2. Haytham Al-Feel. 2013. A step towards the arabic dbpedia. International Journal of Computer Applications, 80:27–33. ↗
  3. (SuCCESS’16) co-located with the 12th International Conference on Semantic Systems (SEMAN-TiCS 2016), Leipzig, Germany, September 12-15, 2016, volume 1695 of CEUR Workshop Proceedings. CEUR-WS.org. find
  4. Ali Fadel, Ibraheem Tuffaha, Bara’ Al-Jawarneh, and Mahmoud Al-Ayyoub. 2019. Arabic text diacritization using deep neural networks. CoRR, abs/1905.01965. ↗
  5. Ali Albarghothi, Feras Khater, and Khaled Shaalan. 2017. Arabic question answering using ontology. Procedia Computer Science, 117:183–191. find
  6. Michael Färber, Frederic Bartscherer, Carsten Menne, and Achim Rettinger. 2018. Linked data quality of dbpedia, freebase, opencyc, wikidata, and yago. Semantic Web, 9:77–129. ↗
  7. Wissam Antoun, Fady Baly, and Hazem Hajj. 2020. AraBERT: Transformer-based model for Arabic language understanding. In Proceedings of the 4th Workshop on Open-Source Arabic Corpora and Processing Tools, with a Shared Task on Offensive Language Detection, pages 9–15, Marseille, France. European Language Resource Association. ↗
  8. Dieter Fensel, Umutcan Şimşek, Kevin Angele, Elwin Huaman, Elias Kärle, Oleksandra Panasiuk, Ioan Toma, Jürgen Umbrich, and Alexander Wahler. 2020. Introduction: What Is a Knowledge Graph?, pages 1–10. Springer International Publishing, Cham. find
  9. Alia O. Bahanshal and Hend S. Al-Khalifa. 2013. Toward recipes for arabic dbpedia. In Proceedings of International Conference on Information Integration and Web-Based Applications & Services, II-WAS ’13, pages 331–335, New York, NY, USA. Association for Computing Machinery. find
  10. Chris Buckley and Ellen M. Voorhees. 2004. Retrieval evaluation with incomplete information. In Proceedings of the 27th Annual International ACM SIGIR Conference on Research and Development in Information Retrieval, SIGIR ’04, pages 25–32, New York, NY, USA. Association for Computing Machinery. find
  11. H. Cai, V. W. Zheng, and K. Chang. 2018. A comprehensive survey of graph embedding: Problems, techniques, and applications. IEEE Transactions on Knowledge & Data Engineering, 30(09):1616– 1637. ↗
  12. Nilesh Chakraborty, D. Lukovnikov, Gaurav Maheshwari, Priyansh Trivedi, Jens Lehmann, and Asja Fischer. 2019. Introduction to neural network based approaches for question answering over knowledge graphs. ArXiv, abs/1907.09361. ↗
  13. Danqi Chen, Adam Fisch, Jason Weston, and Antoine Bordes. 2017. Reading Wikipedia to answer open-domain questions. In Proceedings of the 55th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 1870– 1879, Vancouver, Canada. Association for Computational Linguistics. ↗
  14. Lisa Ehrlinger and Wolfram Wöß. 2016. Towards a definition of knowledge graphs. In Joint Proceedings of the Posters and Demos Track of the 12th International Conference on Semantic Systems - SEMANTiCS2016 and the 1st International Workshop on Semantic Change & Evolving Semantics Emma J. Gerritse, Faegheh Hasibi, and Arjen P. de Vries. 2020. Graph-embedding empowered entity retrieval. Advances in Information Retrieval, pages 97–110. find
  15. Nizar Habash, Houda Bouamor, and Christine Chung. 2019. Automatic gender identification and reinflection in Arabic. In Proceedings of the First Workshop on Gender Bias in Natural Language Processing, pages 155–165, Florence, Italy. Association for Computational Linguistics.
  16. Nizar Y. Habash. 2010. Introduction to arabic natural language processing. Synthesis Lectures on Human Language Technologies, 3(1):1–187. ↗
  17. Faegheh Hasibi, Fedor Nikolaev, Chenyan Xiong, Krisztian Balog, Svein Erik Bratsberg, Alexander Kotov, and Jamie Callan. 2017. Dbpedia-entity v2: A test collection for entity search. In Proceedings of the 40th International ACM SIGIR Conference on Research and Development in Information Retrieval, SIGIR ’17, pages 1265–1268. ACM. find
  18. Ahmed Salama Ismail, Haytham Al-Feel, and Hoda M. O. Mokhtar. 2016. Introducing a new arabic endpoint for dbpedia internationalization project. In Proceedings of the 20th International Database Engineering & Applications Symposium, IDEAS ’16, pages 284–289, New York, NY, USA. Association for Computing Machinery. find
  19. Giorgos Kadilierakis, Pavlos Fafalios, Panagiotis Papadakos, and Yannis Tzitzikas. 2020. Keyword search over rdf using document-centric information retrieval systems. In The Semantic Web, pages 121– 137, Cham. Springer International Publishing. find
  20. Guma Abdulkhader Lakshen, Valentina Janev, and Sanja Vraneš. 2018. Challenges in quality assessment of arabic dbpedia. In Proceedings of the 8th International Conference on Web Intelligence, Mining and Semantics, WIMS ’18, New York, NY, USA. Association for Computing Machinery. find
  21. Tomas Mikolov, Ilya Sutskever, Kai Chen, Greg Corrado, and Jeffrey Dean. 2013. Distributed representations of words and phrases and their compositionality. In Proceedings of the 26th International Conference on Neural Information Processing Systems - Volume 2, NIPS’13, pages 3111–3119, Red Hook, NY, USA. Curran Associates Inc. ↗
  22. Abu Bakr Mohammad, Kareem Eissa, and Samhaa El-Beltagy. 2017. Aravec: A set of arabic word embedding models for use in arabic nlp. Procedia Computer Science, 117:256–265. find
  23. Hussein Mozannar, Elie Maamary, Karl El Hajal, and Hazem Hajj. 2019. Neural Arabic question answering. In Proceedings of the Fourth Arabic Natural Language Processing Workshop, pages 108–118, Florence, Italy. Association for Computational Linguistics.
  24. Shahrzad Naseri, John Foley, J. Allan, and Brendan T. O’Connor. 2018. Exploring summary-expanded entity embeddings for entity retrieval. In CIKM Workshops. find
  25. Fedor Nikolaev and Alexander Kotov. 2020. Joint word and entity embeddings for entity retrieval from a knowledge graph. In Advances in Information Retrieval, pages 141–155, Cham. Springer International Publishing. find
  26. Ossama Obeid, Nasser Zalmout, Salam Khalifa, Dima Taji, Mai Oudah, Bashar Alhafni, Go Inoue, Fadhl Eryani, Alexander Erdmann, and Nizar Habash. 2020. CAMeL tools: An open source python toolkit for Arabic natural language processing. In Proceedings of the 12th Language Resources and Evaluation Conference, pages 7022–7032, Marseille, France. European Language Resources Association.
  27. Miriam Redi, Martin Gerlach, Isaac Johnson, Jonathan Morgan, and Leila Zia. 2021. A taxonomy of knowledge gaps for wikimedia projects (second draft). ArXiv. find
  28. Radim Řehůřek and Petr Sojka. 2010. Software Framework for Topic Modelling with Large Corpora. In Proceedings of the LREC 2010 Workshop on New Challenges for NLP Frameworks, pages 45–50, Valletta, Malta. ELRA. find
  29. Ali Safaya, Moutasem Abdullatif, and Deniz Yuret. 2020. Kuisail at semeval-2020 task 12: Bert-cnn for offensive speech identification in social media. ArXiv, abs/2007.13184. ↗
  30. Heba Samy, Ehab E. Hassanein, and Khaled Shaalan. 2019. Arabic question answering: A study on challenges, systems, and techniques. International Journal of Computer Applications, 181(44):6–14. ↗
  31. Khaled Shaalan, Sanjeera Siddiqui, Manar Alkhatib, and Azza Monem. 2018. Challenges in Arabic Natural Language Processing, chapter Chapter 3. World Scientific Publishing Company. find
  32. Ikuya Yamada, Akari Asai, Jin Sakuma, Hiroyuki Shindo, Hideaki Takeda, Yoshiyasu Takefuji, and Yuji Matsumoto. 2020. Wikipedia2Vec: An efficient toolkit for learning and visualizing the embeddings of words and entities from Wikipedia. In Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing: System Demonstrations, pages 23–30, Online. Association for Computational Linguistics. find
  33. Nikita Zhiltsov, Alexander Kotov, and Fedor Nikolaev. 2015. Fielded sequential dependence model for ad-hoc entity retrieval in the web of data. In Proceedings of the 38th International ACM SIGIR Conference on Research and Development in Information Retrieval, SIGIR ’15, pages 253–262, New York, NY, USA. Association for Computing Machinery. find

Updated

↑ back to top

This page in Markdown → https://md.research.elamur.ai/free/en/papers/esmeir-2021-serag-semantic-entity-retrieval-arabic/