Elamur Research⌕
☰
home / papers / 2017 / Semantic Similarity of Arabic Sentences with Word Embeddings

Semantic Similarity of Arabic Sentences with Word Embeddings

Paper · 2017 · El Moatez Billah Nagoudi and Didier Schwab

Summary

The paper tackles automatic scoring of how close in meaning two short Arabic sentences are. This is useful for retrieval, plagiarism detection and translation, and Arabic has fewer hand-built lexical resources for it than English. The authors use a pretrained Arabic CBOW model from Zahran et al. (2015), built from more than 5.8 billion words of mixed sources. Each sentence becomes one vector, the sum of its word vectors, and two sentences are compared by cosine. They test three variants: a plain unweighted sum, a sum weighted by IDF computed on the BBC and CNN Arabic corpus, and a sum weighted by part-of-speech class using the Gahbiche-Braham et al. CRF tagger. Text is normalized before scoring. On 750 sentence pairs from the MSR-Video corpus, Pearson correlation with human ratings rises from 72.33% (unweighted) to 78.20% (IDF) and 79.69% (POS). The work shows that a cheap, transparent weighting of word contributions gives a clear gain over naive vector addition.

Key points

  • The word vectors come from the Arabic CBOW model of Zahran et al. (2015). Its settings are 300 dimensions, window 5, sub-sampling 1e-5, no hierarchical softmax, 10 negative samples and a minimum frequency of 100.
  • The training data combines Arabic Wikipedia, BBC and CNN Arabic, MultiUN, Arabic Gigaword, LDC newswire, the KSU corpus, Quran text, KDE4 files and others.
  • IDF is computed at the sentence level over the background corpus as log(S / W_S). The example POS weights are proper noun 0.7, noun 0.5, verb 0.4, adjective 0.3 and preposition 0.1.
  • Preprocessing removes stop words, punctuation, diacritics and non-letters. It also unifies alef variants and ta marbuta, fixes final ya before hamza, and maps all digits to a single "Num" token.
  • Pearson correlation on 750 MSR-Video pairs is 72.33% (basic), 78.20% (IDF) and 79.69% (POS). The method's main failure is pairs that share words but differ in meaning.

Interesting ideas

  • I1. Weight each word's contribution by its grammatical class, so that names and nouns count more than prepositions when comparing two sentences. Cheap structural knowledge beats treating all tokens equally.
  • I2. Estimate how informative a word is from a separate background corpus of the same genre, counting sentences rather than documents, and use that as a discount for very common words.
  • I3. Collapse all numerals into one placeholder token before comparison, so that different numbers do not distort lexical matching.
  • I4. The authors openly note that their scoring breaks on pairs with heavy word overlap but different meaning, which is exactly where additive representations fail.
  • I5. Each method is illustrated with one toy sentence pair carried through every step with explicit intermediate numbers, which makes the score easy to audit by hand.

For Mova

How these ideas could serve Mova, our small open language model — its files, stages and tests:

  • I1 (frames): When matching two frames, weight agreement on core arguments and proper-named entities above agreement on function-word slots, and expose those weights in the explanation.
  • I2 (world): Store per-lemma sentence-level IDF in the compiled knowledge layer, so entity and event matching can down-rank generic words like "thing" or "go".
  • I3 (tokens): Have the tokenizer emit a normalized NUM class alongside the surface value, so downstream matching can ignore the digit string while the world layer keeps the exact quantity.
  • I4 (learn): Build a negative-control set of high-overlap, different-meaning sentence pairs, and require Mova's tree-and-frame comparison to separate them where a vector-sum baseline cannot.
  • I5 (answer): Present similarity or entailment judgments as a short worked trace of matched entities and roles with their weights, so a student or tester can follow the decision.

Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.

 Institutions

🎓Université Grenoble AlpesSaint-Martin-d'Hères · France20 works · 8,059 citations🔬Laboratoire d'Informatique de GrenobleGrenoble · France21 works · 688 citations🎓Amar Telidji University of LaghouatLaghouat · Algeria8 works · 121 citations

 Fields, topics and concepts

📚 Shelves 4The field of science this work belongs to (OpenAlex fields).Computer Science · 16,667 worksArtificial Intelligence · 16,194 worksArts and Humanities · 16,039 worksLanguage and Linguistics · 15,891 works

🧭 Topics 1Research subjects and languages the work is about.Semitic languages · 2,908 works

Cite

Cite as El Moatez Billah Nagoudi and Didier Schwab. 2017. Semantic Similarity of Arabic Sentences with Word Embeddings. Proceedings of the Third Arabic Natural Language Processing Workshop.

The paper itself, not this page. BibTeX from the library's record of the work. DOI 10.18653/v1/w17-1303

@inproceedings{nagoudi2017semantic,
  title = {{Semantic Similarity of Arabic Sentences with Word Embeddings}},
  author = {El Moatez Billah Nagoudi and Didier Schwab},
  booktitle = {Proceedings of the Third Arabic Natural Language Processing Workshop},
  year = {2017},
  doi = {10.18653/v1/w17-1303},
  url = {https://doi.org/10.18653/v1/w17-1303}
}

References

23 references, 7 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

  1. Yoshua Bengio, Réjean Ducharme, Pascal Vincent, and Christian Jauvin. 2003. A neural probabilistic language model. Journal of machine learning research, 3(Feb):1137–1155.
  2. Tomas Mikolov, Ilya Sutskever, Kai Chen, Greg S Corrado, and Jeff Dean. 2013b. Distributed representations of words and phrases and their compositionality. In Advances in neural information processing systems, pages 3111–3119. ↗
  3. Yu Chen and Andreas Eisele. 2012. Multiun v2: Un documents with multilingual alignments. In LREC, pages 2500–2504. find
  4. Jörg Tiedemann. 2012. Parallel data, tools and interfaces in opus. In LREC, volume 2012, pages 2214– 2218. find
  5. Tomas Mikolov, Wen-tau Yih, and Geoffrey Zweig. 2013c. Linguistic regularities in continuous space word representations. In Hlt-naacl, volume 13, pages 746–751.
  6. Joseph Turian, Lev Ratinov, and Yoshua Bengio. 2010. Word representations: a simple and general method for semi-supervised learning. In Proceedings of the 48th annual meeting of the association for computational linguistics, pages 384–394. Association for Computational Linguistics.
  7. George A Miller. 1995. Wordnet: a lexical database for english. Communications of the ACM, 38(11):39–41. ↗
  8. Andriy Mnih and Geoffrey E Hinton. 2009. A scalable hierarchical distributed language model. In D. Koller, D. Schuurmans, Y. Bengio, and L. Bottou, editors, Advances in Neural Information Processing Systems 21, pages 1081–1088. Curran Associates, Inc. find
  9. Peter D Turney and Patrick Pantel. 2010. From frequency to meaning: Vector space models of semantics. Journal of artificial intelligence research, 37:141–188. ↗
  10. MSR-video. 2016. Microsoft research video corpus, https://www.microsoft.com/en-us/download/ details.aspx?id=52422, (accessed january 21,2017). ↗
  11. WikiAr. 2006. Arabic wikipedia corpus, http://linguatools.org/tools/corpora/wikipedia-monolingual-corpora/, (accessed january 21,2017). ↗
  12. Roberto Navigli and Simone Paolo Ponzetto. 2012. Babelnet: The automatic construction, evaluation and application of a wide-coverage multilingual semantic network. Artificial Intelligence, 193:217– 250. ↗
  13. William E Winkler. 1999. The state of record linkage and current research problems. In Statistical Research Division, US Census Bureau. Citeseer. find
  14. Mohamed A Zahran, Ahmed Magooda, Ashraf Y Mahgoub, Hazem Raafat, Mohsen Rashwan, and Amir Atyia. 2015. Word representations in vector space and their applications for arabic. In International Conference on Intelligent Text Processing and Computational Linguistics, pages 430–443. Springer. find
  15. Saul B Needleman and Christian D Wunsch. 1970. A general method applicable to the search for similarities in the amino acid sequence of two proteins. Journal of molecular biology, 48(3):443–453. ↗
  16. Jeffrey Pennington, Richard Socher, and Christopher D Manning. 2014. Glove: Global vectors for word representation. In EMNLP, volume 14, pages 1532– 1543. ↗
  17. Quran. 2007. Raw quran text, http://tanzil.net/ download/, (accessed january 20,2017). ↗
  18. Hazem M Raafat, Mohamed A Zahran, and Mohsen Rashwan. 2013. Arabase-a database combining different arabic resources with lexical and semantic information. In KDIR/KMIS, pages 233–240. find
  19. Motaz K Saad and Wesam Ashour. 2010. Osac: Open source arabic corpora. In 6th ArchEng Int. Symposiums, EEECS, volume 10. ↗
  20. Gerard Salton and Christopher Buckley. 1988. Term-weighting approaches in automatic text retrieval. Information processing & management, 24(5):513– 523. ↗
  21. Didier Schwab. 2005. Approche hybride-lexicale et thématique-pour la modélisation, la détection et lex-ploitation des fonctions lexicales en vue de lanalyse sémantique de texte. Ph.D. thesis, Université Montpellier II. find
  22. Gilles Sérasset. 2015. Dbnary: Wiktionary as a lemon-based multilingual lexical resource in rdf. Semantic Web, 6(4):355–361. ↗
  23. Jörg Tiedemann. 2009. News from opus-a collection of multilingual parallel corpora with tools and interfaces. In Recent advances in natural language processing, volume 5, pages 237–248. ↗

Updated

↑ back to top

This page in Markdown → https://md.research.elamur.ai/free/en/papers/nagoudi-2017-semantic-similarity-arabic-sentences-word/