Elamur Research⌕
☰
home / papers / 2023 / Context-Gloss Augmentation for Improving Arabic Target Sense Verification

Context-Gloss Augmentation for Improving Arabic Target Sense Verification

Paper · 2023 · Sanad Malaysha, Mustafa Jarrar and Mohammed Khalilia

Summary

Arabic has very few sense-annotated resources. The only sizeable context-gloss collection for target sense verification (TSV) is ArabGlossBERT. TSV asks whether a given dictionary definition matches a target word's meaning in a sentence. It is a binary yes/no decision, not a choice among all senses. The authors enlarged ArabGlossBERT's training split by round-trip machine translation (Arabic → English → Arabic, via Google Translate API), which yields paraphrased contexts and glosses. They also created extra positive pairs to reduce class imbalance. The result is 352K pairs. They fine-tuned BERT-style models on 13 data configurations and scored each against the untouched original test split, broken down by part of speech. Accuracy stayed between 78% and 84%. Augmentation neither beat nor hurt the 84% baseline overall, though some configurations gained on noun or verb subsets. The work matters as an honest negative-ish result: naive back-translation brings little for sense verification, which echoes the small gain seen earlier for English.

Key points

  • Dataset growth. The original ArabGlossBERT has 167,207 pairs: 152,035 for training and 15,172 for testing. Augmented ArabGlossBERT has 352,081 pairs, with 336,909 for training (144,009 positive, 192,900 negative).
  • Rebalancing. Pairing original glosses with their back-translated versions added 32,839 positives. This moved the training class split from about 37/63 to 42.7/57.3.
  • Untouched test set. Only the training split was back-translated. The test split stayed original, so every configuration is compared on the same benchmark. The baseline (AraBERT-V2, from prior work) reaches 84% accuracy.
  • Subset experiments. Removing 4,427 function-word pairs did not help. Separate noun-only (74,485 pairs) and verb-only (73,123 pairs) models did slightly worse than one joint model.
  • Weak positives. Across all runs and all POS groups, F1 on the Positive class stayed below F1 on the Negative class. Adding more positives did not close this gap.

Interesting ideas

  • I1. Sentinel for truncated translations. Each sentence gets a short sentinel string (".#") appended before translation, and a length-ratio check runs on the output. Together they catch translations that silently dropped part of the input, and only the flagged ones go to manual review; this is a cheap integrity check for any batch MT or LLM rewriting step.
  • I2. Positives from paraphrases. New positive examples come from pairing a gloss with its own paraphrase. Paraphrase augmentation thus fixes class imbalance as a side effect, without any manual labeling.
  • I3. Keep the test set clean. Augmentation is applied to training data only, never to the evaluation split. Without this, gains could just reflect the model learning translation artifacts.
  • I4. Per-POS evaluation and joint training. Breaking accuracy down by part of speech exposed gains that the overall number hid. A single model trained on nouns and verbs together beat two separate specialists, which suggests the categories help each other.
  • I5. Verification instead of ranking. Framing sense choice as a yes/no check of one context against one gloss gives an explainable local decision that can be applied to every candidate sense.

For Mova

How these ideas could serve Mova, our small open language model — its files, stages and tests:

  • I1 (learn): Silver data produced by LLM teachers could carry an end marker plus a length check, so truncated outputs are rejected instead of quietly entering training.
  • I2 (world): When linking WordNet senses to text, alternate wordings of the same gloss could serve as known-equal pairs for testing whether the sense matcher is robust to rewording.
  • I3 (learn): Any synthetic or round-tripped data should stay out of evaluation sets, and a fully-synthetic configuration can act as a negative control.
  • I4 (frames): VerbNet role assignment and sense checks should be reported per POS and per verb class, and a shared model should be preferred over many narrow ones.
  • I5 (frames): Each frame or sense candidate could be checked against its gloss with a recorded verdict, making disambiguation steps auditable rather than one opaque argmax.

Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.

 Institutions

🎓Birzeit UniversityBirzeit · Palestine36 works · 798 citations

 Fields, topics and concepts

📚 Shelves 4The field of science this work belongs to (OpenAlex fields).Computer Science · 16,667 worksArtificial Intelligence · 16,194 worksArts and Humanities · 16,039 worksLanguage and Linguistics · 15,891 works

🧭 Topics 1Research subjects and languages the work is about.Semitic languages · 2,908 works

Cite

Cite as Sanad Malaysha, Mustafa Jarrar and Mohammed Khalilia. 2023. Context-Gloss Augmentation for Improving Arabic Target Sense Verification. Proceedings of the 12th Global Wordnet Conference.

The paper itself, not this page. BibTeX from the library's record of the work. DOI 10.18653/v1/2023.gwc-1.31

@inproceedings{malaysha2023context,
  title = {{Context-Gloss Augmentation for Improving Arabic Target Sense Verification}},
  author = {Sanad Malaysha, Mustafa Jarrar and Mohammed Khalilia},
  booktitle = {Proceedings of the 12th Global Wordnet Conference},
  year = {2023},
  doi = {10.18653/v1/2023.gwc-1.31},
  url = {https://doi.org/10.18653/v1/2023.gwc-1.31}
}

References

27 references, 9 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

  1. Moustafa Al-Hajj and Mustafa Jarrar. 2021. Lu-bzu at semeval-2021 task 2: Word2vec and lemma2vec performance in arabic word-in-context disambiguation. In Proceedings of the 15th International Workshop on Semantic Evaluation (SemEval-2021), pages 748– 755, Online. Association for Computational Linguistics.
  2. Moustafa Al-Hajj and Mustafa Jarrar. 2022. Arabgloss-bert: Fine-tuning bert on context-gloss pairs for wsd. In Proceedings of the International Conference on Recent Advances in Natural Language Processing (RANLP 2021), Held Online, 1-3September, 2021, pages 35–43. find
  3. Kareem Darwish, Nizar Habash, Mourad Abbas, Hend Al-Khalifa, Huseein T. Al-Natsheh, Houda Bouamor, Karim Bouzoubaa, Violetta Cavalli-Sforza, Samhaa R. El-Beltagy, Wassim El-Hajj, Mustafa Jarrar, and Hamdy Mubarak. 2021. A panoramic survey of natural language processing in the arab worlds. Commun. ACM, 64(4):72–81. find
  4. Erik De Vries, Martijn Schoonvelde, and Gijs Schumacher. 2018. No longer lost in translation: Evidence that google translate works for comparative bag-of-words text applications. Political Analysis, 26(4):417–430. ↗
  5. Mohammed El-Razzaz, Mohamed Waleed Fakhr, and Fahima A Maghraby. 2021. Arabic gloss wsd using bert. Applied Sciences, 11(6):2567. ↗
  6. Bradley Hauer and Grzegorz Kondrak. 2022. Wic= tsv= wsd: On the equivalence of three semantic tasks. In Proceedings of the 2022 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, NAACL 2022, Seattle, WA, United States, July 10-15, 2022, pages 2478–2486. ↗
  7. Wissam Antoun, Fady Baly, and Hazem Hajj. 2020. Arabert: Transformer-based model for arabic language understanding. In Proceedings of the 4th Workshop on Open-Source Arabic Corpora and Processing Tools, with a Shared Task on Offensive Language Detection, pages 9–15. ↗
  8. Luyao Huang, Chi Sun, Xipeng Qiu, and Xuan-Jing Huang. 2019. Glossbert: Bert for word sense disambiguation with gloss knowledge. In Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing (EMNLP-IJCNLP), pages 3509–3514. ↗
  9. Michele Bevilacqua, Tommaso Pasini, Alessandro Raganato, Roberto Navigli, et al. 2021. Recent trends in word sense disambiguation: A survey. In Proceedings of the Thirtieth International Joint Conference on Artificial Intelligence, IJCAI-21, pages 4330–4338. find
  10. Mustafa Jarrar. 2006. Towards the notion of gloss, and the adoption of linguistic resources in formal ontology engineering. In Proceedings of the 15th international conference on World Wide Web (WWW2006), pages 497–503. ACM Press, New York, NY. find
  11. Jan A Botha, Zifei Shan, and Dan Gillick. 2020. Entity linking in 100 languages. In Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing (EMNLP), pages 7833–7845.
  12. Mustafa Jarrar. 2011. Building a formal arabic ontology (invited paper). In Proceedings of the Experts Meeting on Arabic Ontologies and Semantic Networks. ALECSO, Arab League. find
  13. Mustafa Jarrar. 2018. Search engine for arabic lexicons. In Proceedings of the 5th Conference on Translation and the Problematics of Cross-cultural Understanding. The Forum for Arab and International Relations. find
  14. Mustafa Jarrar. 2020. Digitization of Arabic Lexicons, pages 214–217. UAE Ministry of Culture and Youth. find
  15. Mustafa Jarrar. 2021. The arabic ontology - an arabic wordnet with ontologically clean content. Applied Ontology Journal, 16(1):1–26. ↗
  16. Mustafa Jarrar and Hamzeh Amayreh. 2019. An arabic-multilingual database with a lexicographic search engine. In The 24th International Conference on Applications of Natural Language to Information Systems (NLDB 2019), volume 11608 of LNCS, pages 234–246. Springer. find
  17. Mustafa Jarrar, Hamzeh Amayreh, and John P. McCrae. 2019. Representing arabic lexicons in lemon - a preliminary study. In The 2nd Conference on Language, Data and Knowledge (LDK 2019), volume 2402, pages 29–33. CEUR Workshop Proceedings. ↗
  18. Mustafa Jarrar, Mohammed Khalilia, and Sana Ghanem. 2022. Wojood: Nested arabic named entity corpus and recognition using bert. In Proceedings of the International Conference on Language Resources and Evaluation (LREC 2022), Marseille, France.
  19. Mustafa Jarrar, Fadi Zaraket, Rami Asia, and Hamzeh Amayreh. 2018. Diacritic-based matching of arabic words. ACM Asian and Low-Resource Language Information Processing, 18(2):10:1–10:21. ↗
  20. Jacob Devlin Ming-Wei Chang Kenton and Lee Kristina Toutanova. 2019. Bert: Pre-training of deep bidirectional transformers for language understanding. In Proceedings of NAACL-HLT, pages 4171–4186.
  21. Guan-Ting Lin and Manuel Giambi. 2021. Context-gloss augmentation for improving word sense disambiguation. arXiv preprint arXiv:2110.07174, abs/2110.07174. ↗
  22. George A Miller, Claudia Leacock, Randee Tengi, and Ross T Bunker. 1993. A semantic concordance. In Human Language Technology: Proceedings of a Workshop Held at Plainsboro, New Jersey, March 21-24, 1993.
  23. Jose G Moreno, Elvys Linhares Pontes, and Gaël Dias. 2021. Ctlr@ wic-tsv: Target sense verification using marked inputs andpre-trained models. In Proceedings of the 6th Workshop on Semantic Deep Learning (SemDeep-6), pages 1–6. find
  24. Nikhil Patel, James Hale, Kanika Jindal, Apoorva Sharma, and Yichun Yu. 2021. Building on huang et al. glossbert for word sense disambiguation. arXiv preprint arXiv:2112.07089, abs/2112.07089. ↗
  25. Niloofar Ranjbar and Hossein Zeinali. 2021. Lotus at semeval-2021 task 2: Combination of bert and paraphrasing for english word sense disambiguation.
  26. In Proceedings of the 15th International Workshop on Semantic Evaluation (SemEval-2021), pages 724– 729. find
  27. Boon Peng Yap, Andrew Koh, and Eng Siong Chng. 2020. Adapting bert for word sense disambiguation with gloss selection objective and example sentences. In Findings of the Association for Computational Linguistics: EMNLP 2020, pages 41–46. find

Updated

↑ back to top

This page in Markdown → https://md.research.elamur.ai/free/en/papers/malaysha-2023-context-gloss-augmentation-improving-arabic/