Elamur Research⌕
☰
home / papers / 2024 / EMAD: A Bridge Tagset for Unifying Arabic POS Annotations

EMAD: A Bridge Tagset for Unifying Arabic POS Annotations

Paper · 2024 · Omar Kallas, Go Inoue and Nizar Habash

Summary

Arabic has many part-of-speech tagsets, built by different groups for different goals. They disagree on what counts as a token, whether tags describe surface morphemes or underlying features, how much detail they encode, and how values are written. Because of this, existing annotated corpora are hard to merge, which makes Arabic data scarcity worse. Kallas, Inoue and Habash propose EMAD, an Arabic-specific intermediate ("bridge") tagset that extends the MADA feature set. Each word becomes a matrix. Its columns are tokens under D3 tokenization, which separates every clitic including the definite article, and its rows are morphological features. Each source tagset is connected to EMAD through a configurable driver containing a many-to-many feature-value map, and the same table converts in both directions. Running the mapping also exposes ambiguous or inconsistent original tags and can fill in missing features automatically. The paper reports an evaluation, but those results fall in a portion of the text not available for this summary. The work matters because it makes combining heterogeneous Arabic corpora a controlled, checkable process. Code: github.com/CAMeL-Lab/emad.

Key points

  • Tagset sizes vary enormously:
    • Subtags: 6 in CATiB6, 17 in UD, roughly 170 in Buckwalter and MADA.
    • Full word-level tags: 38 in CATiB6, 300 in UD, 35,682 in Buckwalter, 243,720 in MADA.
  • Matrix structure: an EMAD tag has 12 feature rows. These cover the diacritized form (diac) and part of speech (pos); the inflectional features aspect (asp), mood (mod), voice (vox), person (per), case (cas) and state (stt); and gender and number, each recorded in a functional and a form version (gen, form_gen, num, form_num). Its columns follow eight ordered morpheme slots:
    • PRC_QUES, PRC_CONJ, PRC_PREP, PRC_VPAR, PRC_DET (proclitics)
    • BASE (the base word)
    • ENC_PRON, ENC_PART (enclitics)
  • Driver components: a driver has five parts: the features, the tag format, the allowed values, the legal combinations, and the map to EMAD.
    • Conversion needs only parts 1–3 and part 5.
    • Part 4 is used solely for well-formedness checking.
  • Inspiration: the design follows Zeman's Interset, but uses a space tailored to Arabic rather than a universal one. It also builds on prior work in which underspecified features were predicted with 94–95% accuracy when mapping CATiB trees to Buckwalter tags.
  • Ambiguity: a word that is ambiguous out of context gets an array of alternative matrices, which can later be ranked in context.

Interesting ideas

  • I1. Gender and number are each stored twice, once as the functional value and once as the form value, so words like broken plurals with feminine-looking endings are never forced into one label.
  • I2. All conversions go through a single hub representation, with one bidirectional mapping table per scheme, so supporting N schemes requires N drivers instead of N² converters.
  • I3. The legal-combination rules of a tagset are kept separate from its value mapping and used as a validator, which catches malformed or inconsistent source tags during conversion.
  • I4. Converting from a coarse tagset to a fine one is treated as feature enrichment: missing values are inferred rather than left blank, and remaining ambiguity is kept as ranked alternatives.
  • I5. Clitics get their own token columns with fully decomposed features, so a pronoun's person, gender and number can be accessed directly instead of through one opaque code.

For Mova

How these ideas could serve Mova, our small open language model — its files, stages and tests:

  • I1 (tokens): English morphology could likewise record surface form separately from agreement behaviour, for nouns like "police" or "data", and let the parser rely on the functional value.
  • I2 (tree): UD dialect conversion and treebank merging could route every scheme through one rich pivot, so each new treebank format needs only one map.
  • I3 (learn): silver data from witness parsers could first pass a well-formedness check for allowed feature combinations, with rejected tokens logged as explainable repair candidates.
  • I4 (tokens): when English data annotated only with UPOS is imported, the tagger could propose ranked full-feature analyses, keeping the alternatives visible instead of silently guessing.
  • I5 (frames): splitting contractions and possessives ("John's", "won't") into tokens whose features are fully spelled out would give coreference and role assignment direct access to agreement information.

Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.

 Fields, topics and concepts

📚 Shelves 4The field of science this work belongs to (OpenAlex fields).Computer Science · 18,614 worksArtificial Intelligence · 17,965 worksArts and Humanities · 17,450 worksLanguage and Linguistics · 17,296 works

🧭 Topics 2Research subjects and languages the work is about.Semitic languages · 3,066 worksMorphology · 420 works

Cite

Cite as Omar Kallas, Go Inoue and Nizar Habash. 2024. EMAD: A Bridge Tagset for Unifying Arabic POS Annotations. Proceedings of the 2024 Joint International Conference on Computational Linguistics, Language Resources and Evaluation (LREC-COLING 2024).

The paper itself, not this page. BibTeX from the library's record of the work.

@inproceedings{kallas2024emad,
  title = {{EMAD: A Bridge Tagset for Unifying Arabic POS Annotations}},
  author = {Omar Kallas, Go Inoue and Nizar Habash},
  booktitle = {Proceedings of the 2024 Joint International Conference on Computational Linguistics, Language Resources and Evaluation (LREC-COLING 2024)},
  year = {2024},
  doi = {10.63317/34zfgedkp5m3},
  url = {https://doi.org/10.63317/34zfgedkp5m3}
}

References

30 references, 15 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

  1. Hanan Aldarmaki and Mona Diab. 2015. Robust part-of-speech tagging of Arabic text. In Proceedings of the Second Workshop on Arabic Natural Language Processing, pages 173–182, Beijing, China. Association for Computational Linguistics.
  2. Bashar Alhafni, Nizar Habash, and Houda Bouamor. 2022. User-centric gender rewriting. In Proceedings of the 2022 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, pages 618–631, Seattle, United States. Association for Computational Linguistics. ↗
  3. Sarah Alkuhlani, Nizar Habash, and Ryan Roth. 2013. Automatic morphological enrichment of a morphologically underspecified treebank. In Proceedings of the 2013 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, pages 460–470, Atlanta, Georgia. Association for Computational Linguistics. find
  4. Tim Buckwalter. 2004. Buckwalter Arabic Morphological Analyzer Version 2.0. LDC catalog number LDC2004L02, ISBN 1-58563-324-0. ↗
  5. Mona Diab. 2007. Improved Arabic Base Phrase Chunking with a New Enriched POS Tag Set. In Proceedings of the Workshop on Computational Approaches to Semitic Languages (CASL), pages 89–96, Prague, Czech Republic.
  6. Mona Diab, Nizar Habash, Owen Rambow, and Ryan Roth. 2013. LDC Arabic treebanks and associated corpora: Data divisions manual. arXiv preprint arXiv:1309.5652. ↗
  7. Nizar Habash, Reem Faraj, and Ryan Roth. 2009. Syntactic Annotation in the Columbia Arabic Treebank. In Proceedings of the International Conference on Arabic Language Resources and Tools, Cairo, Egypt. find
  8. Nizar Habash and Ryan Roth. 2009. CATiB: The Columbia Arabic Treebank. In Proceedings of the Joint Conference of the Association for Computational Linguistics and the International Joint Conference on Natural Language Processing (ACL-IJCNLP), pages 221–224, Suntec, Singapore. ↗
  9. Nizar Habash, Abdelhadi Soudi, and Tim Buckwalter. 2007. On Arabic Transliteration. In A. van den Bosch and A. Soudi, editors, Arabic Computational Morphology: Knowledge-based and Empirical Methods, pages 15–22. Springer, Netherlands. ↗
  10. Nizar Y Habash. 2010. Introduction to Arabic natural language processing, volume 3. Morgan & Claypool Publishers. find
  11. Hulden. 2022. PyFoma https://github.com/mhulden/pyfoma/. ↗
  12. Go Inoue, Salam Khalifa, and Nizar Habash. 2022. Morphosyntactic tagging with pre-trained language models for Arabic and its dialects. In Findings of the Association for Computational Linguistics: ACL 2022, pages 1708–1719, Dublin, Ireland. Association for Computational Linguistics.
  13. Salam Khalifa, Nizar Habash, Fadhl Eryani, Ossama Obeid, Dana Abdulrahim, and Meera Al Kaabi. 2018. A morphologically annotated corpus of emirati Arabic. In Proceedings of the Eleventh International Conference on Language Resources and Evaluation (LREC 2018), Miyazaki, Japan. European Language Resources Association (ELRA).
  14. Shereen Khoja, Roger Garside, and Gerry Knowles. 2001. A tagset for the morphosyntactic tagging of Arabic. In Proceedings of the Corpus Linguistics Conference, pages 341–353, Lancaster, UK. find
  15. Seth Kulick, Ryan Gabbard, and Mitch Marcus. 2006. Parsing the Arabic Treebank: Analysis and Improvements. In Proceedings of the Workshop on Treebanks and Linguistic Theories (TLT), pages 31–42, Prague, Czech Republic. find
  16. Mohamed Maamouri, Ann Bies, Tim Buckwalter, and Wigdan Mekki. 2004. The Penn Arabic Treebank: Building a Large-Scale Annotated Arabic Corpus. In Proceedings of the International Conference on Arabic Language Resources and Tools, pages 102–109, Cairo, Egypt. find
  17. Mitchell P. Marcus, Beatrice Santorini, and Mary Ann Marcinkiewicz. 1993. Building a large annotated corpus of English: The Penn Treebank. Computational Linguistics, 19(2):313–330. ↗
  18. Yuval Marton, Nizar Habash, and Owen Rambow. 2010. Improving Arabic dependency parsing with lexical and inflectional morphological features. In Proceedings of the NAACL HLT 2010 First Workshop on Statistical Parsing of Morphologically-Rich Languages, pages 13–21, Los Angeles, CA, USA. Association for Computational Linguistics.
  19. Joakim Nivre. 2014. Universal dependencies for Swedish. In Proceedings of the Swedish Language Technology Conference (SLTC). find
  20. Ossama Obeid, Nasser Zalmout, Salam Khalifa, Dima Taji, Mai Oudah, Bashar Alhafni, Go Inoue, Fadhl Eryani, Alexander Erdmann, and Nizar Habash. 2020. CAMeL tools: An open source python toolkit for Arabic natural language processing. In Proceedings of The 12th Language Resources and Evaluation Conference, pages 7022–7032, Marseille, France. European Language Resources Association.
  21. Arfath Pasha, Mohamed Al-Badrashiny, Mona Diab, Ahmed El Kholy, Ramy Eskander, Nizar Habash, Manoj Pooleery, Owen Rambow, and Ryan Roth. 2014. Madamira: A fast, comprehensive tool for morphological analysis and disambiguation of Arabic. In Proceedings of the Language Resources and Evaluation Conference (LREC), pages 1094–1101, Reykjavik, Iceland.
  22. Fatiha Sadat and Nizar Habash. 2006. Combination of Arabic preprocessing schemes for statistical machine translation. In Proceedings of the International Conference on Computational Linguistics and the Conference of the Association for Computational Linguistics (COLING-ACL), pages 1–8, Sydney, Australia.
  23. Majdi Sawalha and Eric Atwell. 2013. A standard tag set expounding traditional morphological features for Arabic language part-of-speech tagging. Word Structure, 6(1):43–99. ↗
  24. Otakar Smrž. 2007. ElixirFM — Implementation of Functional Arabic Morphology. In Proceedings of the Workshop on Computational Approaches to Semitic Languages (CASL), pages 1–8, Prague, Czech Republic. ACL.
  25. Dima Taji, Nizar Habash, and Daniel Zeman. 2017. Universal Dependencies for Arabic. In Proceedings of the Third Arabic Natural Language Processing Workshop, pages 166–176, Valencia, Spain. Association for Computational Linguistics.
  26. Daniel Zeman. 2008. Reusable tagset conversion using tagset drivers. In Proceedings of the Sixth International Conference on Language Resources and Evaluation (LREC’08), Marrakech, Morocco. European Language Resources Association (ELRA).
  27. Imad Zeroual, Abdelhak Lakhouaja, and Rachid Be-lahbib. 2017. Towards a standard part of speech tagset for the arabic language. Journal of King Saud University - Computer and Information Sciences, 29(2):171–178. Arabic Natural Language Processing: Models, Systems and Applications. 8. ↗
  28. Language Resource References Mohamed Maamouri, Ann Bies, Seth Kulick, Fatma Gaddeche, Wigdan Mekki, Sondos Krouna, Basma Bouziri, and Wadji Zaghouani. 2010a. Arabic treebank: Part 1 v 4.1. Linguistic Data Consortium (LDC2010T13). ↗
  29. Mohamed Maamouri, Ann Bies, Seth Kulick, Fatma Gaddeche, Wigdan Mekki, Sondos Krouna, Basma Bouziri, and Wadji Zaghouani. 2011. Arabic treebank: Part 2 v 3.1. Linguistic Data Consortium (LDC2011T09). ↗
  30. Mohamed Maamouri, Ann Bies, Seth Kulick, Sondos Krouna, Fatma Gaddeche, and Wajdi Zaghouani. 2010b. Arabic treebank: Part 3 v 3.2. Linguistic Data Consortium (LDC2010T08). ↗

Updated

↑ back to top

This page in Markdown → https://md.research.elamur.ai/free/en/papers/kallas-2024-emad-bridge-tagset-unifying-arabic/