Elamur Research⌕
☰
home / papers / 2015 / A Pilot Study on Arabic Multi-Genre Corpus Diacritization

A Pilot Study on Arabic Multi-Genre Corpus Diacritization

Paper · 2015 · Houda Bouamor, Wajdi Zaghouani, Mona Diab et al.

Summary

Modern Standard Arabic is normally written without most short-vowel marks, so one undiacritized form can map to many readings (up to about 200). This ambiguity hurts NLP tools, and existing diacritized resources cover mostly newswire and religious texts. The authors run a pilot toward a diacritized corpus spanning several genres. They take 16,770 words from five sources, strip existing diacritics, and have annotators restore them under three presentation modes: Basic (bare forms), Intermediate (forms plus POS tags) and Advanced (candidates pre-diacritized automatically with MADAMIRA). Advanced gives the best accuracy, Intermediate beats Basic, and Basic is hardest. When the automatic candidates are poor, as on Tashkeela, annotators seldom fix them, which illustrates annotation "tainting". The authors also try a loosely defined minimal scheme (MIN) on ATB News sentences. Agreement is low (about 27% WER against a lead annotator), so "minimal diacritization" turns out to be subjective. The study matters because it measures how presentation choices trade annotation quality against speed. It also shows that a partial-diacritization standard needs a much tighter definition.

Key points

  • Data per genre: ATB News 2,478 words, ATB BN 3,093, ATB WebLog 3,177, Tashkeela 5,172, and Arabic Wikipedia abstracts 2,850. Wikipedia is the only source without prior gold diacritics.
  • The work targets three kinds of marks: short vowels (including sukuun), nunation and shadda. Hamza, letter dots and other markers are excluded.
  • Advanced mode reached 98.0 accuracy in the reported setting. On Tashkeela, accuracy stayed low in every mode, and annotators made only 194 edits in Advanced mode.
  • In Basic mode, WebLog was the hardest genre by raw accuracy.
  • The MIN pilot used 100 sentences (3,527 words) annotated by four annotators, with the lead annotator serving as gold. Average WER was 27.15, and per-sentence WER ranged from 16.66 to 44.87. One outlier annotator was found who over-diacritized.

Interesting ideas

  • I1. Comparing presentation modes (bare input, input plus a tag hint, pre-filled machine output) isolates how much the pre-annotation helps and how much it anchors annotators onto wrong answers.
  • I2. The paper splits ambiguity into two kinds: lexical, which is unpredictable and not tied to POS, and inflectional, which is positional and tied to POS and syntax. Each kind calls for a different resolution strategy.
  • I3. The "minimal disambiguation" idea annotates only the marks needed to make a form unambiguous in its context, rather than everything.
  • I4. Agreement is scored as WER against a lead annotator, and an outlier is identified by how often it disagrees with the rest. This gives a cheap way to rank the reliability of multiple sources.
  • I5. Sampling the test data by genre revealed difficulty that differs sharply between genres, such as weblogs versus news.

For Mova

How these ideas could serve Mova, our small open language model — its files, stages and tests:

  • I1 (learn): when LLM teachers or silver parses are shown for correction, add control items whose pre-fill is deliberately wrong, then measure how often the corrector accepts them.
  • I2 (tokens): keep lemma-level ambiguity separate from feature-level ambiguity in the morphology layer, so that inflectional choices can be settled by rules driven by the tree.
  • I3 (back): generation could add only the clarifying material (an explicit subject, a disambiguating modifier) that a reader needs to recover the intended graph.
  • I4 (learn): score each witness parser against the consensus per sentence, and down-weight or inspect a witness that keeps diverging.
  • I5 (answer): report reader accuracy separately for fables, school tests and word problems, so that weaknesses in one genre are not hidden in an average.

Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.

 Institutions

🎓Carnegie Mellon UniversityPittsburgh · United States539 works · 179,533 citations🎓George Washington UniversityWashington · United States51 works · 8,148 citations🎓Carnegie Mellon University QatarDoha · Qatar47 works · 2,773 citations

 Fields, topics and concepts

📚 Shelves 6The field of science this work belongs to (OpenAlex fields).Computer Science · 16,667 worksArtificial Intelligence · 16,194 worksArts and Humanities · 16,039 worksLanguage and Linguistics · 15,891 worksSocial Sciences · 1,772 worksEducation · 203 works

🧭 Topics 1Research subjects and languages the work is about.Semitic languages · 2,908 works

Cite

Cite as Houda Bouamor, Wajdi Zaghouani, Mona Diab, Ossama Obeid, Kemal Oflazer, Mahmoud Ghoneim and Abdelati Hawwari. 2015. A Pilot Study on Arabic Multi-Genre Corpus Diacritization. Proceedings of the Second Workshop on Arabic Natural Language Processing.

The paper itself, not this page. BibTeX from the library's record of the work. DOI 10.18653/v1/w15-3209

@inproceedings{bouamor2015pilot,
  title = {{A Pilot Study on Arabic Multi-Genre Corpus Diacritization}},
  author = {Houda Bouamor, Wajdi Zaghouani, Mona Diab, Ossama Obeid, Kemal Oflazer, Mahmoud Ghoneim and Abdelati Hawwari},
  booktitle = {Proceedings of the Second Workshop on Arabic Natural Language Processing},
  year = {2015},
  doi = {10.18653/v1/w15-3209},
  url = {https://doi.org/10.18653/v1/w15-3209}
}

References

18 references, 8 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

  1. Gheith A Abandah, Alex Graves, Balkees Al-Shagoor, Alaa Arabiyat, Fuad Jamour, and Majid Al-Taee. 2015. Automatic Diacritization of Arabic Text using Recurrent Neural Networks. International Journal on Document Analysis and Recognition (IJDAR), 18(2):1–15. ↗
  2. Mansour Alghamdi and Zeeshan Muzaffar. 2007. KACST Arabic Diacritizer. In The First International Symposium on Computers and Arabic Language, pages 25–28. find
  3. Y.A. Alotaibi, A.H. Meftah, and S.A. Selouani. 2013. Diacritization, Automatic Segmentation and Labeling for Levantine Arabic Speech. In Digital Signal Processing and Signal Processing Education Meeting (DSP/SPE), 2013 IEEE, pages 7–11, Napa, CA. find
  4. Sankaranarayanan Ananthakrishnan, Shrikanth Narayanan, and Srinivas Bangalore. 2005. Automatic Diacritization of Arabic Transcripts for Automatic Speech Recognition. In Proceedings of the 4th International Conference on Natural Language Processing, pages 47–54. find
  5. Mohamed Bebah, Amine Chennoufi, Azzeddine Mazroui, and Abdelhak Lakhouaja. 2014. Hybrid Approaches for Automatic Vowelization of Arabic Texts. International Journal on Natural Language Computing (IJNLC), 3(4). ↗
  6. Tim Buckwalter. 2002. Buckwalter Arabic Morphological Analyzer Version 1.0. Technical Report LDC2002L49, Linguistic Data Consortium. ↗
  7. Mona Diab, Mahmoud Ghoneim, and Nizar Habash. 2007. Arabic Diacritization in the Context of Statistical Machine Translation. In Proceedings of MT-Summit, Copenhagen, Denmark.
  8. Ossama Emam and Volker Fischer. 2011. Hierarchical Approach for the Statistical Vowelization of Arabic Text. US Patent 8,069,045. find
  9. Nizar Habash and Owen Rambow. 2005. Arabic Tokenization, Part-of-Speech Tagging and Morphological Disambiguation in One Fell Swoop. In Proceedings of the 43rd Annual Meeting of the Association for Computational Linguistics, pages 573–580, Ann Arbor, Michigan.
  10. Nizar Habash and Owen Rambow. 2007. Arabic Diacritization through Full Morphological Tagging. In Human Language Technologies 2007: The Conference of the North American Chapter of the Association for Computational Linguistics; Companion Volume, Short Papers, pages 53–56, Rochester, New York.
  11. Ehab Hermena, Denis Drieghe, Sam Hellmuth, and Simon P Liversedge. 2015. Processing of Arabic Diacritical Marks: Phonological–Syntactic Disambiguation of Homographic Verbs and Visual Crowding Effects. Journal of Experimental Psychology. Human Perception and Performance, 41(2):494– 507. ↗
  12. Katrin Kirchhoff and Dimitra Vergyri. 2005. Cross-Dialectal Data Sharing for Acoustic Modeling in Arabic Speech Recognition. Speech Communication, 46(1):37–51. ↗
  13. Mohamed Maamouri, Ann Bies, and Seth Kulick. 2008. Enhancing the Arabic Treebank: a Collaborative Effort toward New Annotation Guidelines. In LREC. Citeseer. find
  14. Arfath Pasha, Mohamed Al-Badrashiny, Ahmed El Kholy, Ramy Eskander, Mona Diab, Nizar Habash, Manoj Pooleery, Owen Rambow, and Ryan Roth. 2014. MADAMIRA: A Fast, Comprehensive Tool for Morphological Analysis and Disambiguation of Arabic. In In Proceedings of the 9th International Conference on Language Resources and Evaluation, Reykjavik, Iceland.
  15. Mohsen Rashwan, Mohammad Al-Badrashiny, Mohamed Attia, and Sherif Abdou. 2009. A Hybrid System for Automatic Arabic Diacritization. In The 2nd International Conference on Arabic Language Resources and Tools, Cairo, Egypt. find
  16. Ryan Roth, Owen Rambow, Nizar Habash, Mona Diab, and Cynthia Rudin. 2008. Arabic Morphological Tagging, Diacritization, and Lemmatization Using Lexeme Models and Feature Ranking. In Proceedings of ACL-08: HLT, Short Papers, pages 117–120, Columbus, Ohio.
  17. Dimitra Vergyri and Katrin Kirchhoff. 2004. Automatic Diacritization of Arabic for Acoustic Modeling in Speech Recognition. In Proceedings of the Workshop on Computational Approaches to Arabic Script-based Languages, pages 66–73. Association for Computational Linguistics. ↗
  18. Imed Zitouni, Jeffrey S. Sorensen, and Ruhi Sarikaya. 2006. Maximum Entropy Based Restoration of Arabic Diacritics. In Proceedings of the 21st International Conference on Computational Linguistics and 44th Annual Meeting of the Association for Computational Linguistics, pages 577–584, Sydney, Australia.

Updated

↑ back to top

This page in Markdown → https://md.research.elamur.ai/free/en/papers/bouamor-2015-pilot-study-arabic-multi-genre/