Elamur Research⌕
☰
home / papers / 2016 / Guidelines and Framework for a Large Scale Arabic Diacritized Corpus

Guidelines and Framework for a Large Scale Arabic Diacritized Corpus

Paper · 2016 · Wajdi Zaghouani, Houda Bouamor, Abdelati Hawwari et al.

Summary

Most written Modern Standard Arabic omits short vowels and other diacritics, so a single written word can map to several pronunciations, lemmas and grammatical readings. Training automatic diacritizers needs large, varied, hand-checked data, but existing diacritized MSA resources mostly cover newswire, religious or educational text. Within the OptDiac project, the authors set out to hand-diacritize a 2-million-word corpus spanning more than ten genres, drawn from the Corpus of Contemporary Arabic (CCA). They adapted and simplified the LDC POS guidelines, adding explicit rules for tricky cases. Five trained annotators and a manager worked in a web tool. Instead of typing marks from scratch, annotators chose among candidate diacritizations proposed by MADAMIRA and edited a word by hand only when no candidate was right. Agreement was tracked with a strict word-level error rate. Over three rounds of guideline revision, this rate fell to 9.31%. The work matters because it gives Arabic NLP its first multi-genre diacritized corpus. It also offers a reusable recipe of guidelines, tooling and quality loops for large orthographic annotation projects.

Key points

  • The target is 2 million words of MSA from the CCA corpus (Al-Sulaiti and Atwell, 2006), covering genres such as autobiography, children's stories, economics, health, interviews, recipes, sports and tourism.
  • Annotation is framed as a selection task over MADAMIRA candidates, with manual editing as a fallback. MADAMIRA reports 99.2% lemmatization accuracy and 86.3% diacritization accuracy.
  • Inter-annotator agreement is measured as WER averaged across annotator pairs, and a word counts as wrong if any of its letters is wrong. WER fell from 16.59 to 12.09 to 9.31 over three guideline iterations.
  • The guidelines include specific exceptions:
    • no shadda on the definite article's lam, but shadda on the solar letter that follows it;
    • no sukuun on word-final silent letters;
    • no case endings on proper nouns, except accusative names of Arabic origin;
    • abbreviations are left bare;
    • accusative nunation is placed on the alif;
    • letters before a long alif are left unmarked.
  • Disagreements are ranked by frequency, from most to least common:
    • ambiguous sentence readings;
    • foreign words and proper nouns;
    • misspellings;
    • case endings;
    • shadda;
    • sukuun;
    • dialectal expressions.

Interesting ideas

  • I1. Turn annotation into choosing among ranked machine proposals, with free editing kept as an escape hatch, so humans spend effort only where the analyzer fails.
  • I2. Score agreement at the unit level, so a single wrong mark spoils the whole word, giving a harsh but readable consistency metric.
  • I3. Revise the guidelines in explicit rounds and re-measure agreement after each one, so the effect of each rule change becomes visible.
  • I4. Explicitly exempt predictable or deterministic positions from annotation, and fix conventions for borderline classes such as names, abbreviations and foreign words.
  • I5. Separate lexical marks, which pick out the lemma, from inflectional marks, which encode syntactic role, and keep a frequency-ranked list of disagreement causes.

For Mova

How these ideas could serve Mova, our small open language model — its files, stages and tests:

  • I1 (learn): LLM teacher tiers or human checkers could pick among parses offered by competing witness parsers rather than drawing trees from scratch, which would make silver-data adjudication cheaper and easier to audit.
  • I2 (tokens): morphology and lemma agreement between taggers could be scored as whole-token exact match, so partial credit does not mask analyses that are wrong for downstream steps.
  • I3 (tree): UD dialect conversion rules could be versioned and re-scored against witness-parser agreement after each revision, showing which rule edits actually reduce inconsistency.
  • I4 (tokens): the tokenizer and tagger could declare deterministic cases as rules and keep them out of learned or annotated targets, which keeps training signal focused on genuinely ambiguous tokens.
  • I5 (frames): role assignment errors could be split the same way into lemma-sense versus case/role confusions, with a ranked error taxonomy deciding which repair rules to write first.

Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.

 Institutions

🎓Carnegie Mellon UniversityPittsburgh · United States539 works · 179,533 citations🎓George Washington UniversityWashington · United States51 works · 8,148 citations🎓Carnegie Mellon University QatarDoha · Qatar47 works · 2,773 citations

 Fields, topics and concepts

📚 Shelves 6The field of science this work belongs to (OpenAlex fields).Computer Science · 16,667 worksArtificial Intelligence · 16,194 worksArts and Humanities · 16,039 worksLanguage and Linguistics · 15,891 worksSocial Sciences · 1,772 worksEducation · 203 works

🧭 Topics 1Research subjects and languages the work is about.Semitic languages · 2,908 works

Cite

Cite as Wajdi Zaghouani, Houda Bouamor, Abdelati Hawwari, Mona Diab, Ossama Obeid, Mahmoud Ghoneim, Sawsan Alqahtani and Kemal Oflazer. 2016. Guidelines and Framework for a Large Scale Arabic Diacritized Corpus. Proceedings of the Tenth International Conference on Language Resources and Evaluation (LREC'16).

The paper itself, not this page. BibTeX from the library's record of the work.

@inproceedings{zaghouani2016guidelines,
  title = {{Guidelines and Framework for a Large Scale Arabic Diacritized Corpus}},
  author = {Wajdi Zaghouani, Houda Bouamor, Abdelati Hawwari, Mona Diab, Ossama Obeid, Mahmoud Ghoneim, Sawsan Alqahtani and Kemal Oflazer},
  booktitle = {Proceedings of the Tenth International Conference on Language Resources and Evaluation (LREC'16)},
  year = {2016},
  doi = {10.63317/55fzrvtc3pvd},
  url = {https://doi.org/10.63317/55fzrvtc3pvd}
}

References

25 references, 13 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

  1. Abandah, G. A., Graves, A., Al-Shagoor, B., Arabiyat, A., Jamour, F., and Al-Taee, M. (2015). Automatic Diacritization of Arabic Text using Recurrent Neural Networks. International Journal on Document Analysis and Recognition (IJDAR), 18(2):183–197. ↗
  2. Al-Sulaiti, L. and Atwell, E. S. (2006). The design of a corpus of Contemporary Arabic. International Journal of Corpus Linguistics, 11(2):135–171. ↗
  3. Belinkov, Y. and Glass, J. (2015). Arabic Diacritization with Recurrent Neural Networks. In Proceedings of the 2015 Conference on Empirical Methods in Natural Language Processing, pages 2281–2285, Lisbon, Portugal.
  4. Bouamor, H., Zaghouani, W., Diab, M., Obeid, O., Oflazer, K., Ghoneim, M., and Hawwari, A. (2015). A Pilot Study on Arabic Multi-Genre Corpus Diacritization. In Proceedings of the Association for Computational Linguistics Second Workshop on Arabic Natural Language Processing, pages 80–88, Beijing, China.
  5. Buckwalter, T. (2002). Buckwalter Arabic Morphological Analyzer Version 1.0. Technical Report LDC2002L49, Linguistic Data Consortium. find
  6. Diab, M., Ghoneim, M., and Habash, N. (2007). Arabic Diacritization in the Context of Statistical Machine Translation. In Proceedings of MT-Summit, Copenhagen, Denmark.
  7. Diab, M., Mansouri, A., Palmer, M., Babko-Malaya, O., Zaghouani, W., Bies, A., and Maamouri, M. (2008). A pilot arabic propbank. In Proceedings of the 7th International Conference on Language Resources and Evaluation (LREC). find
  8. Dukes, K. and Habash, N. (2010). Morphological annotation of quranic arabic. In Proceedings of the International Conference on Language Resources and Evaluation (LREC).
  9. Kirchhoff, K. and Vergyri, D. (2005). Cross-Dialectal Data Sharing for Acoustic Modeling in Arabic Speech Recognition. Speech Communication, 46(1):37–51. ↗
  10. Maamouri, M., Bies, A., and Kulick, S. (2008). Enhancing the arabic treebank: a collaborative effort toward new annotation guidelines. In Proceedings of the Sixth International Conference on Language Resources and Evaluation (LREC 2008), Marrakech, Morocco. find
  11. Maamouri, M., Bies, A., Kulick, S., Zaghouani, W., Graff, D., and Ciul, M. (2010). From speech to trees: Applying treebank annotation to arabic broadcast news. In Proceedings of International Conference on Language Resources and Evaluation (LREC 2010). find
  12. Mohit, B., Rozovskaya, A., Habash, N., Zaghouani, W., and Obeid, O. (2014). The first qalb shared task on automatic text correction for arabic. In Proceedings of the EMNLP Workshop on Arabic Natural Language Processing, page 39.
  13. Obeid, O., , Bouamor, H., Zaghouani, W., Ghoneim, M., Hawwari, A., Diab, M., and Oflazer, K. (2016). MAN-DIAC: A Web-based Annotation System For Manual Arabic Diacritization. In Proceedings of the International Conference on Language Resources and Evaluation (LREC-2016) Workshop on Free/Open-Source Arabic Corpora and Corpora Processing Tools (OSACT2). find
  14. Pasha, A., Al-Badrashiny, M., Kholy, A. E., Eskander, R., Diab, M., Habash, N., Pooleery, M., Rambow, O., and Roth, R. (2014). MADAMIRA: A Fast, Comprehensive Tool for Morphological Analysis and Disambiguation of Arabic. In Proceedings of the 9th International Conference on Language Resources and Evaluation, Reykjavik, Iceland.
  15. Rozovskaya, A., Bouamor, H., Habash, N., Zaghouani, W., Obeid, O., and Mohit, B. (2015). The second qalb shared task on automatic text correction for arabic. In Proceedings of the ACL-IJCNLP Workshop on Arabic Natural Language Processing, page 26.
  16. Schulz, E. (2004). A Student Grammar of Modern Standard Arabic. Cambridge University Press. find
  17. Shahrour, A., Khalifa, S., and Habash, N. (2015). Improving Arabic Diacritization through Syntactic Analysis. In Proceedings of the 2015 Conference on Empirical Methods in Natural Language Processing, pages 1309–1315, Lisbon, Portugal.
  18. Snover, M., Dorr, B., Schwartz, R., Micciulla, L., and Makhoul, J. (2006). A study of translation edit rate with targeted human annotation. In Proceedings of AMTA, pages 223–231.
  19. Zaghouani, W., Diab, M., Mansouri, A., Pradhan, S., and Palmer, M. (2010). The revised arabic propbank. In Proceedings of the Association for Computational Linguistics Fourth Linguistic Annotation Workshop, pages 222–226. Association for Computational Linguistics. find
  20. Zaghouani, W., Hawwari, A., and Diab, M. (2012). A pilot propbank annotation for quranic arabic. In Proceedings of the Fourth Workshop on Computational Linguistics for Literature co-located with the North American Association Computational Linguistics conference (NAACL-HLT 2012, page 78. find
  21. Zaghouani, W., Mohit, B., Habash, N., Obeid, O., Tomeh, N., Rozovskaya, A., Farra, N., Alkuhlani, S., and Oflazer, K. (2014). Large scale arabic error annotation: Guidelines and framework. In Proceedings of the Ninth International Conference on Language Resources and Evaluation (LREC-2014), Reykjavik, Iceland, May 26-31, 2014., pages 2362–2369.
  22. Zaghouani, W., Habash, N., Bouamor, H., Rozovskaya, A., Mohit, B., Heider, A., and Oflazer, K. (2015). Correction annotation for non-native arabic texts: Guidelines and corpus. In Proceedings of the Association for Computational Linguistics Fourth Linguistic Annotation Workshop, pages 129–139.
  23. Zaghouani, W., Habash, N., Obeid, O., Mohit, B., Bouamor, H., and Oflazer, K. (2016). Building an arabic machine translation post-edited corpus: Guidelines and annotation. In International Conference on Language Resources and Evaluation (LREC 2016).
  24. Zerrouki, T. (2011). Tashkeela: Arabic vocalized text corpus. find
  25. Zitouni, I., Sorensen, J. S., and Sarikaya, R. (2006). Maximum Entropy Based Restoration of Arabic Diacritics. In Proceedings of the 21st International Conference on Computational Linguistics and 44th Annual Meeting of the Association for Computational Linguistics, pages 577–584, Sydney, Australia.

Updated

↑ back to top

This page in Markdown → https://md.research.elamur.ai/free/en/papers/zaghouani-2016-guidelines-framework-large-scale-arabic/