Elamur Research⌕
☰
home / papers / 2014 / A Conventional Orthography for Tunisian Arabic

A Conventional Orthography for Tunisian Arabic

Paper · 2014 · Inès Zribi, Rahma Boujelbane, Abir Masmoudi et al.

Summary

Tunisian Arabic (TUN) is the everyday spoken variety of Tunisia. It is increasingly written online, but it has no standard spelling, few texts or dictionaries, and it blends into Modern Standard Arabic (MSA) through many mixed forms. Inconsistent spelling makes it hard to build and share NLP resources. The authors adapt CODA (Conventional Orthography for Dialectal Arabic), first defined for Egyptian Arabic (EGY), to TUN. They first describe how TUN differs from MSA and EGY in phonology, morphology and vocabulary, including its many loanwords. They then set out guidelines for phonological spelling, attached and separated clitics, number constructs, foreign sounds and a list of lexical exceptions. The convention is not meant for speakers to type. It is a target form for computational work, and free-form TUN text is to be normalized into it automatically. TUN CODA already supports a spoken corpus, a morphological analyzer and speech-recognition work. Because CODA is shared across dialects, resources built for one dialect can be reused for another.

Key points

  • CODA has five stated goals:
    • an internally consistent convention;
    • designed for computational use;
    • written in Arabic script;
    • a single framework for all dialects;
    • a balance between dialect-specific features and MSA-based conventions.
  • TUN is spoken by about 11 million people across several regional varieties (Tunis, Sahil, Sfax, Northwestern, Southwestern, Southeastern). Its lexicon includes loans from Italian, Turkish, Berber, Spanish, French and Maltese, and speakers frequently code-switch into unadapted French.
  • TUN has simplified its morphology relative to MSA:
    • It drops feminine forms in verb conjugation and merges the 1st-person singular verb form with the 2nd person.
    • It replaces most nominal duals with zuwz "two" plus a plural noun.
    • It adds new clitics: negation ma+ +š, interrogative +šiy, demonstrative ha+, and the reduced prepositions ς a+ and m+.
  • Specific spelling rules include:
    • Foreign /g/, /v/ and /p/ are written with q, f and b.
    • The /n/ added to numerals in construct is written out (e.g., xmsTaAšn "15 [men]").
    • The negation particle is separated from indirect-object enclitics by a space, for cross-dialect consistency.
    • A word list fixes the spelling of words with exceptional or variable forms.
  • Systems already using TUN CODA:
    • transcription of the Spoken Tunisian Arabic Corpus (STAC);
    • the databases of a TUN morphological analyzer;
    • language-model corpora for ASR;
    • an MSA-TUN bilingual lexicon (Hamdi et al., 2013) for converting MSA corpora;
    • a railway-domain speech recognizer (Masmoudi et al., 2014).

Interesting ideas

  • I1. The convention is an internal normalization target for systems rather than a rule for writers, with a separate converter that maps messy input onto it. This keeps downstream tools simple while tolerating noisy text.
  • I2. Words are split into explicit proclitic / stem / suffix / enclitic slots, which turns a long fused word into transparent pieces. This helps wherever many function morphemes attach to one host.
  • I3. A small list of lexical exceptions sits alongside the general rules and pins down words that the rules alone would leave ambiguous. Pairing rules with a short override list is a cheap way to get coverage without overcomplicating the rules.
  • I4. One rule (separating negation from indirect-object clitics) is chosen for consistency across dialects even though it is less natural for TUN itself. This trades local fit for reuse across related resources.
  • I5. An MSA-TUN lexicon is used to convert plentiful standard-language text into the scarce dialect, creating extra language-model training data. Converting a resource-rich variety into a resource-poor one is a general way to bootstrap data.

For Mova

How these ideas could serve Mova, our small open language model — its files, stages and tests:

  • I1 (tokens): A canonical spelling layer for informal English (nonstandard spellings, contractions, chat forms) could run before the tagger, while the original form is kept so every normalization step stays explainable.
  • I2 (tokens): A slot-based segmentation record could document how clitic-like pieces ('s, n't, 'll) are split from their hosts, giving the parser auditable tokens.
  • I3 (learn): Self-induced repair rules could come with an explicit exception lexicon, so known counterexamples are fixed directly instead of being forced into new general rules.
  • I4 (tree): When converting between UD dialects or treebanks, Mova could likewise prefer one shared convention over source-specific quirks, so parsers and silver data from different sources stay comparable.
  • I5 (learn): Mova could generate silver training text for rarer registers (e.g., archaic fable style) by lexically converting standard English sentences, then filter the results with witness parsers.

Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.

 Institutions

🎓Columbia UniversityNew York · United States241 works · 54,053 citations🎓University of SfaxSfax · Tunisia14 works · 231 citations

 Fields, topics and concepts

📚 Shelves 4The field of science this work belongs to (OpenAlex fields).Computer Science · 16,667 worksArtificial Intelligence · 16,194 worksArts and Humanities · 16,039 worksLanguage and Linguistics · 15,891 works

🧭 Topics 1Research subjects and languages the work is about.Semitic languages · 2,908 works

Cite

Cite as Inès Zribi, Rahma Boujelbane, Abir Masmoudi, Mariem Ellouze, Lamia Belguith and Nizar Habash. 2014. A Conventional Orthography for Tunisian Arabic. Proceedings of the Ninth International Conference on Language Resources and Evaluation (LREC'14).

The paper itself, not this page. BibTeX from the library's record of the work.

@inproceedings{zribi2014conventional,
  title = {{A Conventional Orthography for Tunisian Arabic}},
  author = {Inès Zribi, Rahma Boujelbane, Abir Masmoudi, Mariem Ellouze, Lamia Belguith and Nizar Habash},
  booktitle = {Proceedings of the Ninth International Conference on Language Resources and Evaluation (LREC'14)},
  year = {2014},
  doi = {10.63317/2jhid49as43h},
  url = {https://doi.org/10.63317/2jhid49as43h}
}

References

24 references, 4 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

  1. Boujelbane, R., Khemakhem, M. E., and Belguith, L. (2013a). Mapping rules for building dialect Tunisian Lexicon and Generating Corpora. In Proceeding of International Joint Conference on Natural Language Processing (IJCNLP 2013), Nagoya, Japan. find
  2. Boujelbane, R., Khemekhem M. E., BenAyed S., Belguith L. H. (2013b). Building bilingual lexicon to create Dialect Tunisian corpora and adapt language model. In Proceedings of the Second Workshop on Hybrid Approaches to Translation. find
  3. Diab, M., Habash, N., Rambow, O., Altantawy, M., and Benajiba, Y. (2010). COLABA : Arabic Dialect Annotation and Processing, In Proceedings of LREC Workshop on Semitic Language Processing, Malta, May 2010. pp. 66–74. find
  4. Eskander, R., Habash, N., Rambow, O., and Tomeh, N. (2013). Processing Spontaneous Orthography. In Proceedings of Conference of the North American Association for Computational Linguistics (NAACL), Atlanta, Georgia. find
  5. Gibson, M. (1998). Dialect contact in Tunisian Arabic: Sociolinguistic and structural aspects, In Ph.D. Dissertation, University of Reading. find
  6. Habash, Nizar. (2010) Introduction to Arabic Natural Language Processing, Synthesis Lectures on Human Language Technologies, Graeme Hirst, editor. Morgan & Claypool Publishers. find
  7. Habash, N., Diab, M., and Rambow, O. (2012a). Conventional Orthography for Dialectal Arabic. In: Proceedings of the Language Resources and Evaluation Conference (LREC), Istanbul.
  8. Habash, N., Eskander, R., and Hawwari A. (2012b). A Morphological Analyzer for Egyptian Arabic. In the Proceedings of the Workshop on Computational Research in Phonetics, Phonology, and Morphology (SIGMORPHON) in the North American chapter of the Association for Computational Linguistics (NAACL), Montreal, Canada.
  9. Habash, N., Roth, R., Rambow, O., Eskander, R., and Tomeh, N. (2013). Morphological Analysis and Disambiguation for Dialectal Arabic. In Proceedings of Conference of the North American Association for Computational Linguistics (NAACL), Atlanta, Georgia.
  10. Habash, N., Soudi, A., and Buckwalter T. (2007). On Arabic Transliteration. Book Chapter. In Arabic Computational Morphology: Knowledge-based and Empirical Methods. Editors Antal van den Bosch and Abdelhadi Soudi. ↗
  11. Hamdi, A., Boujelbane, R., Habash, R., and Nasr, A. (2013). The Effects of Factorizing Root and Pattern Mapping in Translating between Tunisian Arabic and Standard Arabic. In Proceedings of the Machine translation Summit (MT Summit), Nice, France. find
  12. Holes, C. (2004). Modern Arabic: Structures, Functions, and Varieties. (Georgetown University Press., p220). Washington. find
  13. Lawson, S., and Sachdev, I. (2000). Code Switching in Tunisia: attitudinal and behavioral dimensions. In Journal of Pragmatics 32, (9): 1343-61. find
  14. Maamouri, M., Buckwalter, T., Cieri, C. (2004). Dialectal Arabic Telephone Speech Corpus: Principles, Tool Design, and Transcription Conventions. In: NEMLAR International Conference on Arabic Language Resources and Tools, Cairo, pp. 22–23. find
  15. Masmoudi, A., Khemakhem, M. E., Estève, Y., Belguith, L. and Habash, N. (2014). The Tunisian Dialect phonetic dictionary for speech recognition. In: Proceedings of the Language Resources and Evaluation Conference (LREC), Iceland. find
  16. Mejri, S., and Baccouche, T. (2003). L‟atlas linguistique de Tunisie: repères méthodologiques pour la description du système dialectal. In: Lentin, J., Lonnet, A. (eds.) Mélanges David Cohen, pp. 47–54. Maisonneuve & Larose, Paris. find
  17. Mejri, S., Said, M., and Sfar, I. (2009). Pluringuisme et diglossie en Tunisie. In: Synergies Tunisie, vol. (1), pp. 53–74. find
  18. Pasha, A., Al-Badrashiny, M., Diab, M., El Kholy, A., Eskander, R., Habash, N., Pooleery, M., Rambow, O. and Roth, R. (2014). “MADAMIRA: A Fast, Comprehensive Tool for Morphological Analysis and Disambiguation of Arabic.” In Proceedings of the 9th International Conference on Language Resources and Evaluation, Reykjavik, Iceland.
  19. Peirera, C. (2011). Arabic in the North African Region. In W. Stefan (Ed.), The Semitic Languages. pp. 954–969. find
  20. Saidi, D. (2007). Typology of Motion Event in Tunisian Arabic. In LingO. pp. 196–203. find
  21. Talmoudi, F. (1980) A morphosyntactic study of Romance verbs in the Arabic dialects of Tunis, Susa, and Sfax, In Gӧteborg: Acta Universitatis Gothoburgensis. find
  22. Zawaydeh, B. Stallard, D., and Makhoul, J. (2003). Babylon Transcription Guidelines. http://ldc.upenn.edu/Catalog/docs/LDC2005S08/BBN-Babylontranscription-guidelines.pdf ↗
  23. Zribi, I., Graja, M., Khemakhem, M. E., Jaoua, M., and Belguith, L. H. (2013a). Orthographic Transcription for Spoken Tunisian Arabic. CICLing 2013, Part I, LNCS 7816 (pp. 153–163). find
  24. Zribi I., Khemakhem M. E., and Belguith, L. H. (2013b). Morphological analysis of Tunisian Dialect. In Proceeding of International Joint Conference on Natural Language Processing (IJCNLP 2013), Nagoya, Japan. ↗

Updated

↑ back to top

This page in Markdown → https://md.research.elamur.ai/free/en/papers/zribi-2014-conventional-orthography-tunisian-arabic/