Elamur Research⌕
☰
home / papers / 2024 / Normalized Orthography for Tunisian Arabic

Normalized Orthography for Tunisian Arabic

Summary

Tunisian Arabic (aeb) has no widely accepted spelling in Arabic script. Existing conventions, most notably the CODA* guidelines for dialectal Arabic, rely heavily on Modern Standard Arabic etymology. Speakers who do not know MSA well therefore produce inconsistent transcriptions, and morphological patterns become irregular. The authors propose NOTA, a revision of CODA* rather than a new standard built from scratch. The revision is driven by user testing that the Derja Association ran in 2017. In those sessions, participants transcribed a dictated passage in three ways: with Tunisian CODA rules, with the Tunisian Arabic Corpus as a lookup aid, or with no help at all. Other participants then reviewed each group's output, and applied linguists decided which changes to make. The resulting guidelines do the following:

  • add letters for sounds Arabic script lacks;
  • introduce a Pashto-style diacritic for the fourth vowel;
  • replace several etymology-based spelling rules with morphology-based ones.

The paper contains no quantitative evaluation. Its contribution is the guideline set itself, published on figshare. The authors argue that consistent spelling is a prerequisite for larger Tunisian corpora, lexicons, treebanks and morphological models.

Key points

  • NOTA keeps most CODA* rules and adds three Perso-Arabic letters for /p/, /v/ and /g/, matching how Tunisians already tend to write these sounds.
  • Tunisian has four vowel qualities (/a/, /e/, /i/, /u/). Instead of CODA*'s kasra-plus-alif for long /e/, NOTA uses the Pashto diacritic Zwarakai (U+0659) alongside Fatha, Damma and Kasra.
  • The rules for final Alif Maqsura follow morphology rather than MSA etymology:
    • A verb takes Alif Maqsura only if its 3rd person masculine singular present form ends in long /i/.
    • Nouns derived from such verbs follow the same rule.
  • Other revised rules:
    • Irregular imperatives are regularized by deriving them from the 2nd person present form minus the /t/ prefix.
    • The object pronoun is spelled with a distinct form, which separates "he did not call him" from "they did not call".
    • Loanword vowels are mapped onto native Tunisian vowels, for example [y] to /i/.
  • The largest existing resource, the Tunisian Arabic Corpus, holds 2,462 texts and 1,065,493 words. The kept CODA* rules, the consonant and vowel tables, and worked examples are on figshare (doi 10.6084/m9.figshare.25892974).

Interesting ideas

  • I1. Comparing guidance conditions: Give the same dictation to three groups (written rules, corpus examples to look up, no help), then have another group critique each output; this exposes where the rules themselves cause disagreement. — For Mova (learn): We could run the same three-condition test with LLM teacher tiers or annotators (UD guideline text vs. treebank examples vs. nothing) to find which annotation conventions are under-specified before trusting them as silver data.
  • I2. Morphology over history: When a form is ambiguous, decide its spelling from how the word behaves in its paradigm (its present-tense form), not from historical etymology, so the rule is checkable from the language itself. — For Mova (tokens): Lemma and morphology rules for irregular English forms could be keyed on observable paradigm evidence, which keeps each decision explainable from data.
  • I3. Patching a standard instead of replacing it: Keep an established standard, change only the rules with documented conflicts, and publish the kept rules separately from the changed ones. — For Mova (tree): Our UD dialect conversion could be maintained as an explicit, justified diff against official UD guidelines, so every deviation is traceable and reversible.
  • I4. Removing ambiguity at the surface: Choose a distinct written form for a pronoun so that two readings of one string no longer collide. — For Mova (back): When generating text from the meaning graph, the generator could detect surface strings with two parses and pick a variant that preserves the intended frame.
  • I5. Borrowing from a structurally similar system: For a missing vowel symbol, the authors borrowed a ready-made diacritic from Pashto, which has the same vowel inventory, rather than inventing one. — For Mova (world): When the ontology lacks a category, we could first look for an existing scheme in WordNet, FrameNet or Wikidata with a matching structure before minting a new one.

Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.

Fields: Linguistics & languages

Topics: Treebanks

Cite

The paper itself, not this page. BibTeX from the library's record of the work. DOI 10.48550/arxiv.2402.12940 · arXiv 2402.12940

@misc{turki2024normalized,
  title = {{Normalized Orthography for Tunisian Arabic}},
  author = {Turki, Houcemeddine and Ellouze, Kawthar and Ammar, Hager Ben and Taieb, Mohamed Ali Hadj and Adel, Imed and Aouicha, Mohamed Ben and Farri, Pier Luigi and Bennour, Abderrezak},
  year = {2024},
  doi = {10.48550/arxiv.2402.12940},
  eprint = {2402.12940},
  archivePrefix = {arXiv},
  url = {https://doi.org/10.48550/arxiv.2402.12940}
}

References

15 references, 2 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

  1. 1. AlKliwiter, W., Al-Twairesh, N.: Part-of-speech tagging for Arabic tweots using CRF and Bi-LSTM. Computer Speech& Language 65, 101138 (Jan 2021) 2. Alsudais, A., Alotaibi, W., Alomary, F.: Similarities between Arabic dialects: Investigating geographical proximity. Information Processing& Management 59(1), 10277(J (Can 2022) 3. Althobaiti, M.J.: Creation of annotated country-level dialectal Arabic resources: An unsupervised approach. Natural Language Engineering 28(5), 507-648 (Aug 2021) 4. Baccouche, T.: Lo phonéme /g/ dans lmsparlors arabes citadins de Tunisia. In: Proceedings of the seventh International Congress of Phonetic Sciences/ Actes du Septiéme Congrés international des sciences phonétiques, pp. 652W55. De Gruyter (Dec 1ST2) 5. BBN Technologies, Makhoul, John, Zawaydeh, Bushra, Choi, Frederick, Stallard, David: BBN/AUB DARPA Babylon Levantine Arabic Speech and Transcripts. find our card
  2. Linguistic Data Consortium (2005) 6. Ben Abdelkader, R.:Peace Corps English-Tunisian Arabic Dictionary. find
  3. ERIC Clearinghouse (1977) 7. Bender, E.M., Flickingor, D., Oopon, S.: The grammar matrix: an opon-sourco starter-kit for the rapid development of cross-linguistically consistent broad-coverage precision grammars. In:COLING-02 on Grammar Engineering and Evaluation. Association for Computational Linguistics (2002) S. Besdouri, F.Z., Mekki, A., Zribi, I., Ellouze, M.: Improvement of the cota-orthography system through language modeling. In:2021 IEEE/ACS 16th International Conference on Computer Systems and Applications (AICCSA). find
  4. IEEE (Nov 2021) Néplo, S., Bñtora, J., Benkato, A., Miliéka, J., Pereira, C., Zemanek, P.:Mutual intelligibility of spoken Maltese, Libyan Arabic, and Tunisian Arabic functionally tested:A pilot study. Folia Linguistica 50(2) (Jan 2016) 10. Daoud, M.: The Language Situation in Tunisia. Current Issues in Language Planning 2(1), 1-52 (Nov 2001) 11. Eryani, F., Habash, N.,Bouamor, H.,Khalifa, S.:A Spelling Correction Corpus for Multiple Arabic Dialects. In: Proceedings of the Twelfth Language Resources and Evaluation Conference. pp. 4130-4138. find our card
  5. European Language Resources Association, Marseille, France (May 2020) 12. Gibson, M.: Ttinis Arabic. Encyclopedia of Arabic language and linguistics 4, 563— 571 (2009) 13. Gibson, M.: Dialect Levelling in Tunisian Arabic: Towards a New Spoken Standard. In:Language Contact and Language Conflict in Arabic, pp. 42-58. find
  6. Routledge (May 2013) Normalized Ortliography for Tunisian Arabic 14. C'ugliotta, E., Mallia, M., Panasci, L.: Towards a Unified Digital Resource for Tunisian Arabic. In: Carvalho, S., Khan, A.F., Anic, A.O., Spahiu, B., Gracia, J., McCrae, J.P., Gromann, D.,Heinisch, B., Salgado, A. (eds.) Proceedings of the 4th Conference on Language, Data and Knowledge. pp. 579-590. find
  7. NOVA CLUNL, Portugal, Vienna, Austria (Sep 2023) 15. Habash, N., Diab, M., Rambow, 0.:Conventional Orthography forDialectal Arabic. In: Proceedings of the Eighth International Conference on Language Resources and Evaluation (LREC’12). pp. 711-715. European Language Resources Association (ELRA), Istanbul, Turkey (May 2012) 16. Habash, N., Eryani, F., Khalifa, S., Rambow, 0.,eta1.: Unified Guidelines and Resources forArabic Dialect Orthography. In:Proceedings of the Eleventh International Conference on Language Resources and Evaluation (LREC 2018). European Language Resources Association (ELRA), Miyazaki, Sapan (May 2015) 1T. Hanidi, A., Nasr, A., Habash, N., Gala, N.: POS-tagging ofTtinisian Dialect Using Standard Arabic Resources and Tools. In: Proceedings of the Second Workshop on Arabic Natural Language Processing. Association for Computational Linguistics (2015) 18. Ka11e1, M.A.: Clioix langagiers sur la radio Mosaique FM. Dispositifs d'invisibilité et de normalisation sociales. Langage et société 138(4), 77-96 (Dec 2011) Kassab, A., Ounaies, A., Ben Achour Abdelkéfi, R., Louati, A., Mosbah, C.,Sakli, M.: Histoire Générale de la Tunisie: Tome IV - L’Époque contemporaine. find
  8. Sud Éditions (2010) 20. Maamouri, M.: The Phonology of Tunisian Arabic. find
  9. Cornell University (1967) 21. Maamouri, M.: Language education and liuman development: Arabic diglossia and its impact on the quality of education in the Arab region. ERIC Clearinghouse (1558) 22. Maamouri, M., Bies, A., Buckwalter, T., Diab, M., et al.: Developing and Usinga Pilot Dialectal Arabic Tíeebank. In: Proceedings of the Fifth International Conference on Language Resources and Evaluation (LREC’06). European Language Resources Association (ELRA), Genoa, Italy (May 200fi) 23. Maamouri, M., Graff, D., Jin, H., Cieri, C., Buckwalter, T.: Dialectal Arabic Ortliography-based Transcription& CTS Levantine Arabic Collection. the Parallel STT-NA Tracks Session of the EARS RT-04 Workshop, Palisades IBM Executive Center, New York, Nov. 10,2004 (2004) 24. Masmoudi, A.,Hamdi, J., Hadrich Belguith, L.: Deep Learning forSentiment Analysis of Tunisian Dialect. Computacióny Sistemas 25(1) (Feb 2021) 25. Masmoudi, A., Khmekhem, M.E., Khrouf, M., Belguith, L.H.: Transliteration of Arabizi into Arabic Script for Tunisian Dialoct. ACM Transactions on Asian and Low-Resource Language Information Processing 19(2), 1-21 (Nov 2019) 26. Masmoudi, A., Mdhaffar, S., Sellami, R., Belguith, L.H.: Automatic Diacritics Restoration for Tunisian Dialect. ACM Ti ansactions on Asian and Low-Resource Language Information Processing 18(3), 1-18 (Jul 20líib) 2T. McNeil, K.: Tunisian Arabic Corpus: Creating a Writton Corpus ofan ‘Unwritten’ Language, p. 30-55. find
  10. Edinburgh University Press (2015) 28. McNeil, K.L.: Tunisian Arabic asa Written Language: Vernacularization and Identity. find
  11. Georgetown University (2023) 2/J. Mekki, A., Zribi, I., Ellouze, M., Belguith, L.: Cota 2.0: an automatic corrector of tunisian arabic social media texts. Jordanian Journal of Computers and Information Technology 8(4), 370-387 (2022) 30. Mekki, A., Zribi, I., Ellouze, M., Hadrich Belguith, L.: Critical description of TA linguistic resources. Procedia Computer Science 142, 230-237 (2018) H. Turki et a1. 31. Ranta, A.: Grammatical Framework. Journal of Functional Programming 14(2), 145-189 (Jan 2004) 32. Rosa, C.L.: Mahdia Dialect: An Urban Vernacular in the Tunisian Sahel Context. Languages 6(3), 145 (Aug 2021) 33. Rosso, P.,Rangel, F.,Farlas, I.H., Cagnina, L.,et a1.:A survey on author profiling, deception, and irony detection for the Arabic language. Language and Linguistics Compass 12(4) (Apr 2018) 34. Saddour, I.: The expression of progressivity in Ttinisian Arabic:A study ofprogres-sive markers in oral retellings of simultaneous situations. Revue de Sémantique et Pragmatique 25-26(Espace temps, Interprétations spatiales/ Interprétations temporelles?), 2fi5-280 (Apr 2009) 35. Sayahi, L.: Diglossia and Language Contact: Language variation and change in North Africa. find
  12. Cambridge University Press (Apr 2014) 36. Sghaier, M.A., Zrigui, M.: Rule-Based Machine Translation from Tunisian Dialect to Modern Standard Arabic. Procedia Computer Science 176, 31W319 (2020) 37. Shiri, S.: The Homestay in Intensive Language Study Abroad: Social Networks, Language Socialization, and Developing Intercultural Competence. Foreign Language Annals 48(1), 5-25 (Mar 2015) 36. Singer, H.R.: Grammatik der arabischen Mundart der Medina von Tunis. find
  13. DE GRUYTER (Jun 1954) 39. Turki, H., Ade1, I., Daouda, T., Regragui, N.:A Conventional Orthography for Maghrobi Arabic. In: Procoodings of the International Conference on Language Resources and Evaluation (LREC) (2016) 40. Turki, H., Vrandecic, D., Haindi, H., Ade1, I.: Using WikiData as a Multi-lingual Multi-dialectal Dictionary for Arabic Dialects. In: 2017 IEEE/ACS 14th International Conference on Computer Systems and Applications (AICCSA). find
  14. IEEE (Oct 2017) 41. Younes, .J., Souissi, E., Achour, H., Ferchichi, A.: Language resources for Maghrebi Arabic dialects' NLP: a suiwey. Language Resources and Evaluation 54(4), 1079— 1142 (Apr 2020) 42. Zalmout, N., Habash, N.: Adversarial Multitask Learning for.Ioint Multi-Feature and Multi-Dialect Morphological Modeling. In: Proceedings of the 57th Annual Meeting ofthe Association for Computational Linguistics. Association for Computational Linguistics (2019) 43. Zribi, I., Boujelbane, R., Masmoudi, A., Ellouze, M., et a1.:A Conventional Orthography forTunisian Arabic. In: Proceedings of the Ninth International Coifer-ence on Language Resources and Evaluation (LREC’14). pp. 2355-2361. European Language Resources Association (ELRA), Reykjavik, Iceland (May 2014) 44. Zribi, I., Ellouze, M., Belguith, L.H., Blache, P.: Morphological disambiguation of Tunisian dialect. Journal of King Sand University - Computer and Information Sciences 29(2), 147-155 (Apr 2017) 45. Zribi, I., C'raja, M., Ellouze Khmekhem, M.,Jaoua, M., Hadricli Belguith, L.: Orthographic Transcription for Spoken Tunisian Arabic, p. 153-163. find
  15. Springer Berlin Heidelberg (2013) find

↑ back to top

This page in Markdown → https://md.research.elamur.ai/free/en/papers/turki-2024-normalized-orthography-tunisian-arabic/