Elamur Research⌕
☰
home / papers / 2023 / Towards a Unified Digital Resource for Tunisian Arabic

Towards a Unified Digital Resource for Tunisian Arabic

Paper · 2023 · Elisa Gugliotta, Michele Mallia and Livia Panascì

Summary

Tunisian Arabic (TA) is an under-resourced dialect. It has no standard spelling and mostly oral use, and the few resources that exist were built for different purposes and never connected. The authors present an ongoing pilot study that links two such resources: a TA–Italian lexicographic database and TArC, a corpus of informal TA writing from digital contexts. They place this inside a reusable three-step, FAIR-oriented methodology: reuse existing material instead of building from scratch, standardise and semi-automatically annotate it with existing NLP tools, then publish data and tools. The cycle restarts with a new language. The two resources write lemmas differently (scientific transliteration versus CODA-Star normalisation), so the team joins them through the root instead. Roots for TArC lemmas were produced with CAMeL Tools' MSA analyser and then corrected by hand. In the end, about three quarters of TArC's unique roots also appear in the dictionary. This shows that cross-resource linking is feasible and sets up a queryable platform and planned morphological embeddings.

Key points

  • TArC input to the analyser:
    • The corpus has 43,349 tokens and 5,063 unique lemmas.
    • 9,363 tokens were left out: foreign, emotag, PUNC, NOUN_NUM and NOUN_PROP.
    • The remaining 33,986 lemmas went to the CAMeL Tools morphological analyser.
  • Root choice: of the CAMeL Tools databases tested (MSA, Egyptian, Gulf), only the MSA one returned roots, so it was applied to dialect data.
  • CAMeL Tools results:
    • 23,913 roots were correct (roughly 70%).
    • 6,056 were wrong, meaning MSA-based roots that do not fit TA.
    • 4,017 tokens were not found at all.
  • Linking result: after manual correction, TArC has 1,356 unique roots. Of these, 1,034 (76.3%) match roots in the lexicographic database.
  • Planned outputs:
    • a query platform that correlates both sources through a central Analysis Node;
    • morphological embeddings that combine morphemes, POS tags and lemmas;
    • tools released at github.com/LinguaeVerse.

Interesting ideas

  • I1. Pick a join key both resources share: when two resources normalise surface forms differently, link them through a deeper shared unit (the root) rather than through lemmas, because the root survives differences in spelling conventions.
  • I2. Use a nearby standard tool, then audit its output: the authors ran a tool built for a related standard variety on dialect data and sorted its output into three bins: not found, wrong, and correct. This makes the size of the domain gap explicit and shows exactly where manual repair is needed.
  • I3. Leave out token classes the tool cannot handle: before analysis, they removed punctuation, numerals, proper nouns, foreign words and emotags, which keeps the error statistics meaningful.
  • I4. Keep fused words whole but link their parts: a fused word (like blāš, "without") is kept as its own lemma while also pointing to its components, so no information is lost and the form is not forced into a decomposition.
  • I5. Reuse first, as a repeating cycle: the method starts by collecting existing resources made for other goals, such as sentiment corpora without grammar labels, and enriches them. It then repeats the same loop for the next language.

For Mova

How these ideas could serve Mova, our small open language model — its files, stages and tests:

  • I1 (world): Ontology sources like WordNet, FrameNet and Wikidata could be aligned through a normalised shared key (e.g., a canonical sense or concept ID) rather than raw lemma strings, which often disagree in spelling or inflection.
  • I2 (learn): Silver tags or parses on out-of-domain text (fables, school tests) could be reported with this three-way breakdown, so repair-rule induction targets the "wrong" bin separately from simple coverage gaps.
  • I3 (tokens): Lemmatiser and morphology evaluations could exclude or report such classes separately, so accuracy figures are not inflated or skewed by trivial or out-of-scope tokens.
  • I4 (tokens): Contractions and multiword expressions could keep a whole-form lemma plus links to their parts, which fits UD's multiword-token convention without losing the original token.
  • I5 (learn): New training domains could begin by gathering and re-annotating existing labelled sets, then feeding them through the witness-parser agreement loop, instead of commissioning fresh annotation.

Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.

 Fields, topics and concepts

📚 Shelves 2The field of science this work belongs to (OpenAlex fields).Computer Science · 16,667 worksArtificial Intelligence · 16,194 works

🧭 Topics 1Research subjects and languages the work is about.Semitic languages · 2,908 works

Cite

Cite as Elisa Gugliotta, Michele Mallia and Livia Panascì. 2023. Towards a Unified Digital Resource for Tunisian Arabic. Proceedings of the 4th Conference on Language, Data and Knowledge.

The paper itself, not this page. BibTeX from the library's record of the work.

@inproceedings{gugliotta2023unified,
  title = {{Towards a Unified Digital Resource for Tunisian Arabic}},
  author = {Elisa Gugliotta, Michele Mallia and Livia Panascì},
  booktitle = {Proceedings of the 4th Conference on Language, Data and Knowledge},
  year = {2023},
  url = {https://aclanthology.org/2023.ldk-1.62/}
}

References

68 references, 9 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

  1. Soha Abboud-Haggar. 2006. Dialects: Genesis. In Encyclopedia of Arabic Language and Linguistics, volume I, pages 613–622. Brill, Leiden – Boston. find
  2. Diana Alhafi, Anton Deik, Elhadj Benkhelifa, and Mustafa Jarrar. 2019. Usability Evaluation of Lexicographic e-services. In 2019 IEEE/ACS 16th International Conference on Computer Systems and Applications (AICCSA). IEEE. find
  3. Taïeb Baccouche. 1994. L’emprunt en arabe moderne. Académie tunisienne des sciences des lettres et des arts, Beït al-Hikma. find
  4. Taïeb Baccouche. 2009. Tunisia. In Encyclopedia of Arabic Language and Linguistics, volume IV, pages 571–577. Brill, Leiden – Boston. find
  5. Marcelin Beaussier, Mohamed Ben Cheneb, and Albert Lentin. 2006. Dictionnaire Pratique Arabe-Français (Arabe Maghrébin). Ibis Press, Paris. find
  6. Peter Behnstedt. 1998. Zum Arabischen von Djerba (Tunesien) I. Zeitschrift für Arabische Linguistik, 35, pages 52–83. find
  7. Peter Behnstedt. 1999. Zum Arabischen von Djerba (Tunesien) II: Texte. Zeitschrift für Arabische Linguistik, 36, pages 32–65. find
  8. Rached Ben Abdelkader et al. 1977. Peace Corps English-Tunisian Arabic Dictionary. Peace Corps, Washington D.C. find
  9. Wahid Ben Alaya and Michel Quitout. 2010. L’Arabe tunisien de poche – Guide de conversation. Assimil France, Chennevières sur Marne Cedex. find
  10. Hager Ben Ammar and Valérie Vacchiani. 2016. Parler tunisien fissa! Une méthode originale pour apprendre l’arabe tunisien en 6 mois. Editions Arabesques, Tunis. find
  11. Yoshua Bengio, Réjean Ducharme, Pascal Vincent, and Christian Janvin. 2003. A Neural Probabilistic Language Model. The Journal of Machine Learning Research, 3:1137–1155.
  12. Massimo Bevacqua. 2008. Osservazioni sul linguaggio dei giovani tunisini. Il filo di seta – Studi arabo-islamici in onore di Wasim Dahmash, pages 11–24. find
  13. Gilbert Boris. 1958. Lexique du parler arabe des Marazig. Imprimerie nationale – Librairie C. Klincksieck, Paris. find
  14. Houda Bouamor, Nizar Habash, and Kemal Oflazer. 2014. A Multidialectal Parallel Corpus of Arabic. In Proceedings of the 9th Language Resources and Evaluation Conference (LREC), pages 1240–1245.
  15. Naima Boussofara-Omar. 2006. Diglossia. In Encyclopedia of Arabic Language and Linguistics, volume I, pages 629–637. Brill, Leiden – Boston. For unique root, we mean the roots counted only once. find
  16. Abdelghani Bouziane, Djelloul Bouchiha, and Noureddine Doumi. 2020. Annotating Arabic texts with linked data. In 2020 4th International Symposium on Informatics and its Applications (ISIA), pages 1–5. IEEE. find
  17. Dominique Caubet. 2019. Vers une littératie numérique pour la darija au maroc, une démarche collective. In Catherine Miller, Alexandrine Barontini, Marie-Aimée Germanos, Jairo Guerrero Guerrero, and Christophe Pereira, editors, Studies on Arabic Dialectology and Sociolinguistics. Proceedings of the 12th International Conference of AIDA. Livres de l’IREMAM. ↗
  18. Guido Cifoletti. 1998. Osservazioni sugli italianismi nel dialetto di Tunisi. Incontri linguistici, 21:137–153. find
  19. Guido Cifoletti. 2004. La lingua franca barbaresca. Il Calamo. find
  20. Ryan Cotterell and Hinrich Schütze. 2015. Morphological Word-Embeddings. In Proceedings of the 2015 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, pages 1287–1292, Denver, Colorado. Association for Computational Linguistics. find
  21. Ines Dallaji, Ines Gabsi, Stephan Procházka, and Karlheinz Mörth. 2020. A Digital Dictionary of Tunis Arabic-TUNICO (ELEXIS). Slovenian language resource repository CLARIN.SI. find
  22. Mohamed Daoud. 2007. The Language Situation in Tunisia. Language planning and policy in Africa, Vol. II: Algeria, Côte d’Ivoire, Nigeria and Tunisia, pages 256–308. ↗
  23. Kareem Darwish, Nizar Habash, Mourad Abbas, Hend Al-Khalifa, Huseein T Al-Natsheh, Houda Bouamor, Karim Bouzoubaa, Violetta Cavalli-Sforza, Samhaa R El-Beltagy, Wassim El-Hajj, et al. 2021. A panoramic survey of natural language processing in the arab world. Communications of the ACM, 64(4):72–81. ↗
  24. Humphrey Davies. 2006. Dialect Literature. In Encyclopedia of Arabic Language and Linguistics, volume I, pages 597–604. Brill, Leiden – Boston. find
  25. FMG De Jong, Bente Maegaard, Koenraad De Smedt, Darja Fišer, Dieter Van Uytvanck, et al. 2018. Clarin: Towards FAIR and responsible data science using language resources. In Proceedings of the Eleventh International Conference on Language Resources and Evaluation (LREC 2018), pages 3259–3264. find
  26. Olivier Durand and Maura Tarquini. 2023. Corso di arabo tunisino. Manuale di comunicazione con grammatica ed esercizi. Livelli A1-B2 del Quadro Comune Europeo di Riferimento per le Lingue. Ulrico Hoepli Editore S.p.A., Milano. find
  27. Fadhl Eryani, Nizar Habash, Houda Bouamor, and Salam Khalifa. 2020. A spelling correction corpus for multiple Arabic dialects. In Proceedings of the 12th Language Resources and Evaluation Conference, pages 4130–4138.
  28. Charles A. Ferguson. 1959. 15(2):325–340. Diglossia. find
  29. Word, Darja Fišer and Andreas Witt. 2022. CLARIN: The Infrastructure for Language Resources, volume 1. Walter de Gruyter GmbH & Co KG. find
  30. Elisa Gugliotta. 2022. Realization of a Tunisian Arabish Corpus with use within the scope of NLP-Natural Language Processing. Ph.D. thesis, Sapienza University of Rome and Université Grenoble Alpes. ↗
  31. Elisa Gugliotta and Marco Dinarelli. 2022. Tarc: Tunisian arabish corpus first complete release. In 13th Conference on Language Resources and Evaluation (LREC 2022).
  32. Elisa Gugliotta and Marco Dinarelli. 2023. An empirical analysis of task relations in the multi-task annotation of an arabizi corpus. Accepted paper for the 4th Conference on Language, Data and Knowledge.
  33. Elisa Gugliotta, Marco Dinarelli, and Olivier Kraif. 2020. Multi-task sequence prediction for Tunsian arabizi multi-level annotation. In Proceedings of the Fifth Arabic Natural Language Processing Workshop, pages 178–191. find
  34. Nizar Habash, Fadhl Eryani, Salam Khalifa, Owen Rambow, Dana Abdulrahim, Alexander Erdmann, Reem Faraj, Wajdi Zaghouani, Houda Bouamor, Nasser Zalmout, et al. 2018. Unified guidelines and resources for Arabic dialect orthography. In Proceedings of the 11th Language Resources and Evaluation Conference (LREC).
  35. Mustafa Jarrar and Hamzeh Amayreh. 2019. An Arabic-Multilingual Database with a Lexicographic Search Engine. In Natural Language Processing and Information Systems, pages 234–246. Springer International Publishing. find
  36. Mustafa Jarrar, Fadi A. Zaraket, Tymaa Hammouda, Daanish Masood Alavi, and Martin Waahlisch. 2022. Lisan: Yemeni, Iraqi, Libyan, and Sudanese Arabic dialect corpora with morphological annotations. arXiv preprint arXiv:2212.06468. ↗
  37. J. Jourdan. 1913. Cours normal et pratique d’Arabe Parlé – Vocabulaire – Historiettes – Proverbes – Chants – Dialecte Tunisien, 4e édition. Éditions Bouslama, Tunis. find
  38. Saméh Kchaou, Rahma Boujelbane, and Lamia Hadrich Belguith. 2022. Hybrid pipeline for building Arabic Tunisian dialect-standard Arabic neural machine translation model from scratch. ACM Transactions on Asian and Low-Resource Language Information Processing. ↗
  39. Emna Labidi. 2017. L’artisanat traditionnel à Tunis – Sa terminologie et son lexique. Tunisian and Libyan Arabic Dialects – Common Trends – Recent Developments – Diachronic Aspects, pages 147–160. find
  40. Jérôme Lentin. 2008. Middle Arabic. In Encyclopedia of Arabic Language and Linguistics, volume III, pages 215–224. Brill, Leiden – Boston. find
  41. Livia Panascì. 2021. Studi lessicali sull’arabo di Tunisia, volume I, II, III, IV. PhD Thesis in Civilizations of Asia and Africa, Sapienza University of Rome, Rome. find
  42. Mohamed Maamouri, Ann Bies, Tim Buckwalter, and Wigdan Mekki. 2004. The Penn Arabic Treebank: Building a large-scale annotated Arabic corpus. In NEMLAR conference on Arabic language resources and tools, volume 27, pages 466–467. Cairo. find
  43. Laurette Pretorius and Claudia Soria. 2017. Introduction to the Special Issue. Language resources and evaluation, 51:891–895. ↗
  44. Agnese Macchiarelli. 2023. Sinergie fra vedph e cnr-ilc in termini di condivisione della conoscenza e sostenibilità dei progetti digitali. In DH. 22–Digital Humanities. Per un confronto interdisciplinare tra saperi umanistici a 30 anni dalla nascita del World Wide Web. L’Erma di Bretschneider. find
  45. Jean Quéméneur. 1961a. Notes sur quelques vocables du parler tunisien figurant au Supplement de A. Lentin - 1ère partie. Revue de l’I.B.L.A., Vol. 24/93, pages 1–22. find
  46. Abrerrahmân Marçais, William Guîga. 1961. Textes arabes de Takroûna par William Marçais et Abderrah-mân Guîga – II – Glossaire – Contribution à l’étude du vocabulaire arabe, Tome I – VIII. Imprimerie Nationale – Centre National de la Recherche Scientifique – Librairie Orientaliste Paul Geuthner, Paris. find
  47. Michel Quitout. 2002. Parlons l’arabe tunisien: Langue & culture. L’Harmattan, Paris - Budapest - Torino. find
  48. Jean Quéméneur. 1961b. Notes sur quelques vocables du parler tunisien figurant au Supplement de A. Lentin - 2ème partie. Revue de l’I.B.L.A., Vol. 24/94, pages 167–181. find
  49. Jean Quéméneur. 1962. Glossaire de dialectal 1942-1962. Revue de l’I.B.L.A., Vol. 25/100, pages 325–367. find
  50. Philippe Marçais and M.-S. Hamrouni. 1977. Textes d’arabe maghrébin. Librairie d’Amérique et d’Orient, Adrien Maisonneuve – J. Maisonneuve, succ., Paris. find
  51. Veronika Ritt-Benmimoun. 2014. Grammatik des arabischen Beduinendialekts der Region Douz (Südtunesien). Harrassowitz Verlag, Wiesbaden. ↗
  52. William Marçais. 1950. Les parlers arabes. Initiation à la Tunisie, pages 195–219. find
  53. Vitaly Romanov and Albina Khusainova. 2019. Evaluation of Morphological Embeddings for the Russian Language. In Proceedings of the 2019 3rd International Conference on Natural Language Processing and Information Retrieval, pages 144–148. find
  54. Karen McNeil. 2018. Tunisian Arabic corpus: Creating a written corpus of an ‘unwritten’ language. Arabic corpus linguistics, 30. find
  55. Rana Aref Salama, Abdou Youssef, and Aly Fahmy. 2018. Morphological word embedding for Arabic. Procedia computer science, 142:83–93. find
  56. Salima Mdhaffar, Fethi Bougares, Yannick Esteve, and Lamia Hadrich-Belguith. 2017. Sentiment analysis of Tunisian dialects: Linguistic ressources and experiments. In Third Arabic Natural Language Processing Workshop (WANLP), pages 55–61.
  57. Lotfi Sayahi. 2014. Diglossia and language contact: Language variation and change in North Africa. Cambridge University Press. find
  58. Karima Meftouh, Salima Harrat, and Kamel Smaïli. 2018. Padic: Extension and new experiments. In 7th International Conference on Advanced Technologies ICAT. find
  59. Erhan Sezerer and Selma Tekir. 2021. A Survey on Neural Word Embeddings. arXiv preprint arXiv:2110.01804. ↗
  60. Ouafae Nahli, Elisa Gugliotta, Nadia Khlif, and Giulia Benotto. 2023. Advancing dialectal analysis: Annotating corpora and building lexical resources for Arabic dialects. Forthcoming. find
  61. Hans Stumme. 1896. Grammatik des tunisischen arabisch: nebst Glossar. JC Hinrichs. find
  62. Alfred Nicolas. (s.d.) [1911]. Dictionnaire Français-Arabe, Idiome Tunisien. Imprimeur - Éditeur Frédéric Weber, Tunis. find
  63. Fathi Talmoudi. 1981. Texts in the Arabic Dialect of Sūsa (Tunisia): Transcription, Translation, Notes and Glossary. Acta Universitatis Gothoburgensis, Göteborg. find
  64. Kees Versteegh. 2014. Arabic language. Edinburgh University Press. find
  65. Ossama Obeid, Nasser Zalmout, Salam Khalifa, Dima Taji, Mai Oudah, Bashar Alhafni, Go Inoue, Fadhl Eryani, Alexander Erdmann, and Nizar Habash. 2020. CAMeL tools: An open source python toolkit for Arabic natural language processing. In Proceedings of the 12th Language Resources and Evaluation Conference, pages 7022–7032, Marseille, France. European Language Resources Association.
  66. Mark Wilkinson, Michel Dumontier, I. Jsbrand Jan Aalbersberg, Gabrielle Appleton, Myles Axton, Arie Baak, Niklas Blomberg, Jan-Willem Boiten, Luiz Bonino da Silva Santos, Philip E. Bourne, et al. 2016. The FAIR Guiding Principles for Scientific Data Management and Stewardship. Scientific data, 3(1):1–9. find
  67. Jonathan Owens. 2006. A Linguistic History of Arabic. Oxford: Oxford University Press. ↗
  68. Sane Yagi, Ashraf Elnagar, and Shehdeh Fareh. 2022. A Benchmark for Evaluating Arabic Word Embedding Models. Natural Language Engineering, pages 1–26. ↗

Updated

↑ back to top

This page in Markdown → https://md.research.elamur.ai/free/en/papers/gugliotta-2023-unified-digital-resource-tunisian-arabic/