Elamur Research⌕
☰
home / papers / 2022 / TArC: Tunisian Arabish Corpus, First complete release

TArC: Tunisian Arabish Corpus, First complete release

Summary

Tunisian Arabic has few resources, and fewer still for Arabizi, the Latin-letter-and-digit script Tunisians use online. The available corpora are scarce, often not public, or only lightly annotated. The authors release the complete version of TArC (Tunisian Arabish Corpus) together with the neural tool they built to annotate it. The tool is a multi-task system that adds layers in sequence: word classification, transliteration into Arabic script following the CODA Star orthographic conventions, tokenization, POS tagging, and finally lemmatization, which arrived last in this release. Annotation was semi-automatic, with model output checked by hand. The authors deliberately kept the corpus modest in size and invested in annotation accuracy and depth, aiming to serve both NLP and dialectology. Code and data are public, so the experiments can be reproduced, and the pipeline could produce "twin" corpora for other Arabic dialects. The excerpt does not include the paper's quantitative results.

Key points

  • Two separate but linked resources: the TArC corpus (github.com/eligugliotta/tarc) and the tarc-multi-task-system annotation tool (GRICAD GitLab, Grenoble).
  • Annotation levels:
    • word classification
    • transliteration to Arabic script
    • tokenization
    • POS tagging
    • lemmatization
  • CODA Star was chosen as the normalization convention so that TArC stays compatible with other dialectal Arabic corpora such as the CODA MADAR Corpus.
  • Arabizi lacks spelling norms but records some phonetics that Arabic script hides, such as article assimilation. Earlier work (Guellil et al., 2018) found transliteration necessary to reduce this ambiguity.
  • Comparable Tunisian resources are small or narrow:
    • STAC: 42,388 words of transcribed speech
    • TSAC: 17,060 Facebook comments with polarity labels
    • LETD: 43,222 web messages

Interesting ideas

  • I1. Each annotation layer is predicted in cascade, conditioned on the layers before it, so later decisions (POS, lemma) operate on cleaned input rather than raw noisy spelling. — For Mova (tokens): In crate en, morphology and lemma prediction could explicitly take the normalized token and tagger output as inputs, keeping a visible trace of which earlier layer caused each choice.
  • I2. A word-classification step runs before anything else, separating native dialect words from foreign or other material so each token gets the right treatment downstream. — For Mova (tokens): An early router could flag foreign words, numerals, emoticons and other noise in English input before tagging, so the tagger and lemmatizer are not forced to treat them as ordinary English.
  • I3. The corpus is normalized to a shared orthographic convention (CODA Star) while the original user spelling is kept, which makes the resource joinable with others without losing the raw signal. — For Mova (tree): During UD dialect conversion, gparse training data could keep both the source-treebank form and the normalized form side by side, letting conversions be audited and reversed.
  • I4. The bootstrapping loop has the model pre-annotate and humans correct, and the corrected data then feeds the next round; a new layer (lemmas) was added the same way. — For Mova (learn): Silver data agreed by witness parsers could go through targeted human or LLM-teacher correction on disagreements only, growing new annotation layers incrementally rather than labelling from scratch.
  • I5. The authors prefer a smaller corpus with dense, verified, multi-level annotation over a large shallow one. — For Mova (answer): Evaluation sets for school tests and fables could be small but annotated at every level (tree, frames, world state), so a wrong answer can be traced to the exact stage that failed.

Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.

Fields: Linguistics & languages

Topics: Part-of-speech tagging

Cited by

4 papers of our library cite it (of the 140884 whose bibliographies we read); 2 of them have a page here, 2 not yet.

Cite

The paper itself, not this page. BibTeX from the library's record of the work.

@inproceedings{gugliotta2022tarc,
  title = {{TArC: Tunisian Arabish Corpus, First complete release}},
  author = {Gugliotta, Elisa and Dinarelli, Marco},
  booktitle = {Proceedings of the Thirteenth Language Resources and Evaluation Conference},
  year = {2022},
  doi = {10.63317/5eqrikasc4rn},
  url = {https://doi.org/10.63317/5eqrikasc4rn}
}

References

82 references, 10 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

  1. Abdelkader, B. (1977). Peace Corps English-Tunisian Arabic Dictionary. ERIC Cleringhouse, Washington D.C. find
  2. Abdellatif, K. (2010). Dictionnaire “le Karmous" du tunisien. Online unter: https://www.fichier-pdf. fr/2010/08/31/m14401m/, Zugriff, 25:2017. ↗
  3. Abidi, K., Menacer, M. A., and Smaili, K. (2017). Calyou: A comparable spoken algerian corpus harvested from youtube. In 18th Annual Conference of the International Communication Association (Interspeech). find
  4. Al-Badrashiny, M., Eskander, R., Habash, N., and Rambow, O. (2014). Automatic transliteration of romanized dialectal Arabic. In Proceedings of the eighteenth conference on computational natural language learning, pages 30–38. find
  5. Al-Sabbagh, R. and Girju, R. (2012). Yadac: Yet another dialectal Arabic corpus. In Proceedings of the 8th Language Resources and Evaluation Conference (LREC), pages 2882–2889. find
  6. Al Sallab, A., Hajj, H., Badaro, G., Baly, R., El-Hajj, W., and Shaban, K. (2015). Deep learning models for sentiment analysis in Arabic. In Proceedings of the second workshop on Arabic natural language processing, pages 9–17. ↗
  7. Azouaou, F. and Guellil, I. (2017). Alg/fr: A step by step construction of a lexicon between algerian dialect and french. In The 31st Pacific Asia Conference on Language, Information and Computation PACLIC, volume 31. find
  8. Baccouche, T. (2011). Tunisia. Encyclopedia of Arabic language and linguistics, 4:571–577. find
  9. Bengio, Y. (2012). Practical recommendations for gradient-based training of deep architectures. CoRR, abs/1206.5533. ↗
  10. Bouamor, H., Habash, N., and Oflazer, K. (2014). A Multidialectal Parallel Corpus of Arabic. In Proceedings of the 9th Language Resources and Evaluation Conference (LREC), pages 1240–1245. find our card
  11. Bouamor, H., Habash, N., Salameh, M., Zaghouani, W., Rambow, O., Abdulrahim, D., Obeid, O., Khalifa, S., Eryani, F., Erdmann, A., et al. (2018). The madar arabic dialect corpus and lexicon. In Proceedings of the Eleventh International Conference on Language Resources and Evaluation (LREC 2018). find
  12. Buckwalter, T. and Parkinson, D. (2014). A frequency dictionary of Arabic: Core vocabulary for learners. Routledge. ↗
  13. Chalabi, A. and Gerges, H. (2012). Romanized arabic transliteration. In Proceedings of the Second Workshop on Advances in Text Input Methods, pages 89– 96. find
  14. Cotterell, R., Renduchintala, A., Saphra, N., and Callison-Burch, C. (2014). An algerian arabic-french code-switched corpus. In Workshop on Free/Open-Source Arabic Corpora and Corpora Processing Tools Workshop Programme, page 34. find
  15. Darwish, K., Habash, N., Abbas, M., Al-Khalifa, H., Al-Natsheh, H. T., Bouamor, H., Bouzoubaa, K., Cavalli-Sforza, V., El-Beltagy, S. R., El-Hajj, W., et al. (2021). A panoramic survey of natural language processing in the arab world. Communications of the ACM, 64(4):72–81. ↗
  16. Darwish, K. (2014). Arabizi detection and conversion to Arabic. In Proceedings of the EMNLP 2014 Workshop on Arabic Natural Language Processing (ANLP), pages 217–224. ↗
  17. Diab, M., Habash, N., Rambow, O., Altantawy, M., and Benajiba, Y. (2010). Colaba: Arabic dialect annotation and processing. In Lrec workshop on semitic language processing, pages 66–74. Citeseer. find
  18. El-Haj, M. (2020). Habibi - a multi dialect multi national Arabic song lyrics corpus. In Proceedings of the 12th Language Resources and Evaluation Conference, pages 1318–1326, Marseille, France, May. European Language Resources Association. find our card
  19. Eryani, F., Habash, N., Bouamor, H., and Khalifa, S. (2020). A spelling correction corpus for multiple arabic dialects. In Proceedings of the 12th Language Resources and Evaluation Conference, pages 4130– 4138. find our card
  20. Eskander, R., Al-Badrashiny, M., Habash, N., and Rambow, O. (2014). Foreign words and the automatic processing of Arabic social media text written in roman script. In Proceedings of The First Workshop on Computational Approaches to Code Switching, pages 1–12. find
  21. Farha, I. A. and Magdy, W. (2019). Mazajak: An online Arabic sentiment analyser. In Proceedings of the Fourth Arabic Natural Language Processing Workshop, pages 192–198. find
  22. Gadalla, H., Kilany, H., Arram, H., Yacoub, A., El-Habashi, A., Shalaby, A., and McLemore, C. (1997). Callhome egyptian Arabic transcripts ldc97t19. Web Download. Philadelphia: Linguistic Data Consortium. find
  23. Gibson, M. (2011). Tunis arabic. Encyclopedia of Arabic language and linguistics, 4:563–571. find
  24. Graja, M., Jaoua, M., and Hadrich Belguith, L. (2010). Lexical study of a spoken dialogue corpus in tunisian dialect. In Proceedings of The International Arab Conference on Information Technology, benghazi– libya. Citeseer. find
  25. Graja, M., Jaoua, M., and Belguith, L. H. (2013). Discriminative framework for spoken tunisian dialect understanding. In International Conference on Statistical Language and Speech Processing, pages 102–110. Springer. find
  26. Guellil, I. and Azouaou, F. (2016). Arabic dialect identification with an unsupervised learning (based on a lexicon). application case: Algerian dialect. In 2016 IEEE Intl Conference on Computational Science and Engineering (CSE) and IEEE Intl Conference on Embedded and Ubiquitous Computing (EUC) and 15th Intl Symposium on Distributed Computing and Applications for Business Engineering (DCABES), pages 724–731. IEEE. find
  27. Guellil, I. and Faical, A. (2017). Bilingual lexicon for algerian Arabic dialect treatment in social media. WiNLP: Women & Underrepresented Minorities in Natural Language Processing (Co-located with ACL 2017). find
  28. Guellil, I., Adeel, A., Azouaou, F., and Hussain, A. (2018). Sentialg: Automated corpus annotation for algerian sentiment analysis. In International Conference on Brain Inspired Cognitive Systems, pages 557–567. Springer. ↗
  29. Gugliotta, E. and Dinarelli, M. (2020). Tarc: Incrementally and semi-automatically collecting a tunisian arabish corpus. In Proceedings of The 12th Language Resources and Evaluation Conference, pages 6279–6286. ↗ our card
  30. Gugliotta, E., Dinarelli, M., and Kraif, O. (2020). Multi-task sequence prediction for Tunisian Arabizi multi-level annotation. In Proceedings of the Fifth Arabic Natural Language Processing Workshop, pages 178–191, Barcelona, Spain (Online), December. Association for Computational Linguistics. ↗
  31. Habash, N., Diab, M., and Rambow, O. (2012). Conventional orthography for dialectal Arabic. In Proceedings of the 8th Language Resources and Evaluation Conference (Proceedings of the 8th Language Resources and Evaluation Conference (LREC)), pages 711–718. find our card
  32. Habash, N., Eryani, F., Khalifa, S., Rambow, O., Abdulrahim, D., Erdmann, A., Faraj, R., Zaghouani, W., Bouamor, H., Zalmout, N., et al. (2018). Unified guidelines and resources for arabic dialect orthography. In Proceedings of the Eleventh International Conference on Language Resources and Evaluation (LREC 2018). find our card
  33. Hahn, M. (2020). Theoretical limitations of self-attention in neural sequence models. Transactions of the Association for Computational Linguistics, 8:156–171. ↗
  34. Harrat, S., Meftouh, K., Abbas, M., and Smaili, K. (2014). Building resources for algerian arabic dialects. In 15th Annual Conference of the International Communication Association Interspeech. find
  35. Haspelmath, M. and Tadmor, U. (2009). The loanword typology meaning list: Electronic databases of 29 languages. A collaborative project coordinated by the Max Planck Institute for Evolutionary Anthropology, Department of Linguistics. find
  36. Hochreiter, S. and Schmidhuber, J. (1997). Long short-term memory. Neural Comput., 9(8):1735– 1780, November. ↗
  37. Karmani, B. M. N. and Alimi, A. M. (2015). Construction d’un wordnet standard pour l’arabe tunisien. In Proceedings of Colloque pour les Étudiants Chercheurs en Traitement Automatique du Langage naturel et ses applications, Sousse, Tunisia. find
  38. Karmani, N. B., Soussou, H., and Alimi, A. M. (2014). Building a standardized wordnet in the iso lmf for aeb language. In Proceedings of the Seventh Global Wordnet Conference, pages 71–77. find
  39. Khalifa, S., Habash, N., Abdulrahim, D., and Hassan, S. (2016). A large scale corpus of Gulf Arabic. Proceedings of the 10th Language Resources and Evaluation Conference (LREC). find
  40. Maamouri, M., Bies, A., Buckwalter, T., and Mekki, W. (2004a). The penn arabic treebank: Building a large-scale annotated arabic corpus. In NEMLAR conference on Arabic language resources and tools, volume 27, pages 466–467. Cairo. find
  41. Maamouri, M., Bies, A., Buckwalter, T., and Mekki, W. (2004b). The Penn Arabic Treebank: Building a large-scale annotated Arabic corpus. In NEMLAR conference on Arabic language resources and tools, volume 27, pages 466–467. Cairo. find
  42. Masmoudi, A., Khmekhem, M. E., Esteve, Y., Belguith, L. H., and Habash, N. (2014). A Corpus and Phonetic Dictionary for Tunisian Arabic Speech Recognition. In Proceedings of the 9th Language Resources and Evaluation Conference (LREC), pages 306–310. find
  43. Masmoudi, A., Habash, N., Ellouze, M., Estève, Y., and Belguith, L. H. (2015). Arabic transliteration of romanized tunisian dialect text: A preliminary investigation. In International Conference on Intelligent Text Processing and Computational Linguistics, pages 608–619. Springer. find
  44. McNeil, K. (2018). Tunisian arabic corpus: Creating a written corpus of an ‘unwritten’language. Arabic Corpus Linguistics, 30. find
  45. Mdhaffar, S., Bougares, F., Esteve, Y., and Hadrich-Belguith, L. (2017). Sentiment analysis of tunisian dialects: Linguistic ressources and experiments. In Third Arabic Natural Language Processing Workshop (WANLP), pages 55–61. find
  46. Meftouh, K., Harrat, S., Jamoussi, S., Abbas, M., and Smaili, K. (2015). Machine translation experiments on padic: A parallel Arabic dialect corpus. In The 29th Pacific Asia conference on language, information and computation. find
  47. Meftouh, K., Harrat, S., and Smaïli, K. (2018). Padic: extension and new experiments. In 7th International Conference on Advanced Technologies ICAT. find
  48. Mikolov, T., Chen, K., Corrado, G., and Dean, J. (2013). Efficient estimation of word representations in vector space. arXiv preprint arXiv:1301.3781. ↗
  49. Mion, G. (2017). À propos du futur à tunis. Tunisian and Lybian Arabic Dialects, Common Trends, Recent Developments, Diachronic Aspects, pages 205– 217. find
  50. Moerth, K., Procházka, S., and Dallaji, I. (2014). Laying the foundations for a diachronic dictionary of tunis arabic. a first glance at an evolving new language resource. In Proceedings of the XVI EURALEX International Congress: The User in Focus, pages 377–387. find
  51. Moerth, K., Schopper, D., and Siam, O. (2017). Linking instead of lemmatising: Enriching the tunico corpus with the dictionary of tunis arabic. Tunisian and Lybian Arabic Dialects, Common Trends, Recent Developments, Diachronic Aspects, pages 218– 238. find
  52. Moussa, N. K. B., Soussou, H., and Alimi, A. M. (2015). Tunisian Arabic aeb wordnet: Current state and future extensions. In 2015 First International Conference on Arabic Computational Linguistics (ACLing), pages 3–8. IEEE. find
  53. Pennington, J., Socher, R., and Manning, C. D. (2014). Glove: Global vectors for word representation. In Proceedings of the 2014 conference on empirical methods in natural language processing (EMNLP), pages 1532–1543. find
  54. Rinke, E. M., Dobbrick, T., Löb, C., Zirn, C., and Wessler, H. (2022). Expert-informed topic models for document set discovery. Communication Methods and Measures, 16(1):39–58. ↗
  55. Ritt-Benmimoun, V. (2014). The Tunsian hilāl and su-laym dialects. a preliminary comparative study. In Alf lahǧa wa lahǧa, Proceedings of the 9th Aida Conference, pages 351–360. Lit Verlag, Berlin. find
  56. Saadane, H. and Habash, N. (2015). A conventional orthography for algerian Arabic. In Proceedings of the Second Workshop on Arabic Natural Language Processing, pages 69–79. find
  57. Saadane, H., Seffih, H., Fluhr, C., Choukri, K., and Semmar, N. (2018). Automatic identification of maghreb dialects using a dictionary-based approach. In Eleventh International Conference on Language Resources and Evaluation (LREC 2018). find
  58. Sadat, F., Kazemi, F., and Farzindar, A. (2014). Automatic identification of arabic dialects in social media. In Proceedings of the Second Workshop on Natural Language Processing for Social Media, pages 22–27. Dublin, Ireland. find
  59. Salama, A., Bouamor, H., Mohit, B., and Oflazer, K. (2014). YouDACC: the Youtube dialectal Arabic comment corpus. In Proceedings of the Ninth International Conference on Language Resources and Evaluation (LREC’14), pages 1246–1251, Reykjavik, Iceland, May. European Language Resources Association (ELRA). find
  60. Schäfer, R. and Bildhauer, F. (2013). Web corpus construction. Synthesis Lectures on Human Language Technologies, 6(4):1–145. ↗
  61. Seddah, D., Essaidi, F., Fethi, A., Futeral, M., Muller, B., Suárez, P. J. O., Sagot, B., and Srivastava, A. (2020). Building a user-generated content north-african arabizi treebank: Tackling hell. In ACL 2020-58th Annual Meeting of the Association for Computational Linguistics. find
  62. Sennrich, R., Haddow, B., and Birch, A. (2016). Improving neural machine translation models with monolingual data. In Proceedings of the 54th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 86–96, Berlin, Germany, August. Association for Computational Linguistics. find
  63. Singer, H. R. (1984). Grammatik der arabischen Mundart der Medina von Tunis. Walter de Gruyter. ↗
  64. Soliman, A. B., Eissa, K., and El-Beltagy, S. R. (2017). Aravec: A set of arabic word embedding models for use in arabic nlp. Procedia Computer Science, 117:256–265. find
  65. Stumme, H. (1896). Grammatik des tunisischen arabisch: nebst Glossar. JC Hinrichs. find
  66. Sutskever, I., Vinyals, O., and Le, Q. V. (2014). Sequence to sequence learning with neural networks. arXiv preprint arXiv:1409.3215. ↗
  67. Takezawa, T., Kikui, G., Mizushima, M., and Sumita, E. (2007). Multilingual spoken language corpus development for communication research. In International Journal of Computational Linguistics & Chinese Language Processing, Volume 12, Number 3, September 2007: Special Issue on Invited Papers from ISCSLP 2006, pages 303–324. find
  68. Turki, H., Adel, E., Daouda, T., and Regragui, N. (2016). A conventional orthography for maghrebi arabic. In Proceedings of the International Conference on Language Resources and Evaluation (LREC), Portoroz, Slovenia. find
  69. Vaswani, A., Shazeer, N., Parmar, N., Uszkoreit, J., Jones, L., Gomez, A. N., Kaiser, L., and Polosukhin, I. (2017). Attention is all you need. In Advances in neural information processing systems. find
  70. Weiss, G., Goldberg, Y., and Yahav, E. (2018). On the practical computational power of finite precision RNNs for language recognition. In Proceedings of the 56th Annual Meeting of the Association for Computational Linguistics (Volume 2: Short Papers), pages 740–745, Melbourne, Australia, July. Association for Computational Linguistics. find our card
  71. Younes, J. and Souissi, E. (2014). A quantitative view of Tunsian dialect electronic writing. In 5th International Conference on Arabic Language Processing, pages 63–72. find
  72. Younes, J., Achour, H., and Souissi, E. (2015). Constructing linguistic resources for the tunisian dialect using textual user-generated contents on the social web. In International Conference on Web Engineering, pages 3–14. Springer. find
  73. Younes, J., Souissi, E., and Achour, H. (2016). A hidden markov model for the automatic transliteration of romanized Tunsian dialect. In Proceedings of the 2nd international conference on arabic computational linguistics. find
  74. Younes, J., Souissi, E., Achour, H., and Ferchichi, A. (2018). A sequence-to-sequence based approach for the double transliteration of Tunsian dialect. Procedia computer science, 142:238–245. find
  75. Younes, J., Achour, H., Souissi, E., and Ferchichi, A. (2020). Romanized tunisian dialect transliteration using sequence labelling techniques. Journal of King Saud University-Computer and Information Sciences. ↗
  76. Zalmout, N. and Habash, N. (2017). Don’t throw those morphological analyzers away just yet: Neural morphological disambiguation for Arabic. In Proceedings of the 2017 Conference on Empirical Methods in Natural Language Processing, pages 704–713. find our card
  77. Zalmout, N. and Habash, N. (2019). Joint diacritization, lemmatization, normalization, and fine-grained morphological tagging. arXiv preprint arXiv:1910.02267. ↗ our card
  78. Zbib, R., Malchiodi, E., Devlin, J., Stallard, D., Matsoukas, S., Schwartz, R., Makhoul, J., Zaidan, O., and Callison-Burch, C. (2012). Machine translation of Arabic dialects. In Proceedings of the 2012 conference of the north american chapter of the association for computational linguistics: Human language technologies, pages 49–59. find our card
  79. Zribi, I., Graja, M., Khmekhem, M. E., Jaoua, M., and Belguith, L. H. (2013a). Orthographic transcription for spoken Tunisian Arabic. In International Conference on Intelligent Text Processing and Computational Linguistics, pages 153–163. Springer. find
  80. Zribi, I., Khemekhem, M. E., and Belguith, L. H. (2013b). Morphological analysis of tunisian dialect. In Proceedings of the Sixth International Joint Conference on Natural Language Processing, pages 992–996. ↗
  81. Zribi, I., Boujelbane, R., Masmoudi, A., Ellouze, M., Belguith, L. H., and Habash, N. (2014). A Conventional Orthography for Tunisian Arabic. In Proceedings of the 9th Language Resources and Evaluation Conference (LREC), pages 2355–2361. find
  82. Zribi, I., Ellouze, M., Belguith, L. H., and Blache, P. (2015). Spoken Tunsian Arabic corpus “STAC”: transcription and annotation. Research in computing science, 90:123–135. find

↑ back to top

This page in Markdown → https://md.research.elamur.ai/free/en/papers/gugliotta-2022-tarc-tunisian-arabish-corpus-first/