Elamur Research⌕
☰
home / papers / 2020 / TArC: Incrementally and Semi-Automatically Collecting a Tunisian Arabish Corpus

TArC: Incrementally and Semi-Automatically Collecting a Tunisian Arabish Corpus

Summary

Tunisian Arabish (TA) is the informal practice of writing the Tunisian dialect in Latin letters, with digits standing in for some Arabic sounds. It became common on social media from about 2008, when Arabic keyboards were not widely available in Tunisia. TA has no standard spelling and almost no annotated resources. The authors build TArC, which they present as the first morpho-syntactically annotated corpus of TA. It collects texts written from 2009 onward, so it can also show whether TA spelling is drifting toward a shared convention.

Each token gets several annotation layers:

  • a rendering in Arabic script that follows the CODA* and CODA TUN orthography guidelines,
  • a lemma,
  • a POS tag,
  • an Italian translation of the texts.

The corpus grows incrementally and semi-automatically: models pre-annotate new data and humans correct the output. Because CODA* covers many dialects, mapping TA onto it lets existing Arabic NLP tools be reused and makes cross-dialect comparison possible. The paper also describes key Tunisian dialect features, such as circumfix negation and phonological changes, and how TA writers encode them.

Key points

  • TArC is presented as the first TA corpus that is both transcribed into Arabic characters and morpho-syntactically annotated (lemmas and POS).
  • Token-level Arabic-script transcription follows CODA* (Habash et al., 2018), chosen because it covers 28 Arab city dialects, together with CODA TUN (Zribi et al., 2014).
  • Other freely available Tunisian corpora are small or limited in scope:
    • PADIC: 6,400 sentences across six dialects.
    • TuDiCoI: 21,682 words of railway-station dialogues.
    • STAC: 42,388 words.
    • TARIC: 20 hours of speech, of which only 7,814 words are annotated.
    • TSAC: 17k Facebook comments labeled for polarity, the only one containing TA.
  • Some preliminary automatic experiments scored below 50% token-level accuracy on average.
  • Annotation-time estimates were derived from the time spent correcting tokens that were already transcribed, which is faster than transcribing from scratch.

Interesting ideas

  • I1. Map a noisy, non-standard writing system onto a conventional orthography (CODA*) as a pivot layer, so tools built for the standard form can be reused downstream. — For Mova (tokens): Social-media English could be normalized token-by-token into a canonical spelling layer that keeps the original form alongside it, so the tagger and lemmatizer see standard input and every change stays traceable.
  • I2. Grow the corpus incrementally: a model pre-annotates each new batch, humans correct it, and the corrected data improves the next round of pre-annotation. — For Mova (learn): Silver data agreed on by witness parsers could feed a similar loop, where only disputed tokens go to a human or LLM teacher and the corrected output retrains the models.
  • I3. Budget annotation effort by measuring correction time on pre-filled output rather than from-scratch annotation time, which gives a realistic cost for semi-automatic pipelines. — For Mova (learn): Logging time per corrected token or arc when repairing silver UD trees would show which pipeline stage is worth improving first.
  • I4. Collect texts across a decade on purpose, so the corpus can reveal whether a spontaneous spelling system is converging toward a convention. — For Mova (world): Timestamped text sources would let the knowledge layer record when a word form or sense became current, instead of treating vocabulary as static.
  • I5. Treat translation into the annotators' native language as a cheap extra annotation layer that also helps them check their own understanding of each text. — For Mova (back): A parallel human-written rendering stored per sentence could serve as a reference when evaluating text → graph → text output or translation through the meta-world.

Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.

Fields: Linguistics & languages

Topics: Treebanks

Cited by

6 papers of our library cite it (of the 140884 whose bibliographies we read); 1 of them have a page here, 5 not yet.

Cite

The paper itself, not this page. BibTeX from the library's record of the work.

@inproceedings{gugliotta2020tarc,
  title = {{TArC: Incrementally and Semi-Automatically Collecting a Tunisian Arabish Corpus}},
  author = {Gugliotta, Elisa and Dinarelli, Marco},
  booktitle = {Proceedings of the Twelfth Language Resources and Evaluation Conference},
  year = {2020},
  url = {https://aclanthology.org/2020.lrec-1.770/}
}

References

31 references, 2 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

  1. Akbar, R. (2019). Arabizi among kuwaiti youths: Reshaping the standard arabic orthography. International Journal of English Linguistics, 9(1):301–323. find
  2. Al-Badrashiny, M., Eskander, R., Habash, N., and Rambow, O. (2014). Automatic transliteration of romanized dialectal arabic. In Proceedings of the eighteenth conference on computational natural language learning, pages 30–38. find
  3. Bies, A., Song, Z., Maamouri, M., Grimes, S., Lee, H., Wright, J., Strassel, S., Habash, N., Eskander, R., and Rambow, O. (2014). Transliteration of arabizi into arabic orthography: Developing a parallel annotated arabizi-arabic script sms/chat corpus. In Proceedings of the EMNLP 2014 Workshop on Arabic Natural Language Processing (ANLP), pages 93–103. find
  4. Boujelbane, R. J. (2015). Traitements linguistiques pour la reconnaissance automatique de la parole appliquée à la langue arabe: de l’arabe standard vers l’arabe dialectal. Ph.D. thesis, Aix-Marseille. find
  5. Buckwalter, T. and Parkinson, D. (2014). A frequency dictionary of Arabic: Core vocabulary for learners. Routledge. find
  6. Chalabi, A. and Gerges, H. (2012). Romanized arabic transliteration. In Proceedings of the Second Workshop on Advances in Text Input Methods, pages 89–96. find
  7. Darwish, K. (2013). Arabizi detection and conversion to arabic. CoRR, abs/1306.6755. ↗
  8. Dinarelli, M. and Grobol, L. (2019a). Hybrid neural models for sequence modelling: The best of three worlds. CoRR. find
  9. Dinarelli, M. and Grobol, L. (2019b). Seq2biseq: Bidirectional output-wise recurrent neural networks for sequence modelling. CoRR, abs/1904.04733. ↗
  10. Durand, O. (2009). Dialettologia araba. ’Sapienza’ University of Rome, ’Studi Orientali’ Faculty. find
  11. Graja, M., Jaoua, M., and Hadrich-Belguith, L. (2013). Discriminative framework for spoken tunisian dialect understanding. In International Conference on Statistical Language and Speech Processing, pages 102–110. Springer. find
  12. Habash, N., Diab, M., and Rambow, O. (2012). Conventional orthography for dialectal Arabic. In Proceedings of the Eighth International Conference on Language Resources and Evaluation (LREC’12), pages 711–718, Istanbul, Turkey, May. European Language Resources Association (ELRA). find our card
  13. Habash, N., Eryani, F., Khalifa, S., Rambow, O., Abdulrahim, D., Erdmann, A., Faraj, R., Zaghouani, W., Bouamor, H., Zalmout, N., Hassan, S., Al-Shargi, F., Alkhereyf, S., Abdulkareem, B., Eskander, R., Salameh, M., and Saddiki, H. (2018). Unified guidelines and resources for Arabic dialect orthography. In Proceedings of the Eleventh International Conference on Language Resources and Evaluation (LREC 2018), Miyazaki, Japan, May. European Language Resources Association (ELRA). find our card
  14. Haspelmath, M. and Tadmor, U. (2009). Loanwords in the world’s languages: a comparative handbook. Walter de Gruyter. find
  15. Hert, P. (1999). Quasi-oralité de l’écriture électronique et sentiment de communauté dans les débats scientifiques en ligne. Réseaux, 17(97). find
  16. Marcais, P. (1977). Esquisse grammaticale de l’arabe maghrébin. Langues d’Amérique et d’Orient. find
  17. Masmoudi, A., Khmekhem, M. E., Esteve, Y., Hadrich-Belguith, L., and Habash, N. (2014). A corpus and phonetic dictionary for tunisian arabic speech recognition. In LREC, pages 306–310. find
  18. Masmoudi, A., Habash, N., Ellouze, M., Estève, Y., and Hadrich-Belguith, L. (2015). Arabic transliteration of romanized tunisian dialect text: A preliminary investigation. In Computational Linguistics and Intelligent Text Processing - 16th International Conference, CICLing 2015, Proceedings, Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics), pages 608–619. Springer-Verlag, 1. find
  19. Medhaffar, S., Bougares, F., Estève, Y., and Hadrich-Belguith, L. (2017). Sentiment analysis of Tunisian Dialects: Linguistic ressources and experiments. In Proceedings of the third Arabic Natural Language Processing Workshop, pages 55–61. Association for Computational Linguistics. find
  20. Meftouh, K., Harrat, S., and Smaïli, K. (2018). Padic: extension and new experiments. In 7th International Conference on Advanced Technologies ICAT. find
  21. Mion, G. (2004). Osservazioni sul sistema verbale dell’arabo di tunisi. Rivista degli studi orientali, 78(Fasc. 1/2):243–255. find
  22. Schäfer, R. and Bildhauer, F. (2013). Web corpus construction. Synthesis Lectures on Human Language Technologies, 6(4):1–145. find
  23. Sutskever, I., Vinyals, O., and Le, Q. V. (2014). Sequence to sequence learning with neural networks. In Proceedings of NIPS, Cambridge, MA, USA. MIT Press. find
  24. Younes, J., Achour, H., Souissi, E., and Ferchichi, A. (2018a). Survey on corpora availability for the tunisian dialect automatic processing. In 2018 JCCO Joint International Conference on ICT in Education and Training, International Conference on Computing in Arabic, and International Conference on Geocomputing (JCCO: TICET-ICCA-GECO), pages 1– 7. IEEE. find
  25. Younes, J., Souissi, E., Achour, H., and Ferchichi, A. (2018b). A sequence-to-sequence based approach for the double transliteration of tunisian dialect. Procedia computer science, 142:238–245. find
  26. Zribi, I., Boujelbane, R., Masmoudi, A., Ellouze, M., Hadrich-Belguith, L., and Habash, N. (2014). A conventional orthography for tunisian arabic. In Proceedings of the Ninth International Conference on Language Resources and Evaluation, LREC 2014, Reykjavik, Iceland, May 26-31, 2014, pages 2355–2361. find
  27. Zribi, I., Ellouze, M., Hadrich-Belguith, L., and Blache, P. (2015). Spoken tunisian arabic corpus ”stac”: Transcription and annotation. Research in Computing Science, 90:123–135. 8 Language Resource References Graja, Marwa; Jaoua, Maher; Hadrich-Belguith, Lamia. (2010). Tunisian Dialect Corpus Interlocutor (TuDiCoI). MIRACL Laboratory, Sfax (Tunis), Arabic Natural Language Processing Research Group (ANLP), 1.0. find
  28. Masmoudi, Abir; Ellouze Khemakhem, Mariem; Estève, Yannick; Hadrich-Belguith, Lamia. (2014). Tunisian Arabic Railway Interaction Corpus. MIR-ACL Laboratory, Sfax (Tunis), Arabic Natural Language Processing Research Group (ANLP), 1.0. find
  29. Medhaffar, Salima and Bougares, Fethi and Estève, Yannick and Hadrich-Belguith, Lamia. (2017). Tunisian Hadrich-Belguith Analysis Corpus (TSAC). find
  30. Mourad, Abbas. (2017). Parallel Arabic Dialectal Corpus (PADIC). Scientific and Technical Research Center for the Development of Arabic Language, funded by the Algerian Ministry of Higher Education and Scientific Research, language resources, 2.0, ISLRN 429-053-323-228-4. find
  31. Zribi, Inés; Ellouze Khemakhem, Mariem; Hadrich-Belguith, Lamia; Blache, Philippe. (2015). Spoken Tunisian Arabic corpus (STAC). MIRACL Laboratory, Sfax (Tunis), Arabic Natural Language Processing Research Group (ANLP), 1.0. find

↑ back to top

This page in Markdown → https://md.research.elamur.ai/free/en/papers/gugliotta-2020-tarc-incrementally-semi-automatically-collecting/