Elamur Research⌕
☰
home / papers / 2026 / DATASHI: A Parallel English–Tashlhiyt Corpus for Orthography Normalization and Low-Resource Language Processing.

DATASHI: A Parallel English–Tashlhiyt Corpus for Orthography Normalization and Low-Resource Language Processing.

Summary

Tashlhiyt is an Amazigh language of southern Morocco with several million speakers, but it has very few computational resources. Its written form is highly inconsistent. Speakers mix Latin, Arabic and Neo-Tifinagh scripts, and they often use digits such as "3" or "7" for sounds that Latin letters lack. To address this, the authors built DATASHI, a parallel English–Tashlhiyt corpus of 5,000 sentence pairs. The English sentences were authored across 20 everyday topics. Native speakers with no orthographic training then wrote them down in Tashlhiyt using their own spelling habits. Language experts manually standardized 1,500 of these sentences, creating a gold reference for normalization. On this subset, the authors tested five large language models on turning informal spelling into standard form. Few-shot prompting clearly beat zero-shot, and Gemini-2.5-Pro achieved the lowest word- and character-level error rates. A breakdown of edit types by sound class shows where models struggle with marked Tashlhiyt features. The resource supports normalization, translation and tokenization research and could later support read-speech collection.

Key points

  • DATASHI contains 5,000 English–Tashlhiyt pairs, with 250 sentences in each of 20 thematic domains. The full dataset, code and prompts are released at github.com/Nasseredd/datashi.
  • The non-standard Tashlhiyt side was written by 34 native speakers (17 male, 17 female, aged 20–50). Seven Amazigh experts standardized a 1,500-sentence subset by hand, with no automatic pre-normalization.
  • In the full corpus, the English side has 43,654 tokens and 3,160 types, while non-standard Tashlhiyt has 35,714 tokens and 7,261 types. This vocabulary gap reflects how much unregulated spelling varies.
  • In the 1,500-sentence subset, standardization raised the token count from 10,698 to 12,211 and reduced the vocabulary from 4,583 to 3,683 types.
  • The models evaluated were GPT-5, Claude-Sonnet-4.5, Gemini-2.5-Pro, Mistral and Qwen3-Max, with Gemini-2.5-Pro performing best. Deletions, substitutions and insertions were analyzed separately for geminates, emphatics, uvulars and pharyngeals.

Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.

Fields: Linguistics & languages

Cite

The paper itself, not this page. BibTeX from the library's record of the work. DOI 10.63317/2zekkx242a7h

@inproceedings{monir2026datashi,
  title = {{DATASHI: A Parallel English–Tashlhiyt Corpus for Orthography Normalization and Low-Resource Language Processing.}},
  author = {Monir, Nasser-Eddine and Baou, Zakaria},
  booktitle = {Proceedings of the Fifteenth Language Resources and Evaluation Conference},
  year = {2026},
  doi = {10.63317/2zekkx242a7h},
  url = {https://doi.org/10.63317/2zekkx242a7h}
}

References

33 references, 0 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

  1. Gilles Adda, Laurent Besacier, Sebastian Stüker, and Daan van Esch. 2022. Towards open multimodal corpora for low-resource languages. In Proceedings of the 13th Language Resources and Evaluation Conference (LREC). find
  2. Mistral AI. 2024. Mistral-large-instruct-2411 (123b parameters) [model card]. https://huggingface.co/mistralai/Mistral-Large-Instruct-2411. Accessed: 2025-10-25. ↗
  3. Brahim Ait Laaguid and Azeddine Khaloufi. 2023. Amazigh language use on social media: An exploratory study. Jurnal Arbitrer, 10(1):24–34. Accessed: 2025-10-25. find
  4. Youssef Aït Ouguengay and Aïcha Bouhjar. 2010. For standardised Amazigh linguistic resources. In Proceedings of the Seventh International Conference on Language Resources and Evaluation (LREC’10), Valletta, Malta. European Language Resources Association (ELRA). find
  5. Oussama Akallouch, Mohammed Akallouch, and Khalid Fardousse. 2025. Advances in amazigh language technologies: A comprehensive survey across processing domains. Information, 16(7):600. find
  6. Ahmad Al-Sallab, Wael Salloum, and Nizar Habash. 2018. Arabic dialect normalization for improved nlp performance. In Proceedings of the LREC 2018 Workshop on Arabic NLP, pages 47–55, Miyazaki, Japan. find
  7. Samir Amri and Lahbib Zenkouar. 2017. Coupling an annotated corpus and a lexicon for amazigh pos tagging. Journal of Mobile Multimedia, 13(3&4):222–232. find
  8. Antonios Anastasopoulos, Peng Qi, and Graham Neubig. 2023. Large multilingual models for low-resource languages: Capabilities and limitations. In Proceedings of the 61st Annual Meeting of the Association for Computational Linguistics (ACL), pages 10245–10258, Toronto, Canada. find
  9. Anthropic. 2025. Introducing claude sonnet 4.5. https://www.anthropic.com/news/claude-sonnet-4-5. Accessed: 2025-10-25. ↗
  10. Fadoua Ataa Allah and Siham Boulaknadel. 2012. Natural language processing for amazigh language: Challenges and future directions. In Workshop on Language Technology for Normalisation of Less-Resourced Languages (SALT-MIL8/AfLaT2012), Rabat, Morocco. find
  11. Karim Bensoukas. 2009. The loss of negative verb morphology in tashlhit: A variation approach. Asinag, (2):89–110. Accessed: 2025-10-25. find
  12. Laurent Besacier, Etienne Barnard, Alexey Karpov, and Tanja Schultz. 2014. Automatic speech recognition for under-resourced languages: A survey. Speech Communication, 56:85–100. find
  13. Alan W. Black. 2019. Cmu wilderness multilingual speech dataset. In Proceedings of ICASSP 2019. find
  14. Aicha Bouhjar. 2008. Amazigh language terminology in morocco or management of a “multidimensional” variation. In Proceedings of the Sixth International Conference on Language Resources and Evaluation (LREC’08), Marrakech, Morocco. European Language Resources Association (ELRA). find
  15. Fatima Boukhris, Abdallah Boumalk, El Hous-saïn El Moujahid, and Hamid Souifi. 2008. La nouvelle grammaire de l’amazighe. Institut Royal de la Culture Amazighe, Rabat, Morocco. Accessed: 2025-10-25. find
  16. Siham Boulaknadel. 2018. Sociolinguistic patterns and orthographic practices in online amazigh communication. Revue des Études Amazighes, 12:45–59. find
  17. Salem Chaker. 2019. Syntaxe. Encyclopédie berbère, 43:7644–7655. Accessed: 2025-10-25. find
  18. Salem Chaker and Anissa Sellès. 2016. Writing and rewriting amazigh/berber identity: Orthographies and language ideologies. Writing Systems Research, 8(1):1–18. find
  19. Mark Gales, Kate Knill, Anton Ragni, and Santosh Rath. 2014. Speech recognition and keyword spotting for low-resource languages: Babel project overview. In Proceedings of SLTU 2014. find
  20. Rezaul Haque and Bonaventure Dossou. 2021. Low-resource text normalization for african languages: A case study on wolof and hausa. In Proceedings of the AfricaNLP Workshop at EACL, pages 120–129. find
  21. Abdellah Idhssaine. 2023. A critical review of the sociolinguistics of the amazigh language in morocco: Documentation, teaching, and officializa-tion. Langues et Littératures, 28:179–201. find
  22. Fodil Kessai. 2018. Élaboration d’un dictionnaire électronique de berbère. Ph.D. thesis, Institut National des Langues et Civilisations Orientales. Accessed: 2025-10-25. find
  23. Otman Maarouf. 2025. Amazigh linguistic dataset: Part-of-speech tagging, named entity recognition, and parallel corpus (tifinagh-english). find
  24. Aman Madaan, Abhilasha Suresh, and Graham Neubig. 2023. Prompting Large Language Models for text normalization in low-resource settings. In Findings of the Association for Computational Linguistics: EMNLP 2023, pages 13012–13025, Singapore. find
  25. Kamal Nait Zerad. 2011. Mémento grammatical et orthographique de berbère : Kabyle – Chleuh – Rifain, 1 edition. Éditions L’Harmattan, Paris. Accessed: 2025-10-25. find
  26. OpenAI. 2025. Introducing gpt-5. https://openai.com/index/introducing-gpt-5/. Accessed: 2025-10-25. ↗
  27. Said Oussou. 2021. Amazigh language morphology: Examples from tashlhiyt in ayt hdidou. Lingua. Language and Culture, 1:212–224. find
  28. Tomas Riad. 2022. The secret morphology of tashlhiyt berber. Brill’s Journal of Afroasiatic Languages and Linguistics, 14(2):273–307. find
  29. Rachid Ridouane. 2014. Tashlhiyt berber. Journal of the International Phonetic Association, 44(2):207–221. find
  30. Tanja Schultz. 2002. Globalphone: A multilingual speech and text database developed at karlsruhe university. In Proceedings of the International Conference on Spoken Language Processing (ICSLP). find
  31. Dris Soulaimani. 2016. Becoming amazigh: Standardisation, purity, and questions of identity. The Journal of North African Studies, 21(3):485–500. find
  32. Qwen Team. 2025. Qwen3-max: Just scale it. find
  33. Yi Zhang, Rob van der Goot, and Barbara Plank. 2023. Universal text normalization for low-resource languages. In Findings of the Association for Computational Linguistics: EMNLP 2023, pages 12456–12468, Singapore. find

↑ back to top

This page in Markdown → https://md.research.elamur.ai/free/papers/monir-2026-datashi-parallel-english-tashlhiyt-corpus/