Elamur Research⌕
☰
home / papers / 2018 / Diacritization of Maghrebi Arabic Sub-Dialects

Diacritization of Maghrebi Arabic Sub-Dialects

Paper · 2018 · Ahmed Abdelali, Mohammed Attia, Younes Samih et al.

Summary

Arabic is usually written without short vowels, and systems such as text-to-speech must restore them. Most diacritization research has targeted Modern Standard Arabic (MSA), so the authors address two Maghrebi sub-dialects, Moroccan (MOR) and Tunisian (TUN). Their data are two dialectal New Testament translations of about 8,200 verses each. Dialects mostly put sukun on word endings, so the task reduces to restoring word-internal diacritics.

A corpus analysis shows that most dialect words have only one diacritized form. Simply choosing each word's most frequent form is therefore very accurate, much more so than for MSA. The model is a character-level network: an embedding layer, two stacked bi-LSTMs and a CRF output layer that predicts one of 12 diacritic tags per character. It slightly outperforms an earlier CRF++ baseline, reaching 2.7% WER on MOR and 3.6% on TUN. Training on one dialect and testing on the other fails badly. Joint training on both dialects costs only about one point of WER, which suggests the network recognises the sub-dialect on its own. This matters for dialect text whose regional origin is unknown.

Key points

  • Data: Moroccan and Tunisian Bible translations with 134,324 and 131,923 words, evaluated with 5-fold cross-validation.
  • Ambiguity is low:
    • 89% (MOR) and 82% (TUN) of training words appear in a single diacritized form.
    • Most-frequent-form tagging scores 99.1% and 98.9% on the training data, and about 92% on the test splits.
    • Roughly 93% of test words were already seen in training.
  • Model results: the bi-LSTM-CRF reaches 2.7% / 3.6% WER for MOR / TUN, against 2.9% / 3.8% for the CRF baseline with character n-grams and Brown Clusters.
  • Cross-dialect training:
    • Word error is about 48–49% when training on one dialect and testing on the other.
    • The vocabularies of the two dialects overlap by about 61%, and shared words often take different diacritics.
  • Joint training:
    • The DNN reaches 3.7% (MOR) and 4.9% (TUN) WER.
    • The CRF collapses to 34.2% and 23.8%.
    • Most remaining errors involve shadda and sukun.

Interesting ideas

  • I1. Before designing a contextual model, compute how far a trivial "most frequent form per word" lookup gets you; here it showed the real problem is unseen words rather than ambiguity.
  • I2. Use a hybrid design: dictionary lookup for frequent, unambiguous words and a learned character model only for the rest, with the model itself helping to build the dictionary.
  • I3. Training jointly on related varieties without a variety label can still work if the model is expressive enough to infer the variety from the input.
  • I4. Cross-training (train on A, test on B) works as a cheap probe of how divergent two data sources really are, before deciding to pool them.
  • I5. Break errors down by output label and compare single-source with joint training, because pooling data can help some labels and hurt others.

For Mova

How these ideas could serve Mova, our small open language model — its files, stages and tests:

  • I1 (learn): Run a per-wordform majority baseline for each tagger or lemmatizer task, so we know whether context modelling is actually where the errors come from.
  • I2 (tokens): The en crate could serve frequent unambiguous lemmas and tags from a compiled table, falling back to a character-level model for out-of-vocabulary words, so each decision stays traceable to its source.
  • I3 (tree): Train gparse on a mix of UD English treebanks without genre flags, then check whether a jointly trained parser loses only a little against per-treebank models.
  • I4 (learn): Use train-on-one, test-on-another runs between silver sources as a control that flags incompatible annotation conventions before merging them into training data.
  • I5 (learn): Report per-deprel or per-tag error shifts whenever training data is combined, rather than only an overall LAS change.

Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.

 Fields, topics and concepts

📚 Shelves 4The field of science this work belongs to (OpenAlex fields).Computer Science · 16,667 worksArtificial Intelligence · 16,194 worksArts and Humanities · 16,039 worksLanguage and Linguistics · 15,891 works

🧭 Topics 1Research subjects and languages the work is about.Semitic languages · 2,908 works

Cite

Cite as Ahmed Abdelali, Mohammed Attia, Younes Samih, Kareem Darwish and Hamdy Mubarak. 2018. Diacritization of Maghrebi Arabic Sub-Dialects. arXiv:1810.06619.

The paper itself, not this page. BibTeX from the library's record of the work. DOI 10.48550/arxiv.1810.06619 · arXiv 1810.06619

@misc{abdelali2018diacritization,
  title = {{Diacritization of Maghrebi Arabic Sub-Dialects}},
  author = {Ahmed Abdelali, Mohammed Attia, Younes Samih, Kareem Darwish and Hamdy Mubarak},
  year = {2018},
  doi = {10.48550/arxiv.1810.06619},
  eprint = {1810.06619},
  archivePrefix = {arXiv},
  url = {https://doi.org/10.48550/arxiv.1810.06619}
}

References

23 references, 11 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

  1. Gheith A Abandah, Alex Graves, Balkees Al-Shagoor, Alaa Arabiyat, Fuad Jamour, and Majid Al-Taee. 2015. Automatic diacritization of arabic text using recurrent neural networks. International Journal on Document Analysis and Recognition (IJDAR), 18(2):183–197. ↗
  2. Aqil M Azmi and Reham S Almajed. 2015. A survey of automatic arabic diacritization techniques. Natural Language Engineering, 21(03):477–495. find
  3. Yonatan Belinkov and James Glass. 2015. Arabic diacritization with recurrent neural networks. In Proceedings of the 2015 Conference on Empirical Methods in Natural Language Processing, pages 2281– 2285, Lisbon, Portugal.
  4. Peter F Brown, Peter V Desouza, Robert L Mercer, Vincent J Della Pietra, and Jenifer C Lai. 1992. Class-based n-gram models of natural language. Computational linguistics, 18(4):467–479. find
  5. Kareem Darwish, Hamdy Mubarak, and Ahmed Abdelali. 2017. Arabic diacritization: Stats, rules, and hacks. In Proceedings of the Third Arabic Natural Language Processing Workshop, pages 9–17.
  6. Yann Dauphin, Harm de Vries, and Yoshua Bengio. 2015. Equilibrated adaptive learning rates for non-convex optimization. In Advances in neural information processing systems, pages 1504–1512. ↗
  7. Ya’akov Gal. 2002. An HMM approach to vowel restoration in arabic and hebrew. In Proceedings of the ACL-02 workshop on Computational approaches to Semitic languages, pages 1–7. Association for Computational Linguistics.
  8. Alex Graves. 2012. Supervised sequence labelling with recurrent neural networks, volume 385. Springer. find
  9. Nizar Habash, Ramy Eskander, and Abdelati Hawwari. 2012. A morphological analyzer for egyptian arabic. In Proceedings of the twelfth meeting of the special interest group on computational morphology and phonology, pages 1–9. Association for Computational Linguistics.
  10. Nizar Habash and Owen Rambow. 2007. Arabic diacritization through full morphological tagging. In Human Language Technologies 2007: The Conference of the North American Chapter of the Association for Computational Linguistics; Companion Volume, Short Papers, pages 53–56. Association for Computational Linguistics. ↗
  11. Mustafa Jarrar, Nizar Habash, Faeq Alrimawi, Diyam Akra, and Nasser Zalmout. 2017. Curras: an annotated corpus for the palestinian arabic dialect. Language Resources and Evaluation, 51(3):745–775. ↗
  12. Darwish Kareem, Abdelali Ahmed, Mubarak Hamdy, Samih Younes, and Attia Mohammed. 2018. Diacritization of arabic dialects. In Proceedings of the Eleventh International Conference on Language Resources and Evaluation (LREC 2018). find
  13. Salam Khalifa, Sara Hassan, and Nizar Habash. 2017. A morphological analyzer for gulf arabic verbs. WANLP 2017 (co-located with EACL 2017), page 35.
  14. Diederik P. Kingma and Jimmy Ba. 2014. Adam: A method for stochastic optimization. CoRR, abs/1412.6980. ↗
  15. John Lafferty, Andrew McCallum, and Fernando CN Pereira. 2001. Conditional random fields: Probabilistic models for segmenting and labeling sequence data. Proceedings of the 18th International Conference on Machine Learning 2001 (ICML 2001), pages 282–289. find
  16. Guillaume Lample, Miguel Ballesteros, Sandeep Subramanian, Kazuya Kawakami, and Chris Dyer. 2016. Neural architectures for named entity recognition. arXiv preprint arXiv:1603.01360. ↗
  17. Wang Ling, Yulia Tsvetkov, Silvio Amir, Ramon Fermandez, Chris Dyer, Alan W Black, Isabel Trancoso, and Chu-Cheng Lin. 2015. Not all contexts are created equal: Better word representations with variable attention. In Proceedings of the 2015 Conference on Empirical Methods in Natural Language Processing, pages 1367–1372. find
  18. Rani Nelken and Stuart M Shieber. 2005. Arabic diacritization using weighted finite-state transducers. In Proceedings of the ACL Workshop on Computational Approaches to Semitic Languages, pages 79– 86. Association for Computational Linguistics.
  19. Mohsen Rashwan, Ahmad Al Sallab, M. Raafat, and Ahmed Rafea. 2015. Deep learning framework with confused sub-set resolution architecture for automatic arabic diacritization. In IEEE Transactions on Audio, Speech, and Language Processing, pages 505–516. ↗
  20. Mike Schuster and Kuldip K Paliwal. 1997. Bidirectional recurrent neural networks. IEEE Transactions on Signal Processing, 45(11):2673–2681. ↗
  21. Nitish Srivastava, Geoffrey E Hinton, Alex Krizhevsky, Ilya Sutskever, and Ruslan Salakhutdinov. 2014. Dropout: a simple way to prevent neural networks from overfitting. Journal of machine learning research, 15(1):1929–1958. find
  22. Matthew D Zeiler. 2012. Adadelta: an adaptive learning rate method. arXiv preprint arXiv:1212.5701. ↗
  23. Imed Zitouni, Jeffrey S Sorensen, and Ruhi Sarikaya. 2006. Maximum entropy based restoration of arabic diacritics. In Proceedings of the 21st International Conference on Computational Linguistics and the 44th annual meeting of the Association for Computational Linguistics, pages 577–584. Association for Computational Linguistics.

Updated

↑ back to top

This page in Markdown → https://md.research.elamur.ai/free/en/papers/abdelali-2018-diacritization-maghrebi-arabic-sub-dialects/