Elamur Research⌕
☰
home / papers / 2019 / Morphology-aware Word-Segmentation in Dialectal Arabic Adaptation of Neural Machine Translation

Morphology-aware Word-Segmentation in Dialectal Arabic Adaptation of Neural Machine Translation

Paper · 2019 · Ahmed Tawfik, Mahitab Emam, Khaled Essam et al.

Summary

Parallel data for translating dialectal Arabic into English is scarce. Systems therefore borrow from the much larger Modern Standard Arabic (MSA) corpora. MSA and the dialects often share word stems but attach different clitics, so the way words are split into pieces affects how much knowledge transfers. The authors ask whether linguistically informed segmentation still helps neural MT now that language-agnostic subword methods (BPE, SR) are standard.

They retrained the Unified Dialectal Arabic Segmenter (UDAS), a bi-LSTM-CRF model, on extra LDC MSA and dialectal treebanks. This raised its accuracy for all four dialects. Using Marian, they trained base models on MSA plus dialect data, fine-tuned them on dialect-only data, and compared five segmentation setups for Egyptian, Levantine and Gulf Arabic. The best results in every dialect came from running the dialectal segmenter first and then applying BPE or SR. The gains over the best non-morphological setup ranged from about 0.3 to 1.3 BLEU.

The work suggests that morphology and learned subwords complement each other rather than compete, especially for morphologically rich, low-resource varieties.

Key points

  • The retrained UDAS used 231,846 segmented sentences, drawn from LDC Arabic Treebank and dialectal treebank releases. It reached these accuracies (baseline in parentheses):
    • EGY: 99.4 (95.3)
    • GLF: 98.9 (93.1)
    • LEV: 96.2 (93.9)
    • MGR: 96.1 (91.4)
  • Training data sizes:
    • MSA: 2.5M sentences
    • Gulf: 38K sentences, oversampled 10×
    • Levantine: 219K sentences
    • Egyptian: 502K sentences
    • Egyptian was tested on Callhome with 4 references.
  • All systems used a 40K subword vocabulary and Marian v1.7.6. Adaptation fine-tuned the jointly trained base model on dialect data at a lower learning rate. The final system ensembled two adapted models.
  • Adding dialectal segmentation improved BLEU by 0.28 in the Gulf setting, by 1.27 in the Levantine setting, and by 0.44 in the Egyptian setting. For Levantine, the best adapted score was 23.08 (DS+SR), compared with 21.81 for SR alone. Without the segmenter, SR beat BPE consistently.
  • On 150 sentences, BPE+DS and SR+DS segmentations differed by only 2.55 edits per sentence on average. Pure BPE/SR outputs differed from the morphological segmentation by roughly 10–11.5 edits, suggesting that the agnostic methods miss dialectal morphology.

Interesting ideas

  • I1. Applying a morphological splitter first and a statistical subword model on top keeps clitic boundaries intact while still bounding vocabulary size.
  • I2. The authors patched the trained segmenter with a lookup table of training words it still got wrong, giving a cheap, transparent correction layer over a learned model.
  • I3. A single segmenter trained on pooled dialects beat dialect-specific models and removed the need to identify the dialect first.
  • I4. Comparing segmentations by Levenshtein distance shows how far an agnostic method drifts from linguistic structure, without running the full downstream system.
  • I5. The base model was trained on the rich and scarce data together, then fine-tuned on the scarce variety only, and two adapted models were ensembled. This lets dialect vocabulary enter early while still specializing at the end.

For Mova

How these ideas could serve Mova, our small open language model — its files, stages and tests:

  • I1 (tokens): crate en could split contractions, possessive 's and affixes by rule or morphology before any subword vocabulary is fed to gparse, so subword units never straddle meaningful boundaries.
  • I2 (learn): self-induced repair rules could include an explicit exception memory for tokens or constructions where the model reliably fails on its own training data, with each override inspectable.
  • I3 (tree): gparse could be trained on pooled treebanks from several English varieties and genres, rather than routing input to per-variety parsers, so shared structure is learned once.
  • I4 (learn): an edit-distance check between witness tokenizers or parsers could serve as an early disagreement signal when deciding which silver data to trust.
  • I5 (learn): parsers could first train on generic plus domain silver data jointly, then fine-tune on domain material such as fables or school texts at a lower learning rate.

Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.

 Fields, topics and concepts

📚 Shelves 4The field of science this work belongs to (OpenAlex fields).Computer Science · 16,667 worksArtificial Intelligence · 16,194 worksArts and Humanities · 16,039 worksLanguage and Linguistics · 15,891 works

🧭 Topics 4Research subjects and languages the work is about.Machine translation · 1,087 worksSemitic languages · 2,908 worksMorphology · 405 worksTokenization & segmentation · 119 works

Cite

Cite as Ahmed Tawfik, Mahitab Emam, Khaled Essam, Robert Nabil and Hany Hassan. 2019. Morphology-aware Word-Segmentation in Dialectal Arabic Adaptation of Neural Machine Translation. Proceedings of the Fourth Arabic Natural Language Processing Workshop.

The paper itself, not this page. BibTeX from the library's record of the work. DOI 10.18653/v1/w19-4602

@inproceedings{tawfik2019morphology,
  title = {{Morphology-aware Word-Segmentation in Dialectal Arabic Adaptation of Neural Machine Translation}},
  author = {Ahmed Tawfik, Mahitab Emam, Khaled Essam, Robert Nabil and Hany Hassan},
  booktitle = {Proceedings of the Fourth Arabic Natural Language Processing Workshop},
  year = {2019},
  doi = {10.18653/v1/w19-4602},
  url = {https://doi.org/10.18653/v1/w19-4602}
}

References

22 references, 11 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

  1. Marcin Junczys-Dowmunt, Roman Grundkiewicz, Tomasz Dwojak, Hieu Hoang, Kenneth Heafield, Tom Neckermann, Frank Seide, Ulrich Germann, Alham Fikri Aji, Nikolay Bogoychev, André F. T. Martins, and Alexandra Birch. 2018. Marian: Fast neural machine translation in C++. In Proceedings of ACL 2018, System Demonstrations, pages 116– 121, Melbourne, Australia. Association for Computational Linguistics.
  2. Ghaleb Al-Gaphari and Mohammed Al-Yadoumi. 2012. A method to convert Sana’ani accent to modern standard arabic. International Journal of Information Science and Management (IJISM), 8(1):39– 49. find
  3. Dzmitry Bahdanau, Kyunghyun Cho, and Yoshua Bengio. 2015. Neural machine translation by jointly learning to align and translate. Proc. International Conference on Learning Representations. ↗
  4. Taku Kudo. 2018. Subword regularization: Improving neural network translation models with multiple subword candidates. Proc. of ACL.
  5. Laith H. Baniata, Seyoung Park, and Seong-Bae Park. 2018. A multitask-based neural machine translation model with part-of-speech tags integration for arabic dialects. Applied Sciences, 8(12):2502. ↗
  6. Gaurav Kumar, Yuan Cao, Ryan Cotterell, Chris Callison-Burch, Daniel Povey, and Sanjeev Khudanpur. 2014. Translations of the callhome Egyptian Arabic corpus for conversational speech translation. In Proceedings of International Workshop on Spoken Language Translation (IWSLT). Citeseer.
  7. Markus Freitag and Yaser Al-Onaizan. 2016. Fast domain adaptation for neural machine translation. arXiv preprint arXiv:1612.06897. ↗
  8. Young-Suk Lee, Kishore Papineni, Salim Roukos, Ossama Emam, and Hany Hassan. 2003. Language model based Arabic word segmentation. In Proceedings of the 41st Annual Meeting on Association for Computational Linguistics-Volume 1, pages 399–406. Association for Computational Linguistics.
  9. Philip Gage. 1994. A new algorithm for data compression. The C Users Journal, 12(2):23–38. find
  10. Jiatao Gu, Hany Hassan, Jacob Devlin, and Victor O.K. Li. 2018. Universal neural machine translation for extremely low resource languages. Conference of the North American Chapter of the Association for Computational Linguistics on Human Language Technology. ↗
  11. Robert Parker, David Graff, Ke Chen, Junbo Kong, and Kazuaki Maeda. 2009. Arabic gigaword. find
  12. Peyman Passban, Qun Liu, and Andy Way. 2017. Translating low-resource languages by vocabulary adaptation from close counterparts. ACM Transactions on Asian and Low-Resource Language Information Processing (TALLIP), 16(4):29. ↗
  13. Nizar Habash. 2007. Syntactic preprocessing for statistical machine translation. Proceedings of the 11th MT Summit, 10. find
  14. Hassan Sajjad, Kareem Darwish, and Yonatan Belinkov. 2013. Translating dialectal arabic to english. In Proceedings of the 51st Annual Meeting of the Association for Computational Linguistics (Volume 2: Short Papers), volume 2, pages 1–6. find
  15. Hany Hassan, Mostafa Elaraby, and Ahmed Y. Tawfik. 2017. Synthetic spoken data for neural machine translation. In IWSLT, pages 82–89. find
  16. Matthias Huck, Simon Riess, and Alexander Fraser. 2017. Target-side word segmentation strategies for neural machine translation. In Proceedings of the Second Conference on Machine Translation, pages 56–67. find
  17. Younes Samih, Mohamed Eldesouki, Mohammed Attia, Kareem Darwish, Ahmed Abdelali, Hamdy Mubarak, and Laura Kallmeyer. 2017. Learning from relatives: unified dialectal Arabic segmentation. In Proceedings of the 21st Conference on Computational Natural Language Learning (CoNLL 2017), pages 432–441.
  18. Armand Joulin, Edouard Grave, Piotr Bojanowski, and Tomas Mikolov. 2017. Bag of tricks for efficient text classification. In Proceedings of the 15th Conference of the European Chapter of the Association for Computational Linguistics: Volume 2, Short Papers, pages 427–431. Association for Computational Linguistics. find
  19. Hassan Sawaf. 2010. Arabic dialect handling in hybrid machine translation. In Proceedings of the conference of the association for machine translation in the americas (amta), denver, colorado.
  20. Rico Sennrich, Barry Haddow, and Alexandra Birch. 2016. Neural machine translation of rare words with subword units. Proceedings of the 54th Annual Meeting of the Association for Computational Linguistics.
  21. Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N. Gomez, Łukasz Kaiser, and Illia Polosukhin. 2017. Attention is all you need. In Advances in neural information processing systems, pages 5998–6008.
  22. Rabih Zbib, Erika Malchiodi, Jacob Devlin, David Stallard, Spyros Matsoukas, Richard Schwartz, John Makhoul, Omar F. Zaidan, and Chris Callison-Burch. 2012. Machine translation of Arabic dialects. In Proceedings of the 2012 conference of the north american chapter of the association for computational linguistics: Human language technologies, pages 49–59. Association for Computational Linguistics.

Updated

↑ back to top

This page in Markdown → https://md.research.elamur.ai/free/en/papers/tawfik-2019-morphology-aware-word-segmentation-dialectal/