Elamur Research⌕
☰
home / papers / 2013 / The Effects of Factorizing Root and Pattern Mapping in Bidirectional Tunisian - Standard Arabic Machine Translation

The Effects of Factorizing Root and Pattern Mapping in Bidirectional Tunisian - Standard Arabic Machine Translation

Paper · 2013 · Ahmed Hamdi, Rahma Boujelbane, Nizar Habash et al.

Summary

Arabic dialects such as Tunisian (TUN) have almost no NLP resources, while Modern Standard Arabic (MSA) has many. The authors propose translating dialectal verbs into MSA, and back, so that MSA tools can be reused. The translation operates on deep Semitic morphology: each verb is analysed into a root and a pattern, the two are transferred into the other variety, and a surface form is generated from the result. Analysis and generation use MAGEAD, a multi-tier morphological analyzer and generator. The authors adapted MAGEAD to Tunisian by editing only its linguistic knowledge and leaving the engine unchanged. They compare three ways to map roots and patterns: independent mapping through a root lexicon plus a pattern correspondence table; a joint root+pattern lexicon; and that joint lexicon with a fallback to independent mapping. The joint lexicon with backoff gives the best balance of recall and ambiguity, with about 80% recall for TUN→MSA and 84% for MSA→TUN. Output remains ambiguous, so the authors note that contextual disambiguation is still required.

Key points

  • MAGEAD was adapted to TUN verbs by adding one MBC, removing three, editing five, and adding a new abstract morpheme for indirect object pronouns, which attach as enclitics in TUN.
  • Independent root and pattern mapping reached 68.98% (TUN→MSA) and 72.37% (MSA→TUN) token recall, with ambiguity of 74.37 and 13.70. Using the two most frequent target patterns raised recall to 86.12% and 81.77%, but TUN→MSA ambiguity exceeded 100.
  • A joint root+pattern lexicon improved both measures: recall was 76.43% and 79.24%, and ambiguity fell to 26.82 and 1.47.
  • With backoff to the independent mapping, recall rose to 79.71% and 84.83%, while ambiguity rose to 29.16 and 3.47.
  • An oracle run on the dev set, whose forms had populated the lexicon, set an upper bound of 87.65% and 89.56% recall. In the TUN→MSA direction, 78.2% of the remaining errors came from TUN MAGEAD and 21.8% from MSA MAGEAD, mostly from unimplemented phenomena such as quadriliteral verbs and imperatives of defective verbs.

Interesting ideas

  • I1. The system first looks up root and pattern together, then falls back to mapping them independently, trading a little precision for coverage.
  • I2. The authors measure how much an independence assumption costs by running the joint and factorized mappings on the same test set and comparing the results.
  • I3. An oracle experiment gives the lexicon perfect coverage, which isolates errors coming from the analyzer and generator, and the remaining errors are then split by component.
  • I4. A new language variety is supported by editing a declarative hierarchy of morphological classes with non-monotonic inheritance, while the processing engine stays untouched.
  • I5. Recall is always reported next to ambiguity, measured as the average number of candidate outputs, so recall cannot be raised simply by producing more candidates.

For Mova

How these ideas could serve Mova, our small open language model — its files, stages and tests:

  • I1 (back): Translation through the meta-world could transfer a lemma and its inflection jointly when a known pairing exists, and recombine them separately only when it does not.
  • I2 (learn): Any factorized shortcut, such as treating frame choice and role choice as independent, could be priced the same way before it is adopted.
  • I3 (learn): Pipeline evaluations could include oracle runs at each stage, so a failure can be attributed to the tokenizer, the parser or the frame layer rather than the end result only.
  • I4 (tokens): The crate en morphology could hold English irregularities and dialect variants as inherited class overrides, which keeps the engine variety-independent and every rule inspectable.
  • I5 (answer): Readers that propose several candidate answers or referents could report how many candidates they produced alongside accuracy, which exposes over-generation.

Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.

 Institutions

🔬Laboratoire d’Informatique Fondamentale de MarseilleMarseille · France4 works · 358 citations

 Fields, topics and concepts

📚 Shelves 4The field of science this work belongs to (OpenAlex fields).Computer Science · 16,667 worksArtificial Intelligence · 16,194 worksArts and Humanities · 16,039 worksLanguage and Linguistics · 15,891 works

🧭 Topics 3Research subjects and languages the work is about.Machine translation · 1,087 worksSemitic languages · 2,908 worksMorphology · 405 works

Cite

Cite as Ahmed Hamdi, Rahma Boujelbane, Nizar Habash and Alexis Nasr. 2013. The Effects of Factorizing Root and Pattern Mapping in Bidirectional Tunisian - Standard Arabic Machine Translation. Proceedings of Machine Translation Summit XIV: Papers.

The paper itself, not this page. BibTeX from the library's record of the work.

@inproceedings{hamdi2013effects,
  title = {{The Effects of Factorizing Root and Pattern Mapping in Bidirectional Tunisian - Standard Arabic Machine Translation}},
  author = {Ahmed Hamdi, Rahma Boujelbane, Nizar Habash and Alexis Nasr},
  booktitle = {Proceedings of Machine Translation Summit XIV: Papers},
  year = {2013},
  url = {https://aclanthology.org/2013.mtsummit-papers.16/}
}

References

28 references, 15 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

  1. Abo Bakr, Hitham, Khaled Shaalan, and Ibrahim Ziedan. 2008. A Hybrid Approach for Converting Written Egyptian Colloquial Dialect into Diacritized Arabic. In The 6th International Conference on Informatics and Systems, INFOS2008. Cairo University. find
  2. Al-Sabbagh, Rania and Roxana Girju. 2010. Mining the Web for the Induction of a Dialectical Arabic Lexicon. In Calzolari, Nicoletta, Khalid Choukri, Bente Maegaard, Joseph Mariani, Jan Odijk, Stelios Piperidis, Mike Rosner, and Daniel Tapias, editors, LREC. European Language Resources Association.
  3. Altantawy, Mohamed, Nizar Habash, Owen Rambow, and Ibrahim Saleh. 2010. Morphological Analysis and Generation of Arabic Nouns: A Morphemic Functional Approach. In Proceedings of the International Conference on Language Resources and Evaluation, LREC. Valletta, Malta.
  4. Altantawy, Mohamed, Nizar Habash, and Owen Rambow. 2011. Fast Yet Rich Morphological Analysis. In Proceedings of the 9th International Workshop on Finite-State Methods and Natural Language Processing (FSMNLP 2011), Blois, France. find
  5. Buckwalter, Tim. 2002. Buckwalter Arabic Morphological Analyzer Version 1.0. Linguistic Data Consortium, University of Pennsylvania, 2002. LDC Catalog No.: LDC2002L49. ↗
  6. Chiang, David, Mona Diab, Nizar Habash, Owen Rambow, and Safiullah Shareef. 2006. Parsing Arabic Dialects. In Proceedings of the European Chapter of ACL (EACL). ↗
  7. Dhouib, Elmoncef. 2007. El Makki w-Zakiyya. Publishing House Manshuwrat Manara, Tunis, Tunisia. find
  8. Eskander, Ramy, Nizar Habash, Owen Rambow, and Nadi Tomeh. 2013. Processing Spontaneous Orthography. In Proceedings of the 2013 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies (NAACL-HLT), Atlanta, GA. find
  9. Habash, Nizar and Owen Rambow. 2005. Arabic Tokenization, Part-of-Speech Tagging and Morphological Disambiguation in One Fell Swoop. In Proceedings of the 43rd Annual Meeting of the Association for Computational Linguistics (ACL’05), pages 573– 580, Ann Arbor, Michigan.
  10. Habash, Nizar and Owen Rambow. 2006. MAGEAD: A morphological analyzer and generator for the Arabic dialects. In Proceedings of the 21st International Conference on Computational Linguistics and 44th Annual Meeting of the Association for Computational Linguistics, pages 681–688, Sydney, Australia, July. Association for Computational Linguistics. ↗
  11. Habash, Nizar, Owen Rambow, and George Kiraz. 2005. Morphological Analysis and Generation for Arabic Dialects. In Proceedings of the ACL Workshop on Computational Approaches to Semitic Languages, pages 17–24, Ann Arbor, Michigan.
  12. Habash, Nizar, Abdelhadi Soudi, and Tim Buckwalter. 2007. On Arabic Transliteration. In van den Bosch, A. and A. Soudi, editors, Arabic Computational Morphology: Knowledge-based and Empirical Methods. Springer. ↗
  13. Habash, N., R. Eskander, and A. Hawwari. 2012a. A Morphological Analyzer for Egyptian Arabic. In NAACL-HLT 2012 Workshop on Computational Morphology and Phonology (SIGMORPHON2012), pages 1–9.
  14. Habash, Nizar, Mona Diab, and Owen Rabmow. 2012b. Conventional Orthography for Dialectal Arabic. In Proceedings of the Language Resources and Evaluation Conference (LREC), Istanbul.
  15. Habash, Nizar, Ryan Roth, Owen Rambow, Ramy Eskander, and Nadi Tomeh. 2013. Morphological Analysis and Disambiguation for Dialectal Arabic. In Proceedings of the 2013 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies (NAACL-HLT), Atlanta, GA.
  16. Habash, Nizar. 2007. Arabic Morphological Representations for Machine Translation. In van den Bosch, A. and A. Soudi, editors, Arabic Computational Morphology: Knowledge-based and Empirical Methods. Springer. ↗
  17. Habash, Nizar. 2010. Introduction to Arabic Natural Language Processing. Morgan & Claypool Publishers. ↗
  18. Hamdi, Ahmed, Rahma Boujelbane, Nizar Habash, and Alexis Nasr. 2013. Un système de traduction de verbes entre arabe standard et arabe dialectal par analyse morphologique profonde. In In proceedings of Traitement Automatique du Langage Naturel (TALN 2013). find
  19. Kiraz, George Anton. 2000. Multitiered nonlinear morphology using multitape finite automata: a case study on Syriac and Arabic. Computational Linguistics, 26(1):77–105, March. ↗
  20. Maamouri, Mohamed, Ann Bies, Tim Buckwalter, and Wigdan Mekki. 2004a. The Penn Arabic Treebank: Building a Large-Scale Annotated Arabic Corpus. In NEMLAR Conference on Arabic Language Resources and Tools, pages 102–109, Cairo, Egypt. find
  21. Maamouri, Mohamed, Tim Buckwalter, and Christopher Cieri. 2004b. Dialectal Arabic Telephone Speech Corpus: Principles, Tool design, and Transcription Conventions. In NEMLAR International Conference on Arabic Language Resources and Tools. find
  22. Mohamed, Emad, Behrang Mohit, and Kemal Oflazer. 2012. Transforming standard arabic to colloquial arabic. In Proceedings of the 50th Annual Meeting of the Association for Computational Linguistics (Volume 2: Short Papers), pages 176–180, Jeju Island, Korea, July. Association for Computational Linguistics.
  23. Riesa, Jason and David Yarowsky. 2006. Minimally Supervised Morphological Segmentation with Applications to Machine Translation. In Proceedings of the 7th Conference of the Association for Machine Translation in the Americas (AMTA06), pages 185– 192, Cambridge,MA. find
  24. Salloum, Wael and Nizar Habash. 2011. Dialectal to Standard Arabic Paraphrasing to Improve Arabic-English Statistical Machine Translation. In Proceedings of the First Workshop on Algorithms and Resources for Modelling of Dialects and Language Varieties, pages 10–21, Edinburgh, Scotland. ↗
  25. Salloum, Wael and Nizar Habash. 2013. Dialectal Arabic to English Machine Translation: Pivoting through Modern Standard Arabic. In Proceedings of the 2013 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies (NAACL-HLT), Atlanta, GA.
  26. Sawaf, Hassan. 2010. Arabic dialect handling in hybrid machine translation. In Proceedings of the Conference of the Association for Machine Translation in the Americas (AMTA), Denver, Colorado.
  27. Smrž, Otakar. 2007. Functional Arabic Morphology. Formal System and Implementation. Ph.D. thesis, Charles University in Prague, Prague, Czech Republic. find
  28. Zbib, Rabih, Erika Malchiodi, Jacob Devlin, David Stallard, Spyros Matsoukas, Richard Schwartz, John Makhoul, Omar F. Zaidan, and Chris Callison-Burch. 2012. Machine Translation of Arabic Dialects. In Proceedings of the 2012 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, pages 49–59, Montréal, Canada. Meta-Evaluation of a Diagnostic Quality Metric for Machine Translation Sudip Kumar Naskar ∗ Computer and System Sciences Visva-Bharati University India sudip.naskar@visva-bharati.ac.in Abstract

Updated

↑ back to top

This page in Markdown → https://md.research.elamur.ai/free/en/papers/hamdi-2013-effects-factorizing-root-pattern-mapping/