Elamur Research⌕
☰
home / papers / 2012 / Machine Translation of Arabic Dialects

Machine Translation of Arabic Dialects

Summary

The paper addresses the shortage of parallel data for translating Arabic dialects, which dominate informal online writing but differ sharply from Modern Standard Arabic (MSA). The authors filtered a roughly 350M-word LDC web corpus down to about 4M words of likely dialectal text, using hand-picked dialect keywords drawn from the Levantine Fisher and Egyptian CallHome transcripts. Amazon's Mechanical Turk workers then labeled each document's dialect, split long passages into sentences, and translated the Levantine and Egyptian material into English. The result is about 1.5M words of parallel data, costing roughly three cents per word.

Systems trained on this modest dialectal corpus beat an MSA system built on about 150M words by 6.3 BLEU on Egyptian test data and 7.0 BLEU on Levantine. The authors also simulated pivoting through MSA by having workers rewrite dialect test sentences into MSA. This helped only when little dialectal training data was available. The main lesson is that small amounts of in-domain dialect data outweigh large out-of-domain resources, and that crowdsourcing can supply such data cheaply.

Key points

  • The corpora contain about 1.1M words of Levantine-English and 380k words of Egyptian-English text. Table 1 reports 1,516,856 translated words at a total cost of $44,065, or 2.9 cents per word.

  • In the dialect classification step, three Turkers labeled each whole document, and a label was kept when two agreed. The resulting split was:

    • 43% Gulf/Iraqi
    • 28% Levantine
    • 11% Egyptian
    • 16% unclassifiable

    Gulf/Iraqi was not translated because almost no MTurk workers came from that region.

  • The pipeline sustained about 200k translated words per week, at around one tenth the cost of professional translation.

  • In the MSA-pivot experiment (Table 7), manually mapping Levantine input to MSA raised the MSA-only baseline from 12.29 to 14.59 BLEU. Once 400k or more dialect words were added to training, direct translation won. With 1500KW of dialect data, direct translation scored 19.11 BLEU versus 17.20 for pivoting, even though the pivoted input had fewer OOVs.

  • The models were also checked on an independent set of Egyptian Facebook public pages (E-Facebook-Tune and E-Facebook-Test).

Interesting ideas

  • I1. Keyword pre-filtering before crowdsourcing. Cheap filtering with a small list of dialect-marker words, mined from speech transcripts, concentrates annotation money on promising text. This applies to any low-resource variety, such as surzhyk or regional Ukrainian, hidden inside a large mixed web corpus.
  • I2. Anti-cheating measures for crowd translation. Source sentences were shown as images so workers could not paste them into online MT. Submissions were spot-checked against Google and Bing output, and translations with many non-dictionary words were rejected. These are simple, transferable ways to protect a crowdsourced translation budget.
  • I3. Gold-item quality control with a trusted tier. Known-answer items were mixed in and scored automatically, using METEOR and BLEU-1 for translations and precision/recall of split points for segmentation. Reliable workers moved to a better-paid queue with fewer control items, which raises useful output without dropping quality checks.
  • I4. Human simulation as an upper bound. Instead of building an automatic dialect-to-MSA converter, the authors had people do the conversion. This gives an optimistic ceiling for the pivot strategy, a cheap way to decide whether a pipeline component is worth building at all.
  • I5. Coverage is not the same as fit. The pivot approach lost despite lower OOV rates, showing that vocabulary coverage alone can mislead. Domain and genre mismatch should be measured directly when choosing training data.

Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.

Fields: Natural language processing

Topics: Machine translation

Cited by

119 papers of our library cite it (of the 140884 whose bibliographies we read); 21 of them have a page here, 98 not yet.

Cite

The paper itself, not this page. BibTeX from the library's record of the work.

@inproceedings{zbib2012machine,
  title = {{Machine Translation of Arabic Dialects}},
  author = {Zbib, Rabih and Malchiodi, Erika and Devlin, Jacob and Stallard, David and Matsoukas, Spyros and Schwartz, Richard and Makhoul, John and Zaidan, Omar F. and Callison-Burch, Chris},
  booktitle = {Proceedings of the 2012 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies},
  year = {2012},
  url = {https://aclanthology.org/N12-1006/}
}

References

22 references, 9 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

  1. Hitham M. Abo Bakr, Khaled Shaalan, and Ibrahim Ziedan. 2008. A hybrid approach for converting written Egyptian colloquial dialect into diacritized Arabic. In The 6th International Conference on Informatics and Systems, INFOS2008, Cairo, Egypt. find
  2. Satanjeev Banerjee and Alon Lavie. 2005. Meteor: An automatic metric for MT evaluation with improved correlation with human judgments. In In Proc. of ACL 2005 Workshop on Intrinsic and Extrinsic Evaluation Measures for MT and/or Summarization, Ann Arbor, Michigan. find our card
  3. Chris Callison-Burch and Mark Dredze. 2010. Creating speech and language data with Amazon’s Mechanical Turk. In Proceedings of the NAACL HLT 2010 Workshop on Creating Speech and Language Data with Amazon’s Mechanical Turk, pages 1–12, Los Angeles, June. find
  4. David Chiang, Mona Diab, Nizar Habash, Owen Rambow, and Safiullah Shareef. 2006. Parsing Arabic dialects. In Proceedings of the Conference of the European Chapter of the Association for Computational Linguistics, Trento, Italy. ↗ our card
  5. David Chiang, Kevin Knight, and Wei Wang. 2009. 11,001 new features for statistical machine translation. In NAACL ’09: Proceedings of the 2009 Human Language Technology Conference of the North American Chapter of the Association for Computational Linguistics, Boulder, Colorado. find
  6. Jacob Devlin. 2009. Lexical features for statistical machine translation. Master’s thesis, University of Maryland, December. find
  7. Mohamed Embarki and Moha Ennaji, editors. 2011. Modern Trends in Arabic Dialectology. The Red Sea Press. find
  8. Charles A. Ferguson. 1959. Diglossia. Word, 15:325– 340. find
  9. Nizar Habash and Owen Rambow. 2005. Arabic tokenization, part-of-speech tagging and morphological disambiguation in one fell swoop. In Proceedings of the 43th Annual Meeting of the Association for Computational Linguistics (ACL), Ann Arbor, Michigan. find our card
  10. Nizar Habash and Owen Rambow. 2006. MAGEAD: A morphological analyzer and generator for the Arabic dialects. In Proceedings of the 44th Annual Meeting of the Association for Computational Linguistics (ACL), Sydney, Australia. ↗ our card
  11. Nizar Habash and Fatiha Sadat. 2006. Arabic preprocessing schemes for statistical machine translation. In Proceedings of the 2006 Human Language Technology Conference of the North American Chapter of the Association for Computational Linguistics, New York, New York. find our card
  12. Nizar Y. Habash. 2010. Introduction to Arabic Natural Language Processing. Morgan & Claypool. ↗
  13. Young-Suk Lee. 2004. Morphological analysis for statistical machine translation. In HLT-NAACL ’04: Proceedings of HLT-NAACL 2004, Boston, Massachusetts. ↗
  14. Franz Josef Och and Hermann Ney. 2003. A systematic comparison of various statistical alignment models. Computational Linguistics, 29(1):19–51. ↗
  15. Kishore Papineni, Salim Roukos, Todd Ward, and Wei-Jing Zhu. 2002. Bleu: a method for automatic evaluation of machine translation. In Proceedings of the 40th Annual Meeting of the Association for Computational Linguistics (ACL), Philadelphia, PA. find our card
  16. Jason Riesa and David Yarowsky. 2006. Minimally supervised morphological segmentation with applications to machine translation. In Proceedings of the 7th Conf. of the Association for Machine Translation in the Americas (AMTA 2006), Cambridge, MA. find
  17. Wael Salloum and Nizar Habash. 2011. Dialectal to standard Arabic paraphrasing to improve Arabic-English statistical machine translation. In Proceedings of the 2011 Conference of Empirical Methods in Natural Language Processing, Edinburgh, Scotland, UK. ↗
  18. Hassan Sawaf. 2010. Arabic dialect handling in hybrid machine translation. In Proceedings of the 9th Conf. of the Association for Machine Translation in the Americas (AMTA 2010), Denver, Colorado. find
  19. Libin Shen, Jinxi Xu, and Ralph Weischedel. 2008. A new string-to-dependency machine translation algorithm with a target dependency language model. In Proceedings of the 46th Annual Meeting of the Association for Computational Linguistics (ACL), pages 577–585, Columbus, Ohio. find our card
  20. Matthew Snover, Bonnie Dorr, Richard Schwartz, Linnea Micciulla, and Ralph Weischedel. 2006. A study of translation error rate with targeted human annotation. In Proceedings of the 7th Conf. of the Association for Machine Translation in the Americas (AMTA 2006), pages 223–231, Cambridge, MA. find
  21. Omar F. Zaidan and Chris Callison-Burch. 2011a. The Arabic online commentary dataset: an annotated dataset of informal Arabic with high dialectal content. In Proceedings of the 49th Annual Meeting of the Association for Computational Linguistics: Human Language Technologies, pages 37–41, Portland, Oregon, June. find our card
  22. Omar F. Zaidan and Chris Callison-Burch. 2011b. Crowdsourcing translation: Professional quality from non-professionals. In Proceedings of the 49th Annual Meeting of the Association for Computational Linguistics: Human Language Technologies, pages 1220– 1229, Portland, Oregon, June. find our card

↑ back to top

This page in Markdown → https://md.research.elamur.ai/free/en/papers/zbib-2012-machine-translation-arabic-dialects/