Elamur Research⌕
☰
home / papers / 2014 / Improved Sentence-Level Arabic Dialect Classification

Improved Sentence-Level Arabic Dialect Classification

Paper · 2014 · Christoph Tillmann, Saab Mansour and Yaser Al-Onaizan

Summary

The paper tackles sentence-level identification of Egyptian Arabic (ARZ) versus Modern Standard Arabic (MSA). The authors need this to make their Arabic-English SMT system dialect-aware. They use the ARZ-MSA part of the Arabic Online Commentary (AOC) dataset, which consists of crowd-annotated reader comments from the Al-Youm Al-Sabe' newspaper. As a second, out-of-domain test they add the BOLT DEV12 tune set.

The method is a linear SVM over sparse binary features computed mostly per token:

  • token n-grams
  • PoS unigrams
  • membership in two ARZ dictionaries
  • token- and sentence-level labels from the AIDA dialect tool

Language-model scores and "meta" features of informality were also tried, but the best system does not need them.

Under 10-fold stratified cross-validation the best configuration reaches 89.1% accuracy, 1.3 points above the prior LM-based result. Informality features hurt badly on the second data source. A first SMT experiment splits the training data with the classifier and builds per-dialect phrase tables, which capture dialect-dependent word senses. The work shows that light feature engineering can match more specialized pipelines.

Key points

  • Data:
    • AOC ARZ-MSA portion: 13,512 MSA sentences (334K words) and 12,527 ARZ sentences (327K words).
    • DEV12 (LDC2012E30, BOLT Phase 1 dev-tune): 1,219 sentences labeled by a native speaker.
  • Training uses L1-regularized L2-loss support vector classification with C = 0.5. The general-purpose tokenizer follows Lee et al. (2003) and comes with a maximum-entropy PoS tagger.
  • Two dictionaries were used:
    • about 41,000 ARZ tokens taken from AIDA resources;
    • about 70 hand-checked ARZ tokens found by contrasting IBM Model 1 lexicons trained on ARZ vs. MSA data.
  • Best result is 89.1% accuracy on AOC, a 1.3% absolute gain over Zaidan and Callison-Burch (2014). The AIDA-based features give a significant improvement.
  • On the extra data source, informality features (exclamation runs, punctuation, token length, word lengthening) significantly reduce accuracy.

Interesting ideas

  • I1. A compact dialect word list can be mined by training separate translation lexicons on two varieties, ranking words whose translation profiles diverge most, and letting a native speaker keep the true markers; this gives a cheap, high-precision resource.
  • I2. Testing on a second source exposed features that measured style rather than dialect, since they did well in-domain but failed elsewhere.
  • I3. An external tool's verdict (AIDA) is used as one weighted feature, not as the final answer, so the classifier learns how much to trust it.
  • I4. The error analysis traced a mistake to a dictionary word that was really a fragment of a split abbreviation, showing how context-free lexicon triggers fail.
  • I5. Splitting parallel training data with the classifier produced separate phrase tables in which dialect-only senses (e.g., 'car' vs. 'Arab') rank differently.

For Mova

How these ideas could serve Mova, our small open language model — its files, stages and tests:

  • I1 (tokens): the same contrastive mining over two English registers or treebanks could propose variant spellings and lemmas for the morphology layer, with a human or LLM confirming each entry.
  • I2 (learn): every learned signal should be checked on a differently sourced test set as a negative control, so shortcuts based on genre or punctuation are caught before they enter silver data.
  • I3 (learn): witness parsers' outputs could enter a linear agreement model as features, giving an explainable record of which witness swayed each silver decision.
  • I4 (tokens): dictionary-based features should fire only after the tokenizer has resolved abbreviations and fragments, and each firing should be logged so wrong triggers stay traceable.
  • I5 (back): conditioning lexical choices on a detected register of the source text could steer generation and meta-world translation toward the sense that fits that variety.

Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.

 Institutions

🏢IBMArmonk · United States139 works · 70,691 citations🔬IBM Research - Thomas J. Watson Research CenterYorktown Heights · United States53 works · 49,193 citations🎓RWTH Aachen UniversityAachen · Germany22 works · 4,936 citations

 Fields, topics and concepts

📚 Shelves 4The field of science this work belongs to (OpenAlex fields).Computer Science · 16,667 worksArtificial Intelligence · 16,194 worksArts and Humanities · 16,039 worksLanguage and Linguistics · 15,891 works

🧭 Topics 1Research subjects and languages the work is about.Semitic languages · 2,908 works

Cite

Cite as Christoph Tillmann, Saab Mansour and Yaser Al-Onaizan. 2014. Improved Sentence-Level Arabic Dialect Classification. Proceedings of the First Workshop on Applying NLP Tools to Similar Languages, Varieties and Dialects.

The paper itself, not this page. BibTeX from the library's record of the work. DOI 10.3115/v1/w14-5313

@inproceedings{tillmann2014improved,
  title = {{Improved Sentence-Level Arabic Dialect Classification}},
  author = {Christoph Tillmann, Saab Mansour and Yaser Al-Onaizan},
  booktitle = {Proceedings of the First Workshop on Applying NLP Tools to Similar Languages, Varieties and Dialects},
  year = {2014},
  doi = {10.3115/v1/w14-5313},
  url = {https://doi.org/10.3115/v1/w14-5313}
}

References

24 references, 9 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

  1. Timothy Baldwin and Marco Lui. 2010. Language Identification: The Long and the Short of the Matter. In Proc. of HLT’10, pages 229–237, Los Angeles, California, June. find
  2. William Cavnar and John M. Trenkle. 1994. N-gram-based Text Categorization. In In Proceedings of SDAIR-94, 3rd Annual Symposium on Document Analysis and Information Retrieval, pages 161–175. find
  3. Boxing Chen, George Foster, and Roland Kuhn. 2013. Adaptation of reordering models for statistical machine translation. In Proc. of HLT’13, pages 938–946, Atlanta, Georgia, June. find
  4. David Chiang, Steve DeNeefe, and Michael Pust. 2011. Two Easy Improvements to Lexical Weighting. In Proc. of HLT’11, pages 455–460, Portland, Oregon, USA, June. find
  5. Michael Collins. 2002. Discriminative Training Methods for Hidden Markov Models: Theory and Experiments with Perceptron Algorithms. In Proceedings of EMNLP’02, pages 1–8, Philadelphia,PA, July.
  6. Ted Dunning. 1994. Statistical Identification of Language. technical report mccs 94-273. Technical report, New Mexico State University. find
  7. Heba Elfardy and Mona Diab. 2012. Aida: Automatic Identification and Glossing of Dialectal Arabic. In Proceedings of the 16th EAMT Conference (Project Papers), pages 83–83, Trento, Italy, May.
  8. Heba Elfardy and Mona Diab. 2013. Sentence level dialect Identification in arabic. In Proc. of the ACL 2013 (Volume 2: Short Papers), pages 456–461, Sofia, Bulgaria, August.
  9. Rong-En Fan, Kai-Wei Chang, Cho-Jui Hsieh, Xiang-Rui Wang, and Chih-Jen Lin. 2008. LIBLINEAR: a Library for Large Linear Classification. Machine Learning Journal, 9:1871–1874. Joseph L Fleiss. 76(5):378. 1971. Measuring nominal scale agreement among many raters. ↗
  10. George Foster, Cyril Goutte, and Roland Kuhn. 2010. Discriminative Instance Weighting for Domain Adaptation in Statistical Machine Translation. In Proc. of EMNLP’10, pages 451–459.
  11. C.-J. Hsieh, K.-W. Chang, C.-J. Lin, S.S. Keerthi, and S.Sundararajan. 2008. A Dual Coordinate Descent Method for Large-scale linear SVM. In ICML, pages 919–926, Helsinki,Finland. find
  12. Chu-Ren Huang and Lung-Hao Lee. 2008. Contrastive Approach towards Text Source Classication based on top-bag-of-word Similarity. In PACLIC 2008, pages 404–410, Cebu City, Philippines. find
  13. Philipp Koehn and Josh Schroeder. 2007. Experiments in Domain Adaptation for Statistical Machine Translation. In Proceedings of the Second Workshop on Statistical Machine Translation (WMT07), pages 224–227.
  14. Canasai Kruengkrai, Prapass Srichaivattana, Virach Sornlertlamvanich, and Hitoshi Isahara. 2005. Language Identification based on string kernels. In In Proceedings of the 5th International Symposium on Communications and Information Technologies (ISCIT-2005, pages 896–899. find
  15. Young-Suk Lee, Kishore Papineni, Salim Roukos, Ossama Emam, and Hany Hassan. 2003. Language Model Based Arabic Word Segmentation. In Proc. of the 41st Annual Conf. of the Association for Computational Linguistics (ACL 03), pages 399–406, Sapporo, Japan, July.
  16. Marco Lui and Paul Cook. 2013. Classifying English Documents by National Dialect. In Proc. Australasian Language Technology Workshop, pages 5–15.
  17. Mona Mona Diab, Heba Elfardy, and Yassine Benajiba. 2009–2011. AIDA Automatic Identification of Arabic Dialectal Text. a Tool for Dialect Identification & Classification, Named Entity Recognition, English and Modern Standard Arabic Glossing and Normalization. find
  18. Rico Sennrich. 2012. Perplexity minimization for translation model domain adaptation in statistical machine translation. In Proc. of EACL’12, pages 539–549.
  19. Liling Tan, Marcos Zampieri, Nicola Ljubešić, and Jörg Tiedemann. 2014. Merging Comparable Data Sources for the Discrimination of Similar Languages: The DSL Corpus Collection. In 7th Workshop on Building and Using Comparable Corpora at LREC’14, Reykjavik, Iceland, September. find
  20. D. Trieschnigg, D. Hiemstra, M. Theune F. Jong, and T. Meder. 2012. An Exploration of Language Identication Techniques for the Dutch Folktale Database. In Adaptation of Language Resources and Tools for Processing Cultural Heritage Workshop (LREC 2012), Istanbul, Turkey, May. find
  21. Psychological Bulletin, Omar F. Zaidan and Chris Callison-Burch. 2011. Crowdsourcing translation: Professional quality from non-professionals. In Proc. of ACL / HLT 11, pages 1220–1229, Portland, Oregon, USA, June.
  22. Omar F. Zaidan and Chris Callison-Burch. 2014. Arabic Dialect Classification. CL, 40(1):171–202. find
  23. Marcos Zampieri and Binyam Gebrekidan Gebre. 2012. Automatic Identication of Language Varieties: The case of Portuguese. In Konvens 12, pages 233–237, Vienna, Austria. find
  24. Bing Zhao and Yaser Al-Onaizan. 2008. Generalizing Local and Non-Local word-reordering patterns for syntax-based machine translation. In Proc. of EMNLP’08, pages 572–581, Honolulu, Hawaii, October. find

Updated

↑ back to top

This page in Markdown → https://md.research.elamur.ai/free/en/papers/tillmann-2014-improved-sentence-level-arabic-dialect/