📄 INVITED TALK 1: Computational Processing of Arabic Dialects

Summary

This item is the one-page abstract of an invited talk, not a full research paper. It reports no experiments, datasets, metrics or results. The talk addresses the processing of Arabic. Arabic is described as a family of varieties. Modern Standard Arabic (MSA) is the official written norm used in news, education and culture across the Arab world. The regional dialects are informal, mostly spoken, and serve everyday conversation. Because NLP tools and corpora for Arabic have been built mainly for MSA, the dialects remain comparatively under-resourced. Interest in them is nonetheless growing. The talk surveys the difficulties dialects pose and the progress made so far. Two lines of work are mentioned: building new dialect-specific resources, and reusing existing MSA resources, or resources from another dialect, for a target dialect. The second line gets particular emphasis. Its value is that it suggests a cheaper path to coverage for low-resource varieties that sit close to a well-resourced standard.

Key points

• The talk was given at the LT4CloseLang workshop, whose topic is NLP for languages and variants that are near relatives of one another. The workshop was held with EMNLP 2014 in Doha, Qatar, on October 29, 2014.
• It frames Arabic as a standard variety (MSA) plus many informal spoken dialects, with most NLP research and resources focused on MSA.
• Two strategies for dialects are named: creating dialect resources and exploiting MSA or other-dialect resources. The latter is the talk's focus.
• The speaker, Nizar Habash (NYU Abu Dhabi), co-founded the Columbia Arabic Dialect Modeling group (CADIM) in 2005 with Mona Diab and Owen Rambow.
• He is the author of the book "Introduction to Arabic Natural Language Processing". His research covers machine translation and morphological analysis, generation and disambiguation.

Interesting ideas

• I1. Model a language as one standard variety surrounded by related informal variants, rather than as a single uniform system, so that tools can state which variety they target.
• I2. Bootstrap a low-resource variant by reusing the tools and annotations of a closely related, well-resourced standard instead of building everything from scratch.
• I3. Transfer can also go sideways, from one non-standard variant to another, not only from the standard down to a variant.
• I4. Treat the gap between standard and variant as an explicit challenge to describe, not as noise to ignore.

For Mova

How these ideas could serve Mova, our small open language model — its files, stages and tests:

• I1 (tokens): The English tokenizer and morphology could label input as standard written vs informal or dialectal and route nonstandard forms through a mapping layer before lemmatization.
• I2 (tree): Dialectal or informal English could be parsed by adapting the standard-English UD treebanks and the perceptron parser or gparse, combined with targeted UD dialect-conversion rules, rather than annotating new treebanks.
• I3 (learn): Silver data agreed by witness parsers on one informal register, such as dialogue in tales, could seed training for a neighbouring register before any gold data exists.
• I4 (learn): Evaluation sets could separate standard and informal English so that negative controls reveal where standard-trained components break on variant text.