Elamur Research⌕
☰
home / papers / 2018 / Towards a Computational Lexicon for Moroccan Darija: Words, Idioms, and Constructions

Towards a Computational Lexicon for Moroccan Darija: Words, Idioms, and Constructions

Paper · 2018 · Jamal Laoudi, Claire Bonial, Lucia Donatelli et al.

Summary

The paper tackles a practical gap. Moroccan Darija (MD) is an Arabic dialect with tens of millions of speakers, but it has no standard spelling and almost no computational resources. It now appears mostly in informal online writing. The authors ask what a computational lexicon for such a language should contain, and they build one in three layers of decreasing compatibility with classic dictionaries. The first layer is word-level entries with English glosses, drawn from reader comments on the news portal Hespress. The second is a collection of idioms whose meaning is not fully compositional. The third is an early constructicon of grammatical patterns. To get mostly-MD comments, they filtered the data with a token-level MD/MSA classifier. A native speaker then translated and glossed the shortest posts in context. For idioms, they discuss semantic typologies and translation strategies. For constructions, they first elicited MD renderings of known English constructions and then recorded native MD variants. The work matters for MD↔English translation and as a recipe for lexicons of other unwritten dialects.

Key points

  • Source data and filtering: Comments came from Hespress, a mix of MSA, MD, English, French and Amazigh. A post was kept only if it had at least 6 MD tokens and MD tokens outnumbered MSA tokens by at least 4:1. This left over 128,000 mostly-MD posts.
  • Translated corpus: A native speaker translated the 2,000 shortest posts. After removing other-language and offensive posts, 1,836 remained, with 17,261 tokens and 5,528 token types. Each type has on average 2.32 glosses.
  • Comparison with MADAR: MADAR is built on 1,045 concept keys translated in 25 cities, including Fes and Rabat. Of its 1,032 unique MD items, 342 also appear in the new lexicon. The two resources differ in two ways:
    • MADAR entries were translated into MD, whereas these were written originally in MD, so they show more spelling and morphological variants.
    • MADAR is travel-focused, while this lexicon covers politics, sports and entertainment.
  • Idiom analysis: Idioms are analysed using Nunberg et al.'s properties (conventionality, opacity, compositionality) and Fernando's three classes (pure idioms, semi-idioms, literal idioms).
  • Construction findings:
    • English comparatives and superlatives map onto MD constructions of similar form.
    • The X-er the Y-er pattern has several dissimilar MD renderings; Table 10 shows three.
    • English Caused-Motion and Intransitive Motion constructions come out as paraphrases in MD.

Interesting ideas

  • I1. Thresholded filtering of code-mixed text: Keep a text only if it has a minimum count of target-variety tokens and they dominate the competing variety by a fixed ratio; this is a cheap way to carve a clean subset out of mixed data.
  • I2. Three-layer lexicon: Plain words, fixed idioms stored as "words with spaces", and productive patterns kept in a separate constructicon, so each kind of unit is stored in the form that fits how variable it is.
  • I3. Idiom class decides treatment: Sort each idiom as pure, semi or literal, since opaque idioms, idioms with one figurative part, and transparent fixed phrases each call for different handling.
  • I4. Recording every native variant of one construction: Several dissimilar MD renderings of the comparative correlative were documented as alternatives for the same meaning, rather than a single canonical equivalent.
  • I5. Checking a lexicon against an independent one while noting provenance: Overlap with MADAR was measured, and the authors pointed out that entries translated into a language look different from entries native to it (less variation, narrower domain).

For Mova

How these ideas could serve Mova, our small open language model — its files, stages and tests:

  • I1 (learn): The same two-threshold gate could decide which raw sentences enter silver data, for example requiring enough in-vocabulary tokens and a low rate of foreign or noisy tokens before the witness parsers vote.
  • I2 (frames): Fixed idioms could become single lexical units before frame building, while flexible constructions become pattern rules matched over UD subtrees that contribute their own meaning to the frame.
  • I3 (frames): Tagging idiom entries with this class would tell prag whether to replace the whole phrase's meaning, reinterpret only one constituent, or keep literal role assignment.
  • I4 (back): Generation from the meaning graph could keep an inventory of alternative surface patterns per construction and pick among them, which gives more natural paraphrase and tests round-trip stability.
  • I5 (world): When compiled knowledge from WordNet, FrameNet or Wikidata is merged with lexicons induced from Mova's own texts, overlap counts plus a provenance label would reveal domain gaps and missing variants.

Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.

 Institutions

🎓Saarland UniversitySaarbrücken · Germany148 works · 18,287 citations🔬DEVCOM Army Research LaboratoryAdelphi · United States14 works · 2,292 citations

 Fields, topics and concepts

📚 Shelves 4The field of science this work belongs to (OpenAlex fields).Computer Science · 16,667 worksArtificial Intelligence · 16,194 worksArts and Humanities · 16,039 worksLanguage and Linguistics · 15,891 works

🧭 Topics 1Research subjects and languages the work is about.Idioms & multiword expressions · 217 works

Cite

Cite as Jamal Laoudi, Claire Bonial, Lucia Donatelli, Stephen Tratz and Clare Voss. 2018. Towards a Computational Lexicon for Moroccan Darija: Words, Idioms, and Constructions. Proceedings of the Joint Workshop on Linguistic Annotation, Multiword Expressions and Constructions (LAW-MWE-CxG-2018).

The paper itself, not this page. BibTeX from the library's record of the work.

@inproceedings{laoudi2018computational,
  title = {{Towards a Computational Lexicon for Moroccan Darija: Words, Idioms, and Constructions}},
  author = {Jamal Laoudi, Claire Bonial, Lucia Donatelli, Stephen Tratz and Clare Voss},
  booktitle = {Proceedings of the Joint Workshop on Linguistic Annotation, Multiword Expressions and Constructions (LAW-MWE-CxG-2018)},
  year = {2018},
  url = {https://aclanthology.org/W18-4910/}
}

References

45 references, 8 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

  1. Katsiaryna Aharodnik, Anna Feldman, and Jing Peng. 2018. Designing a Russian Idiom-Annotated Corpus. In Proceedings of the Language Resources and Evaluation Conference (LREC). find
  2. Monireh Akbari. 2013. Strategies for translating idioms. Journal of Academic and Applied Studies (Special Issue on Applied Linguistics) Vol, 3(8):32–41. find
  3. Faisal Al-Shargi, Aidan Kaplan, Ramy Eskander, Nizar Habash, and Owen Rambow. 2016. Morphologically Annotated Corpora and Morphological Analyzers for Moroccan and Sanaani Yemeni Arabic. In Proceedings of the Language Resources and Evaluation Conference (LREC).
  4. Linnéa Bäckström, Benjamin Lyngfelt, and Emma Sköldberg. 2014. Towards interlingual constructicography: On correspondence between constructicon resources for english and swedish. Constructions and Frames, 6(1):9– 33. find
  5. Mona Baker. 1992. In Other Words: A Coursebook on Translation. Routledge, United Kingdom. find
  6. Laura Banarescu, Claire Bonial, Shu Cai, Madalina Georgescu, Kira Griffitt, Ulf Hermjakob, Kevin Knight, Philipp Koehn, Martha Palmer, and Nathan Schneider. 2013. Abstract Meaning Representation for Sembanking. In Proceedings of the 7th Linguistic Annotation Workshop and Interoperability with Discourse, page 178–186.
  7. Francis Bond and Ryan Foster. 2013. Linking and extending an open multilingual wordnet. In Proceedings of the 51st Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), volume 1, page 1352–1362.
  8. Claire Bonial, Bianca Badarau, Kira Griffitt, Ulf Hermjakob, Kevin Knight, Tim O'Gorman, Martha Palmer, and Nathan Schneider. 2018. Abstract Meaning Representation of Constructions: The More We Include, the Better the Representation. In Proceedings of the 2018 Language Resources and Evaluation Conference (LREC). find
  9. Houda Bouamor, Nizar Habash, and Kemal Oflazer. 2014. The Multidialectal Parallel Corpus of Arabic. In Proceedings of the Language Resources and Evaluation Conference (LREC). find
  10. Houda Bouamor, Nizar Habash, Mohammad Salameh, Wajdi Zaghouani, Owen Rambow, Dana Abdurahim, Ossama Obeid, Salam Khalifa, Fadhi Eryani, Alexander Erdmann, and Kemal Oflazer. 2018. The MADAR Arabic Dialect Corpus and Lexicon. In Proceedings of the Language Resources and Evaluation Conference (LREC). find
  11. Kristin E. Brustad. 2002. The Syntax of Spoken Arabic. Georgetown University Press. We expect to release these resources to the research community, with further documentation in addition to this paper. find
  12. William Croft and D. Alan Cruse. 2004. Cognitive Linguistics. Cambridge University Press. find
  13. Kareem Darwish, Ahmed Abdelali, Hamdy Mubarak, Younes Samih, and Mohammed Attia. 2018. Diacritization of Moroccan and Tunisian Arabic Dialects: A CRF Approach. In Proceedings of the 3rd Workshop on Open-Source Arabic Corpora and Processing Tools. find
  14. Bonnie Dorr and Clare R. Voss. 2018. The Case for Systematically Derived Spatial Language Usage. In Proceedings of the NAACL 2018 Workshop on Spatial Language Understanding (SpLU). find
  15. Abdennebi El Haloui and Steven L. Bowman. 2011. Moroccan Arabic Verb Dictionary. Artisanal Treasures. find
  16. Afsaneh Fazly, Paul Cook, and Suzanne Stevenson. 2009. Unsupervised Type and Token Identification of Idiomatic Expressions. Computational Linguistics, page 61–103. ↗
  17. Christiane Fellbaum, editor. 1998. WordNet: An Electronic Lexical Database. MIT Press. find
  18. Chitra Fernando. 1996. Idioms and Idiomaticity. Oxford University Press. Google-Books-ID: 5lViAAAAMAAJ. find
  19. Charles Fillmore, Paul Kay, and Mary Catherine O'connor. 1988. Regularity and idiomaticity in grammatical constructions: The case of let alone. Language, page 501–538. ↗
  20. Charles Fillmore, Russell Lee-Goldman, and Russell Rhodes. 2012. The Framenet Constructicon. Sign-based construction grammar, page 309–372. find
  21. Charles Fillmore. 2008. Border conflicts: Framenet meets construction grammar. In Proceedings of the XIII EURALEX international congress, volume 4968. find
  22. Markus Forsberg, Richard Johansson, Linnéa Bäckström, Lars Borin, Benjamin Lyngfelt, Joel Olofsson, and Julia Prentice. 2014. From construction candidates to constructicon entries: An experiment using semi-automatic methods for identifying constructions in corpora. Constructions and Frames, 6(1):114–135. find
  23. Adele Goldberg. 1995. Constructions: A construction grammar approach to argument structure. University of Chicago Press. find
  24. Adele Goldberg. 2003. Constructions: a new theoretical approach to language. In TRENDS in Cognitive Sciences, volume 7(5). ↗
  25. Richard Slade Harrell and Harvey Sobelman, editors. 1966. A Dictionary of Moroccan Arabic: Moroccan-English English-Moroccan. Georgetown University Press. find
  26. Chikara Hashimoto and Daisuke Kawahara. 2008. Construction of an idiom coprus and its application to idiom identification based on wsd incorporating idiom-specific features. In Proceedings of the Empirical Methods for Natural Language Processing Conference (EMNLP). find
  27. Khalil Mrini and Francis Bond. 2017. Building the Moroccan Darija Wordnet (MDW) using Bilingual Resources. In Proceedings of the International Conference on Natural Language, Signal and Speech Processing (ICNLSSP). find
  28. Grace Muzny and Luke Zettlemoyer. 2013. Automatic Idiom Identification in Wiktionary. In Proceedings of the Empirical Methods for Natural Language Processing Conference (EMNLP). find
  29. Geoffrey Nunberg, Ivan A. Sag, and Thomas Wasow. 1994. Idioms. Language, 70(3):491–538. find
  30. Kyoko Hirose Ohara. 2016. Toward constructicon building for japanese in japanese framenet. Revista Veredas, 17(1). find
  31. Jing Peng and Anna Feldman. 2017. Automatic Idiom Recognition with Word Embeddings. In Proceedings of the Annual International Symposium on Information Management and Big Data. find
  32. Priyanka and R.M.K. Sinha. 2014. A System for Identification of Idioms in Hindi. In Seventh International Conference on Contemporary Computing (IC3). find
  33. Ivan A. Sag, Timothy Baldwin, Francis Bond, Ann Copestake, and Dan Flickinger. 2002. "Multiword Expressions: A Pain in the Neck for NLP?". In Proceedings of the Third International Conference on Computational Linguistics and Intelligent Text Processing (CICLing'02).
  34. Younes Samih and Wolfgang Maier. 2016a. An Arabic-Moroccan Darija Code-Switched Corpus. In Proceedings of the Language Resources and Evaluation Conference (LREC).
  35. Younes Samih and Wolfgang Maier. 2016b. Detecting Code-Switching in Moroccan Arabic Social Media. In SocialNLP Workshop at International Joint Conference on Artificial Intelligence (IJCAI). ↗
  36. Amir Shojaei. 2012. Translation of Idioms and Fixed Expressions: Strategies and Difficulties. Theory and Practice in Language Studies, 2(6), June. ↗
  37. Gary F. Simons and Charles D. Fennig, editors. 2018. Ethnologue: Languages of the World, Twenty-first edition. Online version: http://www.ethnologue.com. ↗
  38. Toshiyuki Takezawa, Genichiro Kikui, Masahide Mizushima, and Eiichiro Sumita. 2007. Multilingual Spoken Language Corpus Development for Communication Research. International Journal of Computational Linguistics & Chinese Language Processing: Special Issue, 12(3):303–324. find
  39. Leonard Talmy. 1985. Lexicalization patterns: Semantic structure in lexical forms. Language typology and syntactic description, 3(99):36–149. find
  40. Leonard Talmy. 2000. Toward a cognitive semantics, volume 2. MIT press. find
  41. Tiago Timponi Torrent, Ludmila Meireles Lage, Thais Fernandes Sampaio, Tatiane da Silva Tavares, and Ely Edison da Silva Matos. 2014. Revisiting border conflicts between FrameNet and Construction Grammar: Annotation policies for the Brazilian Portuguese Constructicon. Constructions and Frames, 6(1):34–51. ↗
  42. Simonetta Vietri. 2014. The Lexicon-Grammar of Italian Idioms. In Proceedings of the International Conference on Computational Linguistics (COLING). find
  43. Clare R Voss, Stephen Tratz, Jamal Laoudi, and Douglas Briesch. 2014. Finding Romanized Arabic Dialect in Code-Mixed Tweets. In Proceedings of the Language Resources and Evaluation Conference (LREC).
  44. Mohammad Ali Yaghan. 2008. “Arabizi”: A Contemporary Style of Arabic Slang. Design Issues, 24(2):39–52. ↗
  45. Rabih Zbib, Erika Malchiodi, Jacob Devlin, David Stallard, Spyros Matsoukas, Richard Schwartz, John Makhoul, Omar F. Zaidan, and Chris Callison-Burch. 2012. Machine Translation of Arabic Dialects. In Proceedings of the 2012 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, NAACL HLT '12, page 49–59, Stroudsburg, PA, USA. Association for Computational Linguistics.

Updated

↑ back to top

This page in Markdown → https://md.research.elamur.ai/free/en/papers/laoudi-2018-computational-lexicon-moroccan-darija-words/