📄 Tools for Arabic Natural Language Processing: a case study in qalqalah prosody

Summary

The paper addresses the automatic detection of qalqalah, a "bouncing" articulation that Qur'anic recitation (taǧwīd) applies to five voiced plosive consonants when they lack a vowel or stand before a pause. These conditions are visible in fully diacritised script, so the authors encode them as Unicode regular expressions in Python/NLTK. They run the patterns over the 77430-word Boundary-Annotated Qur'an Dataset for Machine Learning. A verse-level events algorithm first collects every verse containing a candidate letter, tagged with its chapter-verse ID. Word-level patterns then sort the hits into three strengths: kubrā (very strong), kabīrah (strong) and saġīrah (weak). The authors build frequency lists for each type. They then use the Semantic Pathways toolkit to compare Meccan and Medinan chapters by log-likelihood keyness, and several frequent qalqalah content words turn out to be significant discriminators. From this they propose that qalqalah may serve as a salience cue in recitation. Finally, their Arabic-to-IPA transcription technology produces an open pronunciation guide that links each qalqalah word to its IPA form and verse location, useful for learners and phonetic research.

Key points

• The source data is the Boundary-Annotated Qur'an (version 2.0), 77,430 words of fully vowelised text. A parallel paper adds syllabified IPA tiers to it.
• The qalqalah letter set is {ب ج د ط ق}. The regexes rely on the Arabic Unicode range \u0621–\u0652 plus explicit codes for šadda, sukūn, short vowels and tanwīn.
• Type rules:
  • kubrā: a geminate letter at the end of a word, before a pause or at verse end.
  • kabīrah: the same position but not geminate.
  • saġīrah: a word-internal letter carrying sukūn.
  • All three depend on the reciter actually pausing.
• NLTK FreqDist gives the top-ten word types per type. Words are tagged with the traditional Arabic {N, V, P} scheme, so adverbs of time count as nouns.
• In the Semantic Pathways comparison, rabbi ("lord") is key for Meccan chapters, while al-kitābi and ʿaḏābun are significant for Medinan ones. The SALMA tagger's letter/diacritic n-gram counts were used to check that the Arabic→IPA mapping is complete.

Interesting ideas

• I1. Detection runs as a cheap high-recall pass (any verse containing a candidate letter) followed by precise per-type patterns, which keeps the rules simple and lets each one be audited separately.
• I2. Before character-level conversion, multi-character orthographic sequences such as diacritic clusters for diphthongs are resolved as units, rather than mapping symbol by symbol.
• I3. The completeness of a hand-built mapping is checked by counting every character n-gram that occurs in the corpus and confirming that each one is covered.
• I4. A formal surface feature (a pronunciation effect) is tested for meaning by checking whether its words are over-represented among log-likelihood keywords when sub-corpora are compared.
• I5. A coarse tag set native to the language's own grammar ({N, V, P}) is kept to avoid forcing mismatched foreign categories onto the data.

For Mova

How these ideas could serve Mova, our small open language model — its files, stages and tests:

• I1 (tokens): Rare orthographic phenomena such as clitics or ambiguous apostrophes could be found with a broad candidate sweep followed by narrowly scoped, separately testable confirmation rules.
• I2 (tokens): Multi-character units like contractions, hyphenated compounds or numeric expressions could be normalised first, so the tagger and lemmatiser never see half-split pieces.
• I3 (learn): Hand-written rule tables (lemma exceptions, UD conversion rules) could be audited against corpus n-gram inventories to surface unhandled cases automatically.
• I4 (world): Entity salience in a text could be estimated by keyness against a reference corpus, giving an explainable ranking of which entities the text foregrounds.
• I5 (tree): A coarse fallback tier beside fine UD tags would let parsing and dialect conversion degrade gracefully where fine categories disagree across treebanks.