📄 Determining Case in Arabic: Learning Complex Linguistic Behavior Requires Complex Linguistic Features

Summary

The paper addresses automatic case assignment (nominative, accusative, genitive) for nouns and adjectives in Modern Standard Arabic. Case is usually written only with optional short-vowel diacritics, and it is a leading cause of errors in full diacritization. Even native speakers lack intuitions about it, because the spoken dialects have lost case. The authors work on gold trees from the Arabic Treebank (ATB) and compare a machine-learned classifier with hand-written rules.

The key move is a change of representation. Features read directly from ATB phrase structure, which encodes predicate-argument structure, are not enough. So the authors rebuild the trees around case assigners: idafa, tamyiz with numbers, inna-type conjunctions, copulas, and agreement. On this representation the learned system reaches about 4.2% error.

A manual error analysis then shows that most remaining "errors" are actually treebank annotation mistakes. Once these are discounted, the error rate falls to roughly 0.8%, and the rules become more accurate than the learner. This makes the tool useful both for diacritization and for treebank quality control.

Key points

• Data: ATB part 3 (LDC), with the training/devtest split of Zitouni et al. (2006) and the devtest halved into dev and test sets. Training has about 367K words; dev and test have about 33K words each.
• Of 133,250 case-marked training nominals, 66.4% are GEN, 18.5% ACC and 15.1% NOM. Nominals marked NO-CASE (invariables) are excluded from evaluation.
• Motivation from earlier work: diacritization word error rate drops by 9.4% absolute (to 5.5%) when word-final diacritics need not be predicted.
• The case-oriented representation helps the statistical system much more than the rules. The gap between the two grows to a 25% error reduction in favour of the learner.
• Error analysis on 105 dev sentences: the rules reach 94.1% accuracy and the learner 96.2%. Gold POS or tree errors account for 69% of the learner's errors and 86% of the rules' errors. The learner's genuine system-error share (31%) is about twice that of the rules (14%), meaning it absorbs treebank noise.

Interesting ideas

• I1. Re-encode the tree around the phenomenon you are predicting (here, who assigns case to whom) instead of reusing the annotation's native predicate-argument view, because the right features may simply not be visible in the original structure.
• I2. Separate case-by-assignment from case-by-agreement: modifiers and conjuncts copy the value of their governor rather than receiving it from an assigner, so the two decisions should be modelled differently.
• I3. Classify every system error in the analysis as either a genuine system mistake or a gold POS or tree error, because a large share of apparent errors may really be bad annotation.
• I4. A high-precision rule system makes a good annotation auditor: when it disagrees with the gold data, the gold is often wrong, whereas a learner tends to reproduce the annotators' mistakes.
• I5. Hand-written rules count as empirical NLP when they are developed and tested on held-out splits like any model, and on some phenomena they match or beat learning.

For Mova

How these ideas could serve Mova, our small open language model — its files, stages and tests:

• I1 (tree): before training morphology- or agreement-sensitive classifiers on UD trees, derive a task-specific view (e.g. agreement controller → target edges) rather than reading features straight off deprels.
• I2 (tokens): feature prediction (number, case on pronouns, person) can propagate through amod/conj chains from the head instead of predicting each token independently, which also yields a clear explanation trail.
• I3 (learn): when evaluating against silver or gold treebanks, sample disagreements and label them system vs. gold error, so that noisy references do not hide the true accuracy.
• I4 (learn): use explainable rule checks (e.g. agreement or case constraints on English pronouns) as an extra witness that flags suspicious silver trees for repair rather than training on them.
• I5 (frames): rule-based frame and role assignment can be kept and benchmarked on dev/test splits alongside learned components, choosing per phenomenon based on measured accuracy.