📄 The Columbia System in the QALB-2014 Shared Task on Arabic Error Correction

Summary

This paper presents Columbia University's entry in QALB-2014, the first shared task on automatic error correction for Modern Standard Arabic. The data is the QALB corpus of Aljazeera reader comments corrected by native speakers: 1.2M training words and about 50K words each for development and test. The team did not build one monolithic model. They built a cascade of specialised modules, each aimed at one error family:

• MADAMIRA's Alif/Ya spelling fixes
• an n-gram maximum-likelihood replacement table
• a Naïve Bayes corrector for unseen words
• a decision-tree punctuation inserter
• hand-written split/merge rules backed by SAMA

Two alternative versions swap in a dialect-to-MSA translator (Elissa) or a character-level SVM model (GSEC). The best version, CLMB-1, scored F1 67.18 on development and 67.91 on test with the M2 scorer, and it ranked first of nine teams. The work matters because it shows that a modular pipeline, built up step by step, can lead a whole-text correction task in a morphologically rich language.

Key points

• The training data has no error categories, but types can be recovered by aligning inputs with their corrections:
  • Punctuation is the largest class, about 39% of errors in the task description; another section says about 30%.
  • Alif/Ya spelling accounts for almost 30%.
• There were three submissions (CLMB-1, CLMB-2, CLMB-3).
  • CLMB-2 adds the Elissa dialectal corrector.
  • CLMB-3 uses GSEC, trained on only a quarter of the data because training was slow, followed by a unigram MLE.
• The CLMB-1 ablation on the development set shows how each module adds to F1:

• The punctuation model is a WEKA decision tree. It uses only the previous word's POS and whether the next word carries a "w"/"f" conjunctive proclitic. This small feature set beat the full set of MADAMIRA features over a wider window.
• On test, CLMB-1 reached P 73.34 / R 63.23 / F1 67.91. CLMB-2 scored F1 66.25 and CLMB-3 scored F1 65.22.

Interesting ideas

• I1. Start the cascade with a very precise but low-recall module, then add recall-oriented modules one at a time, measuring each addition with an ablation table; the gain from each stage becomes visible and easy to defend.
• I2. Build a lookup table from aligned source/target n-grams and use it only for confidently observed mappings, passing unseen words to a context-aware classifier; this cleanly separates memorisation from generalisation.
• I3. Train the punctuation predictor on text with all marks stripped, but at inference time only add missing marks and never overwrite the writer's own periods and commas; this asymmetric policy protects precision.
• I4. Accept a word split only when the leftover piece gets a valid analysis from a morphological analyzer, so splitting is guided by the lexicon rather than by surface form alone.
• I5. Recover error categories automatically by aligning raw inputs with their corrections, even when annotators gave only fixes; this turns plain parallel corrections into a typed error profile.

For Mova

How these ideas could serve Mova, our small open language model — its files, stages and tests:

• I1 (learn): Self-induced repair rules could be admitted in the same staged way, reporting P/R/F1 after each rule family so that any rule that lowers precision is caught.
• I2 (learn): From silver data agreed by witness parsers, keep only unambiguous token or label repairs as direct rules, and send ambiguous cases to the trained model or the LLM teacher tier.
• I3 (tokens): A sentence-boundary repairer for noisy English input could insert likely missing boundaries while trusting existing ones, which keeps UD sentence segmentation stable.
• I4 (tokens): Run-together words (e.g. missing spaces) could be split only when every resulting piece has an analysis in crate en's lexicon/morphology, which gives an explainable tokenizer repair.
• I5 (learn): Diffing parser output against corrected trees could classify disagreements by type (attachment, label, POS), which would show which repair rules are worth inducing first.