📄 Fast and Robust Arabic Error Correction System

Summary

The paper presents a system built for the EMNLP 2014 shared task on automatic correction of Arabic text, using the QALB corpus. The authors argue that classic spellers are slow because they enumerate many candidate variants for each faulty word and then rank them, and that such spellers ignore the score of the system as a whole. Their alternative treats whole correction rules as the units to rank. They hand-design families of rules and estimate each rule's reliability on the training data as the share of its edits that turn out correct. The rule families cover morphological normalisation with BAMA-v2.0, colloquial-to-standard replacements, single-character typos, split and merged words, punctuation, a genitive agreement fix, and formatting clean-up. At test time, rules are applied in order of reliability. A running estimate of precision, recall and F-score, computed without gold labels, decides when to stop. The system scored F = 0.6817 on development data and 0.6573 on test data under the M2 scorer. It is presented as a fast, transparent alternative to candidate-generation correctors.

Key points

• Data is the QALB corpus from the shared task. The organisers did not release the tools behind the supplied features, so the authors computed their own features.
• Each rule's probability is its training-set precision: correct edits divided by performed edits. Pure formatting rules (removing kashida, collapsing repeated letters, putting commas between numbers) were manually given probability 1.
• Expected gold edits on the test set are estimated as the training error rate multiplied by the number of test words. Each applied rule adds N performed edits and p·N expected correct edits. The loop halts once the estimated F-score falls.
• BAMA-v2.0 drives several rule families:
  • It normalises alef/hamza, taa marbouta/haa and yaa/alef maqsura confusions, changing a word only when all of its analyses share one form.
  • It validates single-character replace, insert, delete and transpose edits.
  • It validates word splits.
• Evaluation used the M2 scorer, giving F = 0.6817 on development data and 0.6573 on test data. Punctuation is acknowledged as the weakest area.

Interesting ideas

• I1. Rank rules by their measured precision and apply them greedily, stopping when a label-free estimate of F-score (prior error rate plus per-rule precision) stops improving; this gives a principled cutoff without test annotations.
• I2. Generate one candidate edit at a time and accept it only if a morphological analyzer or lexicon recognises the result, skipping words known to be correct; this avoids full candidate enumeration.
• I3. Change a token only when every analysis returned by the analyzer agrees on the same surface form, and otherwise leave it untouched; this is a cheap unanimity guard against harmful edits.
• I4. Estimate reliability per concrete instance of a rule, such as a specific character pair, rather than per rule type, since identical mechanics can differ widely in accuracy.

For Mova

How these ideas could serve Mova, our small open language model — its files, stages and tests:

• I1 (learn): self-induced repair rules could be ordered by precision measured on silver data, and adding further rules would stop once the projected F on unlabelled text begins to decline.
• I2 (tokens): a typo-normalisation pass before tagging could try edits one by one and keep only forms the en lexicon or morphology accepts, with a protected list of known-good words.
• I3 (tokens): the lemmatizer could commit to a normalised form only when all readings agree, passing genuine ambiguity to the parser instead of guessing.
• I4 (learn): confidence for repair rules or UD dialect conversions could be stored for each specific pattern, so that a weak instance can be dropped without discarding the whole rule.