📄 An Automated System for Essay Scoring of Online Exams in Arabic based on Stemming Techniques and Levenshtein Edit Operations

Summary

The paper addresses automatic grading of free-text (essay or short-answer) questions in Arabic online exams. Earlier Arabic systems still needed an instructor to grade open answers by hand. The author builds a lightweight, fully automatic pipeline that compares a student's answer with a stored model answer. Both texts are cleaned: digits, diacritics and non-Arabic letters are removed. Stop words and the article "AL" with its attached variants are dropped, letter variants are normalised, and affixes are stripped. This last step comes in two variants: heavy (root-oriented) stemming and light stemming, which only removes affixes. Each remaining word of the model answer gets an equal share of the mark. Student words are matched against reference words using Levenshtein distance normalised by the longer word's length, and credit is full, half or zero depending on similarity thresholds. The system was deployed as a web service and illustrated on two worked examples, where both stemmers produced the same score. No quantitative comparison against human graders is reported. The work matters mainly as a simple, transparent baseline for a low-resource setting.

Key points

• Word similarity is defined as 1 minus edit distance divided by the length of the longer word. Each reference word carries a weight of 1 divided by the number of words in the model answer.
• Per-word credit depends on similarity:
  • full weight at a similarity of 0.96 or higher;
  • half weight between 0.80 and 0.96;
  • nothing below 0.80. The instructor can change all of these thresholds.
• The total mark is also banded:
  • 1 or at least 0.96 counts as correct;
  • 0.75 to 0.96 is flagged as "might be correct, must be checked";
  • below 0.75 counts as wrong.
• Preprocessing has several fixed parts:
  • stop-word and "AL"-derivative lists are stored in the database;
  • alef variants are normalised to bare alef, and ta marbuta to ha;
  • prefixes and suffixes are stripped only when a word is longer than 3 letters;
  • light stemming uses its own list of 10 suffixes.
• In the two examples the scores were 0.8 and 0.75, so both answers went to manual checking. Extra words in the student answer were simply ignored. Future work targets maths exams that include numbers and Latin symbols.

Interesting ideas

• I1. Turning Levenshtein distance into a length-normalised similarity and mapping it onto discrete credit tiers (full, half, none) makes fuzzy matching both forgiving of typos and easy to explain.
• I2. Affix stripping is skipped for words of 3 letters or fewer, a cheap guard against destroying short words through over-stemming.
• I3. Scores are split into confident-correct, confident-wrong and an explicit uncertain middle band sent to a human, instead of forcing every case into a binary verdict.
• I4. Scoring is anchored on the reference answer: weights come from its words, and extra student words are dropped. This means padding an answer with extra words is never penalised.
• I5. The paper keeps two stemming depths (root-based and light) as interchangeable modules and runs both on the same answers to see whether they agree.

For Mova

How these ideas could serve Mova, our small open language model — its files, stages and tests:

• I1 (answer): School-course test readers could grade short free-text answers word by word with partial credit for near-miss spellings, and report exactly which reference word was missed.
• I2 (tokens): A similar minimum-length guard in the en lemmatiser's fallback suffix rules could stop unknown short words from being mangled.
• I3 (learn): Silver-data agreement scores could use the same three-way split, with only the uncertain band escalated to a higher LLM teacher tier.
• I4 (learn): Evaluation suites could add negative controls, such as answers padded with many candidate words, to check that answer scoring cannot be gamed this way.
• I5 (tokens): Running both a full lemma and a coarser stem match and checking whether they agree could serve as a cheap consistency signal for morphology errors.