📄 ArabIcros: AI-Powered Arabic Crossword Puzzle Generation for Educational Applications

Summary

The paper tackles the lack of tools and data for building Arabic crossword puzzles for teaching. Writing good Arabic clues takes wordplay skill, and no earlier system produced Arabic crosswords automatically.

The authors built a clue-answer corpus from two Arabic periodicals. They transcribed pairs by hand or with human-supervised OCR, validated them, and released the cleaned set publicly.

The generator works in two ways:

• Path (a), from a passage: it uses few-shot prompting of GPT3.5-Turbo and GPT4 to pull keywords from an educational text, write clues for them, and then screen the results with a zero-shot check. Prompts were tested in both English and Arabic.
• Path (b), from a bare keyword: it uses fine-tuned GPT3 variants and GPT3.5-Turbo, with a learned classifier that rejects weak clues.

A layout module then assembles the chosen pairs into a grid.

Stage-level success rates are high, from about 89% to 97%, but the full text-to-pair pipeline succeeds about 79% of the time. The work matters as the first such system for Arabic and as a reusable dataset for educational NLP in that language.

Key points

• Dataset: 57,706 collected entries, which reduce to 25,908 unique pairs. Answers run from 1 to 21 characters, mostly 2–9. Sources are the Al-Joumhouria Journal (5,661 pairs, manual) and the Al-Ghad Electronic Journal (OCR-assisted). It is published on Hugging Face as Kamyar-zeinalipour/AR_CW.
• Preprocessing: diacritics, duplicate pairs and Arabic-specific "reversal" markers were stripped. Clue types observed include synonym/antonym definitions, general knowledge, scrambled letters, and taking part of a given word.
• Path (a) results:
  • Keyword extraction: 97.38%, best with GPT3.5-Turbo and Arabic prompts.
  • Clue writing: 94.62%, best with GPT4 and English prompts.
  • Validation: 89.32%, best with GPT3.5-Turbo and Arabic prompts.
  • End-to-end: 78.95%, best with GPT4 and English prompts.
• Path (b) results: GPT3-DaVinci, GPT3-Curie and GPT3.5-Turbo were fine-tuned and tested on 2,000 educational keywords. Fine-tuned GPT3.5-Turbo was best, yielding acceptable clues 81% of the time.
• Acceptability classifiers: GPT3-Davinci led with 85.74% accuracy (F1 0.8431). The others scored as follows: Curie 81.29%, Ada 79.19%, Babbage 78.69%, and Bert-base-Arabic 71.42% (F1 0.6896).

Interesting ideas

• I1. The authors reused the human verdicts collected while evaluating the fine-tuned generators as labels for a separate accept/reject classifier, so evaluation effort doubles as training data.
• I2. They noticed the LLM sometimes wrote clues from its own background knowledge instead of the supplied passage, and they added a dedicated check that drops clues not grounded in the text.
• I3. Prompt language (English vs Arabic) was treated as a tuning variable for each sub-step, and the best choice differed by stage and model.
• I4. They scored each pipeline stage separately and also reported an end-to-end figure, which reveals how errors compound: high stage scores still produced a much lower overall rate.
• I5. Before costly LLM validation, a trivial structural rule (discard answers longer than three words) removes obviously unusable candidates.

For Mova

How these ideas could serve Mova, our small open language model — its files, stages and tests:

• I1 (learn): Judgments gathered when auditing silver parses or teacher outputs could be kept and used to train a cheap filter that pre-screens future silver data.
• I2 (answer): Readers answering questions on fables could use a similar test that every answer traces back to the text's world, not to outside knowledge, as a guard against "knowing" rather than reading.
• I3 (learn): LLM teacher tiers could choose prompt format or model separately per task (e.g., role labelling vs coreference) instead of using one global setup.
• I4 (answer): Reporting per-step accuracy (tree, frames, world) alongside final test scores would show which explainable step is losing the most answers.
• I5 (back): Generated text or quiz items could first pass cheap form constraints, such as length and single-token answers, before any expensive semantic check.