📄 Automated Question Generation for Science Tests in Arabic Language Using NLP Techniques

Summary

The paper addresses automatic creation of assessment questions for Arabic science material. Earlier Arabic systems relied on rule-based transformation of declarative sentences and suffered from parsing errors, NER mistakes, and long, complex sentences. Most neural question generators also need a gold answer span alongside the context. The authors avoid that requirement with a three-stage pipeline: keyword and key-phrase extraction, question generation, and ranking. KeyBert picks candidate phrases by comparing n-gram embeddings with the whole passage, and part-of-speech tagging filters out off-topic or malformed candidates. Each surviving phrase, together with its passage, is fed to the pretrained AraT5 model, which writes a question targeting it. The data are 111 passages taken manually from Palestinian grade 11–12 biology textbooks. GPT-4 Turbo produced reference questions for the same passages, and the system's output scored 83.50% precision, 78.68% recall and 80.95% F1 against them. Expert raters gave an average of 4.2/5 (84%). The work shows that a small, answer-free pipeline can produce usable Arabic test questions with modest resources.

Key points

• The pipeline has three stages: key-phrase extraction (KeyBert), question generation (AraT5), and ranking of the generated questions.
• KeyBert's candidates come from plain n-grams, so the authors add PoS tagging to keep phrases on topic.
• The corpus contains 111 entries drawn from the biology textbooks that Palestinian students use in the eleventh and twelfth grades. Its most frequent words concern blood, cells, DNA and neurons.
• Questions from OpenAI GPT-4 Turbo served as the benchmark. Against it the system scored P 83.50%, R 78.68% and F1 80.95%.
• Field experts rated the questions 4.2 out of 5 on average (84%).

Interesting ideas

• I1. The extracted key phrase serves as the answer the question must target, so no human-annotated answers are needed. — For Mova (back): Mova could generate questions from its own text graph, taking an entity or state from the world as the answer and producing the question that asks for it.
• I2. Statistical phrase candidates are cleaned with part-of-speech patterns, which removes off-topic or ill-formed n-grams. — For Mova (world): instead of n-grams, Mova can take noun-phrase subtrees from the UD tree as candidates and rank them by how central their entity is in the world of the text.
• I3. A large LLM's output is used as a cheap reference set when no gold questions exist, with human ratings added as a check. — For Mova (learn): LLM teacher questions could become a silver benchmark for Mova's question generator, provided negative controls are added, since agreeing with one teacher model does not prove the questions are correct.
• I4. Content selection is treated as its own step, in which the most representative pieces of a passage are chosen before any question is written. — For Mova (answer): a school-course test builder could first select the key facts of a textbook passage, then quiz Mova's students only on those facts, making the tests focused and explainable.
• I5. Generation is split into separate steps (select, generate, rank), so each step can be inspected and blamed when output is bad. — For Mova (back): the same split would let each generated question be traced to the selected fact and the ranking reason, which fits Mova's explainability requirement.