Elamur Research⌕
☰
home / papers / 2022 / ALRT: Cutting Edge Tool for Automatic Generation of Arabic Lexical Recognition Tests

ALRT: Cutting Edge Tool for Automatic Generation of Arabic Lexical Recognition Tests

Paper · 2022 · Osama Hamed, Saeed Salah and Abed Alhakim Freihat

Summary

The paper tackles the lack of quick, automated vocabulary-size placement tests for Arabic. Lexical Recognition Tests (LRTs) ask a learner to decide whether each item is a real word or an invented one. For Arabic, the invented items (nonwords) have so far been crafted by hand or semi-automatically. The authors present ALRT (V1.0), a tool that produces Arabic LRTs automatically.

The tool works from a large collection of freely available Arabic text drawn from books, social media and news. This text was cleaned and reduced to one vocabulary item per line in UTF-8. To make a nonword, ALRT swaps letters in a real word for letters that look or sound alike, using orthographic similarity rules and a phonological similarity map. Character n-gram statistics limit which swaps are allowed, and any candidate that turns out to be a real corpus word is discarded. Word-frequency thresholds sort items into difficulty levels, so tests can be graded.

A subject-matter expert reviewed the generated nonwords as an extra quality check. The test engine is released on GitHub. Possible uses include placement testing for Arabic learners and support for spellcheckers.

Key points

  • ALRT V1.0 extends the authors' earlier framework (Salah et al., 2022) for generating Arabic LRTs automatically. The test engine is published at github.com/ohamed/ar-lrts.
  • Nonword candidates come from two lists that are then merged:
    • one built with orthographic similarity rules;
    • one built with a phonological similarity map.
  • A candidate is dropped if it occurs in the processed corpus, which ensures it is not a real word.
  • Two frequency thresholds (Threshold1, Threshold2) select words in a target frequency band, allowing multi-level tests.
  • For every word, the tool stores all character n-grams from bigrams up to length−1 grams. The frequencies of preceding and following characters then guide which similar letter is substituted.
  • Planned work includes:
    • integrating the ALP lemmatizer;
    • adding tokenization, part-of-speech and diacritization recognition tests.

Interesting ideas

  • I1. Build distractors by swapping in visually or phonetically confusable letters, then reject anything found in a corpus; the result is items that look like real words but are guaranteed not to be.
  • I2. Use character n-gram frequencies around the edited position to keep only substitutions that yield plausible letter sequences, rather than random edits.
  • I3. Use corpus frequency bands as a cheap, explainable control for item difficulty.
  • I4. Treat automatic generation as the main path and add a light expert check only to validate the output.
  • I5. Generated nonwords can serve both as test items and as reference material for proofreading.

For Mova

How these ideas could serve Mova, our small open language model — its files, stages and tests:

  • I1 (learn): Such pseudo-words make clean negative controls, letting us check that the tagger and lemmatizer flag unknown forms instead of confidently assigning them a real lemma.
  • I2 (tokens): The same constraint could rank candidate corrections or guessed lemmas for out-of-vocabulary words, preferring forms whose letter sequences are common in English.
  • I3 (answer): School-course tests and word problems could be graded by the frequency band of their key vocabulary, separating reading-difficulty failures from reasoning failures.
  • I4 (learn): Silver data or generated test items could be sampled for brief human or LLM-teacher review, keeping the pipeline automatic while estimating its error rate.
  • I5 (tokens): A bank of near-miss nonwords could stress-test spelling normalization before parsing, checking that real typos are repaired and invented forms are not silently accepted.

Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.

 Fields, topics and concepts

📚 Shelves 2The field of science this work belongs to (OpenAlex fields).Computer Science · 16,667 worksArtificial Intelligence · 16,194 works

🧭 Topics 1Research subjects and languages the work is about.Semitic languages · 2,908 works

💡 Concepts 1Key ideas and methods explained on our concept pages.Type–token ratio · 14 works

Cite

Cite as Osama Hamed, Saeed Salah and Abed Alhakim Freihat. 2022. ALRT: Cutting Edge Tool for Automatic Generation of Arabic Lexical Recognition Tests. Proceedings of the Third International Workshop on NLP Solutions for Under Resourced Languages (NSURL 2022) co-located with ICNLSP 2022.

The paper itself, not this page. BibTeX from the library's record of the work.

@inproceedings{hamed2022alrt,
  title = {{ALRT: Cutting Edge Tool for Automatic Generation of Arabic Lexical Recognition Tests}},
  author = {Osama Hamed, Saeed Salah and Abed Alhakim Freihat},
  booktitle = {Proceedings of the Third International Workshop on NLP Solutions for Under Resourced Languages (NSURL 2022) co-located with ICNLSP 2022},
  year = {2022},
  url = {https://aclanthology.org/2022.nsurl-1.7/}
}

References

18 references, 1 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

  1. Ehsan Mohammed Abdelgadir and VSV Laxmi Ramana. 2017. A Handbook on “Introduction to Phonetics & Phonology”: For Arabic students. Notion Press. find
  2. David A Balota, Melvin J Yap, Keith A Hutchison, Michael J Cortese, Brett Kessler, Bjorn Loftis, James H Neely, Douglas L Nelson, Greg B Simpson, and Rebecca Treiman. 2007. The english lexicon project. Behavior research methods, 39(3):445–459. ↗
  3. Wouter Duyck, Timothy Desmet, Lieven PC Verbeke, and Marc Brysbaert. 2004. Wordgen: A tool for word selection and nonword generation in dutch, english, german, and french. Behavior Research Methods, Instruments, & Computers, 36(3):488–499. ↗
  4. Heba Elfardy and Mona Diab. 2012. Aida: Automatic identification and glossing of dialectal arabic. In Proceedings of the 16th eamt conference (project papers), pages 83–83.
  5. Nick C Ellis. 2002. Frequency effects in language processing: A review with implications for theories of implicit and explicit language acquisition. Studies in second language acquisition, 24(2):143–188. ↗
  6. A. A. Freihat, M. Abbas, G. Bella, and F. Giunchiglia. 2018a. Towards an optimal solution to lemmatization in arabic. In Proceedins of the 4th International Conference on Arabic Computational Linguistics (ACLing 2018), pages 1–9. find
  7. A. A. Freihat, G. Bella, H. Mubarak, and F. Giunchiglia. 2018b. A single-model approach for arabic segmentation, pos tagging, and named entity recognition. In 2018 2nd International Conference on Natural Language and Speech Processing (ICNLSP), pages 1–8. find
  8. Hicham Gueddah and Abdallah Yousfi. 2013. The impact of arabic inter-character proximity and similarity on spell-checking. In 2013 8th International Conference on Intelligent Systems: Theories and Applications (SITA), pages 1–4. IEEE. find
  9. Osama Hamed and Torsten Zesch. 2015. Generating nonwords for vocabulary proficiency testing. In Proceeding of the 7th Language and Technology Conference: Human Language Technologies as a Challenge for Computer Science and Linguistics, pages 473– 477. *Figure 3: The proposed block diagram of Arabic Lexical Recognition Test (LRT)* Osama Hamed and Torsten Zesch. 2017. The role of diacritics in designing lexical recognition tests for arabic. Procedia Computer Science, 117:119–128. ↗
  10. Osama Hamed and Torsten Zesch. 2018. The role of diacritics in adapting the difficulty of arabic lexical recognition tests. NLP for Computer Assisted Language Learning (NLP4CALL 2018), 23. find
  11. Osama Amin Hamed. 2019. Automatic generation of lexical recognition tests using natural language processing. Ph.D. thesis, Dissertation, Duisburg, Essen, Universität Duisburg-Essen, 2019. ↗
  12. Mohamed Osman Hegazi. 2016. An approach for arabic root generating and lexicon development. Int. J. Comp. Sci. Netw. Sec.(IJCSNS), 16(1):9. find
  13. Ineke Huibregtse, Wilfried Admiraal, and Paul Meara. 2002. Scores on a yes-no vocabulary test: Correction for guessing and response style. Language testing, 19(3):227–245. ↗
  14. Emmanuel Keuleers and Marc Brysbaert. 2010. Wuggy: A multilingual pseudoword generator. Behavior research methods, 42(3):627–633. ↗
  15. Kristin Lemhöfer and Mirjam Broersma. 2012. Introducing lextale: A quick and valid lexical test for advanced learners of english. Behavior research methods, 44(2):325–343. ↗
  16. Kathleen Rastle, Jonathan Harrington, and Max Coltheart. 2002. 358,534 nonwords: The arc nonword database. The Quarterly Journal of Experimental Psychology Section A, 55(4):1339–1362. ↗
  17. Robert Stephen Ricks. 2015. The development of frequency-based assessments of vocabulary breadth and depth for L2 Arabic. Georgetown University. find
  18. Saeed Salah, Mohammad Nassar, Raid Zaghal, and Osama Hamed. 2022. Towards the automatic generation of arabic lexical recognition tests using orthographic and phonological similarity maps. Journal of King Saud University-Computer and Information Sciences, 34(10):8429–8439. ↗

Updated

↑ back to top

This page in Markdown → https://md.research.elamur.ai/free/en/papers/hamed-2022-alrt-cutting-edge-tool-automatic/