Elamur Research⌕
☰
home / papers / 2011 / Arabic Text to Arabic Sign Language Translation System for the Deaf and Hearing-Impaired Community

Arabic Text to Arabic Sign Language Translation System for the Deaf and Hearing-Impaired Community

Paper · 2011 · Abdulaziz Almohimeed, Mike Wald and R.I. Damper

Summary

The paper presents a prototype that translates written Arabic into Arabic Sign Language (ArSL). The authors argue that earlier ArSL systems failed because their builders treated ArSL as a word-by-word mirror of Arabic or as fingerspelling, when it is an independent language with its own grammar. Three deaf native signers and an ArSL interpreter guided the design. ArSL has no writing system and few linguistic descriptions, so the team recorded a small video corpus in the domain of classroom instructions used in deaf schools. Because the corpus is small, they chose chunk-based example-based MT (EBMT) over rule-based or statistical methods. Arabic input is first diacritized with Google Tashkeel so the morphological analyser returns a single analysis. Fragments are then matched against pre-compiled chunks, and sign video clips are joined with short pauses. Under leave-one-out evaluation, the system reached 46.7% WER and 29.4% PER. Most errors came from signs that were absent from the corpus. The work shows a workable recipe for translating into a low-resource language that has no written form.

Key points

  • The corpus contains 203 signed sentences covering 710 distinct signs, all instructional language from deaf education.
  • The system runs in two phases: a one-time pre-compilation of chunks and their signs, then translation through matching, alignment and recombination. The output is a WMV video.
  • Untranslated words are looked up in the dictionary by surface word, then by stem, then by root. Larger and overlapping chunk matches take priority over isolated ones.
  • In the output, a one-second transition clip separates consecutive signs. The three native signers rated test sentences built this way 5/5 for clarity of meaning and 4/5 for fluency.
  • Leave-one-out cross validation gave an average WER of 46.7% and PER of 29.4%. The dominant error was a missing sign, so the authors see more data as the remedy.

Interesting ideas

  • I1. Look up unknown items through a cascade of surface form, then stem, then root, so a small lexicon still covers many inflected words.
  • I2. Remove ambiguity upstream before analysis: restoring diacritics leaves the analyser exactly one reading instead of many competing ones.
  • I3. Prefer the largest available matched fragment and let overlapping matches outrank isolated ones, so whole phrases are reused before falling back to single units.
  • I4. Test the output representation itself with native judges, scoring meaning and fluency separately, before evaluating the full translation pipeline.
  • I5. Report an order-sensitive metric (WER) next to an order-free one (PER), and trace errors back to gaps in coverage rather than to the method.

For Mova

How these ideas could serve Mova, our small open language model — its files, stages and tests:

  • I1 (tokens): Unseen English word forms could fall back from surface form to lemma to derivational base, with each step logged so the guess stays explainable.
  • I2 (tokens): Normalizing troublesome input (contractions, spelling variants) before tagging could shrink the candidate set handed to morphology and keep later choices traceable.
  • I3 (back): Phrase-level templates tied to graph fragments could be chosen biggest-first during text generation, with word-level realization only as a fallback.
  • I4 (back): The generator's surface-assembly scheme could be checked on gold meaning graphs first, so realization errors are kept apart from analysis errors.
  • I5 (learn): Evaluation could pair strict and order-insensitive scores and tag each failure as missing knowledge or a faulty step, which shows whether to collect data or fix rules.

Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.

 Institutions

🎓University of SouthamptonSouthampton · United Kingdom10 works · 2,495 citations

 Fields, topics and concepts

📚 Shelves 7The field of science this work belongs to (OpenAlex fields).Computer Science · 16,667 worksArtificial Intelligence · 16,194 worksArts and Humanities · 16,039 worksLanguage and Linguistics · 15,891 worksPsychology · 1,552 worksDevelopmental and Educational Psychology · 499 worksHuman-Computer Interaction · 141 works

🧭 Topics 2Research subjects and languages the work is about.Machine translation · 1,087 worksSemitic languages · 2,908 works

Cite

Cite as Abdulaziz Almohimeed, Mike Wald and R.I. Damper. 2011. Arabic Text to Arabic Sign Language Translation System for the Deaf and Hearing-Impaired Community. Proceedings of the Second Workshop on Speech and Language Processing for Assistive Technologies.

The paper itself, not this page. BibTeX from the library's record of the work.

@inproceedings{almohimeed2011arabic,
  title = {{Arabic Text to Arabic Sign Language Translation System for the Deaf and Hearing-Impaired Community}},
  author = {Abdulaziz Almohimeed, Mike Wald and R.I. Damper},
  booktitle = {Proceedings of the Second Workshop on Speech and Language Processing for Assistive Technologies},
  year = {2011},
  url = {https://aclanthology.org/W11-2311/}
}

References

22 references, 1 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

  1. Mahmoud Abdel-Fateh. 2004. Arabic Sign Language: A perspective. Journal of Deaf Studeis and Deaf Education, 10(2):212–221. find
  2. Hend S. Al-Khalifa. 2010. Introducing Arabic sign language for mobile phones. In ICCHP’10 Proceedings of the 12th International Conference on Computers Helping People with Special Needs, pages 213–220 in Springer Lecture Notes in Computer Science, Part II, vol. 6180, Linz, Austria. find
  3. Muhammad Al Khuli. 1982. A Dictionary of Theoretical Linguistics: English-Arabic with an Arabic-English Glossary. Library of Lebanon, Beirut, Lebanon. find
  4. Imad Al Sughaiyer and Ibrahim Al Kharashi. 2004. Arabic morphological analysis techniques: A comprehensive survey. Journal of the American Society for Information Science and Technology, 55(3):189–213. ↗
  5. Abdulaziz Almohimeed, Mike Wald, and R. I. Damper. 2009. A new evaluation approach for sign language machine translation. In Assistive Technology from Adapted Equipment to Inclusive Environments, AAATE 2009, Volume 25, pages 498–502, Florence, Italy. ↗
  6. Abdulaziz Almohimeed, Mike Wald, and Robert Damper. 2010. An Arabic Sign Language corpus for instructional language in school. In Proceedings of the Seventh International Conference on Language Resources and Evaluation, LREC, pages 81–91, Valetta, Malta. find
  7. Abeer Alnafjan. 2008. Tawasoul. Master’s thesis, Department of Computer Science, King Saud University, Riyadh, Saudi Arabia. find
  8. Stan Augarten. 1984. Bit by Bit: An Illustrated History of Computers. Tickner and Fields, New York, NY. find
  9. Hennie Brugman and Albert Russel. 2004. Annotating multimedia/multi-modal resources with ELAN. In Proceedings of the Fourth International Conference on Language Resources and Evaluation, LREC, pages 2065–2068, Lisbon, Portugal. find
  10. Tim Buckwalter. 2004. Issues in arabic orthography and morphology analysis. In Proceedings of the Workshop on Computational Approaches to Arabic Script-based Languages, CAASL, pages 31–34, Geneva, Switzerland.
  11. Metri George. 1990. Al Khaleel: A Dictionary of Arabic Syntax Terms. Library of Lebanon, Beirut, Lebanon. find
  12. Sami M. Halawani. 2008. Arabic Sign Language translation system on mobile devices. IJCSNS International Journal of Computer Science and Network Security, 8(1):251–256. find
  13. Vladimir I. Levenshtein. 1966. Binary codes capable of correcting deletions, insertions, and reversals. Soviet Physics Doklady, 10(8):707–710. find
  14. Mohamed Mohandes. 2006. Automatic translation of Arabic text to Arabic Sign Language. ICGST International Journal on Artificial Intelligence and Machine Learning, 6(4):15–19. find
  15. Mohanned Momani and Jamil Faraj. 2007. A novel algorithm to extract tri-literal arabic roots. In Proceedings ACS/IEEE International Conference on Computer Systems and Applications, pages 309–315, Amman, Jordan. find
  16. Sara Morrissey. 2008. Data-Driven Machine Translation for Sign Languages. Ph.D. thesis, Dublin City University, Dublin, Ireland. find
  17. Franz Josef Och and Hermann Ney. 2003. A systematic comparison of various statistical alignment models. Computational Linguistics, 29(1):19–51. ↗
  18. Sinja Ojala. 2011. Studies on individuality in speech and sign. Technical Report No. 135, TUCS Dissertations, Turku Centre for Computer Science, University of Turku, Finland. find
  19. Nasredine Semmar, Faı̈za Elkateb-Gara, and Christian Fluhr. 2005. Using a stemmer in a natural language processing system to treat Arabic for cross-language information retrieval. In Proceedings of the Fifth Conference on Language Engineering, pages 1–10, Cairo, Egypt. find
  20. Tony Veale and Andy Way. 1997. Gaijin: A bootstrapping approach to example-based machine translation. In Proceedings of the Second Conference on Recent Advances in Natural Language Processing, RANLP, pages 27–34, Tzigov Chark, Bulgaria. find
  21. Andy Way and Nano Gough. 2003. wEBMT: developing and validating an example-based machine translation system using the world wide web. Computational Linguistics, 29(3):421–457. ↗
  22. Andy Way and Nano Gough. 2005. Comparing example-based and statistical machine translation. Natural Language Engineering, 11(3):295–309. ↗

Updated

↑ back to top

This page in Markdown → https://md.research.elamur.ai/free/en/papers/almohimeed-2011-arabic-text-arabic-sign-language/