Elamur Research⌕
☰
home / papers / 2011 / Using Deep Morphology to Improve Automatic Error Detection in Arabic Handwriting Recognition

Using Deep Morphology to Improve Automatic Error Detection in Arabic Handwriting Recognition

Paper · 2011 · Nizar Habash and Ryan Roth

Summary

Habash and Roth address a sub-problem of Arabic handwriting recognition (HR). Instead of fixing recognition mistakes, they only try to locate them. Arabic is hard for HR because letters connect and diacritic dots are small or ambiguous. Its rich morphology also makes word-list lookup unreliable. The authors ran the BBN Byblos HR system on LDC handwriting scans to produce N-best hypothesis lists. They then built a problem zone detection (PZD) system that labels each hypothesis word as PROB or OK. Gold labels come from aligning hypotheses with references. Inserted and substituted words count as problems, and a deleted word marks its neighbours. The detector uses non-linguistic features alongside morphological and syntactic ones, such as lemmas and part-of-speech tags. The best configuration improves F-score by about 15 points absolute over a simple baseline. Error analysis shows that lemma models help most on words that make the sentence semantically incoherent. The work matters because a later correction step, or a human post-editor, can concentrate effort on flagged words. It also shows that deep morphology helps even with a very large language model.

Key points

  • Data: 600 dpi LDC scans of newswire, weblog and newsgroup text. BBN Byblos produced N-best lists of up to 300 hypotheses per segment, with segments averaging 6.87 words. Training sets used 2000 or 4000 segments, each with its top 10 hypotheses plus the reference.
  • In the dev set, 25.8% of words are problematic. Of these, 81.4% are substitutions, 10.6% insertions and 7.9% deletions.
  • Breakdown of substitutions:
    • 26.5% are simple orthographic variants (Alef Hamza forms, Ya/Alef-Maqsura).
    • Another 6.9% share a lemma with the correct word.
    • 59.7% are complex errors.
    • Only 2.4% are pure dot misplacement.
  • Combining systems by simple voting ("any two agree") reached P/R/F of 62.8/72.4/67.3, about 1 point above the best single system. A three-system combination gave 70.2/64.0/66.9, trading recall for precision.
  • Finer tag sets were compared by error reduction over each set's trivial baseline:
    • PROB/OK: 36.3%
    • SUB/INS/DEL/OK: 34.8%
    • SUB/INS/OK: 40.1%
    • SUB/OK: 38.9%

Interesting ideas

  • I1. Treating error detection as a separate task from error correction lets a cheap detector steer an expensive fixer toward a few suspicious spots and away from correct words.
  • I2. A missing word cannot be tagged, so its neighbours are tagged instead; the "zone" makes an invisible deletion detectable through its context.
  • I3. Lemma-level models catch errors that are well-formed words but semantically out of place, which surface-form models miss in morphologically rich text.
  • I4. When tag sets of different granularity are compared, each is scored by error reduction over its own trivial baseline rather than by raw accuracy.
  • I5. An agreement threshold among several systems works as a dial: requiring more agreement favours precision, requiring less favours recall.

For Mova

How these ideas could serve Mova, our small open language model — its files, stages and tests:

  • I1 (learn): self-induced repair rules could fire only on tokens or arcs a detector has flagged, reducing damage to analyses that were already right.
  • I2 (tree): when a parse suggests an elided or dropped element, marking the adjacent tokens as a problem zone gives later stages a concrete place to look.
  • I3 (tokens): a lemma-sequence plausibility score could flag noisy or mistokenized input words before they reach the parser.
  • I4 (learn): evaluations of label schemes (e.g., UD dialect variants, coarse vs. fine roles) should report gains over a matched trivial baseline, which also serves as a natural negative control.
  • I5 (learn): the witness-parser agreement threshold for silver data can be tuned per use case, stricter for training data and looser for flagging candidates for LLM-teacher review.

Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.

 Institutions

🎓Columbia UniversityNew York · United States241 works · 54,053 citations

 Fields, topics and concepts

📚 Shelves 7The field of science this work belongs to (OpenAlex fields).Computer Science · 16,667 worksArtificial Intelligence · 16,194 worksArts and Humanities · 16,039 worksLanguage and Linguistics · 15,891 worksComputer Vision and Pattern Recognition · 2,869 worksEngineering · 133 worksMedia Technology · 34 works

🧭 Topics 2Research subjects and languages the work is about.Semitic languages · 2,908 worksMorphology · 405 works

Cite

Cite as Nizar Habash and Ryan Roth. 2011. Using Deep Morphology to Improve Automatic Error Detection in Arabic Handwriting Recognition. Proceedings of the 49th Annual Meeting of the Association for Computational Linguistics: Human Language Technologies.

The paper itself, not this page. BibTeX from the library's record of the work.

@inproceedings{habash2011using,
  title = {{Using Deep Morphology to Improve Automatic Error Detection in Arabic Handwriting Recognition}},
  author = {Nizar Habash and Ryan Roth},
  booktitle = {Proceedings of the 49th Annual Meeting of the Association for Computational Linguistics: Human Language Technologies},
  year = {2011},
  url = {https://aclanthology.org/P11-1088/}
}

References

26 references, 4 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

  1. Fadi Biadsy, Jihad El-Sana, and Nizar Habash. 2006. Online Arabic handwriting recognition using Hidden Markov Models. In The 10th International Workshop on Frontiers in Handwriting Recognition (IWFHR’10), La Baule, France. ↗
  2. Kareem Darwish and Douglas W. Oard. 2002. Term Selection for Searching Printed Arabic. In SIGIR ’02: Proceedings of the 25th annual international ACM SI-GIR conference on Research and development in information retrieval, pages 261–268, New York, NY, USA. ACM. find
  3. Rickard Domeij, Joachim Hollman, and Viggo Kann. 1994. Detection of spelling errors in Swedish not using a word list en clair. J. Quantitative Linguistics, 1:1–195. ↗
  4. David Graff. 2007. Arabic Gigaword 3, LDC Catalog No.: LDC2003T40. Linguistic Data Consortium, University of Pennsylvania. find
  5. Nizar Habash and Owen Rambow. 2005. Arabic Tokenization, Part-of-Speech Tagging and Morphological Disambiguation in One Fell Swoop. In Proceedings of the 43rd Annual Meeting of the Association for Computational Linguistics (ACL’05), pages 573–580, Ann Arbor, Michigan, June. Association for Computational Linguistics.
  6. Nizar Habash, Abdelhadi Soudi, and Tim Buckwalter. 2007. On Arabic Transliteration. In A. van den Bosch and A. Soudi, editors, Arabic Computational Morphology: Knowledge-based and Empirical Methods. Springer. ↗
  7. Nizar Habash, Owen Rambow, and Ryan Roth. 2010. MADA+TOKAN Manual. Technical Report CCLS-10-01, Center for Computational Learning Systems (CCLS), Columbia University. ↗
  8. Nizar Habash. 2010. Introduction to Arabic Natural Language Processing. Morgan & Claypool Publishers. ↗
  9. Mohamed Ben Halima and Adel M. Alimi. 2009. A multi-agent system for recognizing printed Arabic words. In Khalid Choukri and Bente Maegaard, editors, Proceedings of the Second International Conference on Arabic Language Resources and Tools, Cairo, Egypt, April. The MEDAR Consortium. find
  10. Daniel Jurafsky and James H. Martin. 2000. Speech and Language Processing. Prentice Hall, New Jersey, USA. find
  11. Okan Kolak and Philip Resnik. 2002. OCR error correction using a noisy channel model. In Proceedings of the second international conference on Human Language Technology Research. find
  12. Taku Kudo and Yuji Matsumoto. 2003. Fast Methods for Kernel-Based Text Analysis. In Proceedings of the 41st Annual Meeting of the Association for Computational Linguistics (ACL’03), pages 24–31, Sapporo, Japan, July. Association for Computational Linguistics.
  13. Karen Kukich. 1992. Techniques for Automatically Correcting Words in Text. ACM Computing Surveys, 24(4). ↗
  14. Leah S. Larkey, Lisa Ballesteros, and Margaret E. Connell, 2007. Arabic Computational Morphology: Knowledge-based and Empirical Methods, chapter Light Stemming for Arabic Information Retrieval. Springer Netherlands, Kluwer/Springer edition. find
  15. Zhidong Lu, Issam Bazzi, Andras Kornai, John Makhoul, Premkumar Natarajan, and Richard Schwartz. 1999. A Robust, Language-Independent OCR System. In the 27th AIPR Workshop: Advances in Computer Assisted Recognition, SPIE. find
  16. Walid Magdy and Kareem Darwish. 2006. Arabic OCR Error Correction Using Character Segment Correction, Language Modeling, and Shallow Morphology. In Proceedings of 2006 Conference on Empirical Methods in Natural Language Processing (EMNLP 2006), pages 408–414, Sydney, Austrailia.
  17. Volker Märgner and Haikal El Abed. 2009. Arabic Word and Text Recognition - Current Developments. In Khalid Choukri and Bente Maegaard, editors, Proceedings of the Second International Conference on Arabic Language Resources and Tools, Cairo, Egypt, April. The MEDAR Consortium. find
  18. Mohsen Moftah, Waleed Fakhr, Sherif Abdou, and Mohsen Rashwan. 2009. Stem-based Arabic language models experiments. In Khalid Choukri and Bente Maegaard, editors, Proceedings of the Second International Conference on Arabic Language Resources and Tools, Cairo, Egypt, April. The MEDAR Consortium. find
  19. Prem Natarajan, Shirin Saleem, Rohit Prasad, Ehry MacRostie, and Krishna Subramanian, 2008. Arabic and Chinese Handwriting Recognition, volume 4768 of Lecture Notes in Computer Science, pages 231–250. Springer, Berlin, Germany. find
  20. Kemal Oflazer. 1996. Error-tolerant finite-state recognition with applications to morphological analysis and spelling correction. Computational Linguistics, 22:73–90. ↗
  21. U. Pal, P. K. Kundu, and B. B. Chaudhuri. 2000. OCR error correction of an inflectional Indian language using morphological parsing. J. Information Sci. and Eng., 16:903–922. ↗
  22. Ryan Roth, Owen Rambow, Nizar Habash, Mona Diab, and Cynthia Rudin. 2008. Arabic Morphological Tagging, Diacritization, and Lemmatization Using Lexeme Models and Feature Ranking. In Proceedings of ACL-08: HLT, Short Papers, pages 117–120, Columbus, Ohio, June. Association for Computational Linguistics.
  23. Shirin Saleem, Huaigu Cao, Krishna Subramanian, Marin Kamali, Rohit Prasad, and Prem Natarajan. 2009. Improvements in BBN’s HMM-based Offline Handwriting Recognition System. In Khalid Choukri and Bente Maegaard, editors, 10th International Conference on Document Analysis and Recognition (ICDAR), Barcelona, Spain, July. find
  24. Toufik Sari and Mokhtar Sellami. 2002. MOrpho-LEXical analysis for correcting OCR-generated Arabic words (MOLEX). In The 8th International Workshop on Frontiers in Handwriting Recognition (IWFHR’02), Niagara-on-the-Lake, Canada. find
  25. Andreas Stolcke. 2002. SRILM - an Extensible Language Modeling Toolkit. In Proceedings of the International Conference on Spoken Language Processing (ICSLP), volume 2, pages 901–904, Denver, CO. find
  26. Stephanie Strassel. 2009. Linguistic Resources for Arabic Handwriting Recognition. In Khalid Choukri and Bente Maegaard, editors, Proceedings of the Second International Conference on Arabic Language Resources and Tools, Cairo, Egypt, April. The MEDAR Consortium. find

Updated

↑ back to top

This page in Markdown → https://md.research.elamur.ai/free/en/papers/habash-2011-deep-morphology-improve-automatic-error/