Elamur Research⌕
☰
home / papers / 2013 / Reranking with Linguistic and Semantic Features for Arabic Optical Character Recognition

Reranking with Linguistic and Semantic Features for Arabic Optical Character Recognition

Paper · 2013 · Nadi Tomeh, Nizar Habash, Ryan Roth et al.

Summary

Arabic OCR is hard because of connected letter shapes, diacritics and rich morphology. The BBN Byblos system recognises characters with an HMM and relies on an n-gram LM for fluency. The authors keep the decoder unchanged and instead rescore its n-best output. Each hypothesis receives features drawn from five families:

  • the original HMM/LM scores
  • simple surface cues
  • word n-gram LMs
  • POS- and lemma-based LMs built with MADA
  • a new semantic coherence score based on word-pair relatedness

They compare ten learning-to-rank methods (pointwise, pairwise and listwise) and stack the best ones under a RankSVM meta-ranker. Experiments cover typewritten ("print") and handwritten ("hand") Arabic. Reranking lowers WER by 10.1% relative on print and 11.4% on hand, which closes roughly a third of the gap to the n-best oracle. Pairwise rankers do best. The study shows that cheap linguistic and semantic signals can improve a black-box recogniser without changing the decoder.

Key points

  • Baseline vs. oracle vs. best WER: print 13.8% / 9.8% / 12.4%; hand 35% / 20.9% / 30.9%. This removes about 35% (print) and 29% (hand) of the distance to the oracle.

  • The rankers tested were:

    • pointwise: MART;
    • pairwise: RankSVM, SGDSVM, PegasosSVM, PAPSVM, RankBoost, RankMIRA;
    • listwise: AdaRank, Coordinate Ascent, ListNet.

    The pointwise model performed worse than the baseline, and pairwise methods generally beat listwise ones.

  • LMs were trained on 220M words of Arabic Gigaword 3 plus 2.4M words of ground truth for each of print and hand, using SRILM with orders 1–5. LM-MADA applies the same models to POS and lemma sequences produced by MADA 3.0.

  • Results by feature family:

    • On print, LM-word and LM-MADA were the most useful.
    • On hand, SemCoh helped 6 of the 10 rankers.
    • Merging all feature families together usually did not beat the best single family.

  • The stacked ensemble reached the best hand result (30.9%) but gave no gain on print (12.4%). The data comes from the LDC-collected Anfal set.

Interesting ideas

  • I1. A word's confidence is the share of n-best hypotheses that contain it, and a hypothesis is scored by its average word confidence; agreement inside one system's own candidate list becomes a free reliability signal.
  • I2. Semantic coherence is computed as Lin-style Information Content over word pairs at directional distances up to 3. This sits between a noisy bag-of-words model and a sparse position-exact n-gram model.
  • I3. The authors leave the decoder untouched and add global, arbitrary features only at an n-best reranking stage; features too complex for the search stay out of the search.
  • I4. Each feature family is added to the base features separately, and the authors report that combining everything often hurts. Gains are also expressed as a fraction of the oracle gap rather than only as raw error.
  • I5. Stacking treats every trained ranker's score as one feature for a held-out meta-ranker. This helped on the noisier handwritten data but not on clean print.

For Mova

How these ideas could serve Mova, our small open language model — its files, stages and tests:

  • I1 (learn): when witness parsers disagree, measure how often each arc appears across their outputs and k-best lists, then use that consensus share to weight or filter silver training trees.
  • I2 (world): the same IC measure could be estimated over UD relations rather than surface distance, giving an explainable plausibility score for which entity–predicate links a text world should prefer.
  • I3 (tree): gparse or the perceptron parser could emit k-best trees, then a pairwise reranker could apply whole-tree features such as frame validity and VerbNet role fit.
  • I4 (learn): evaluations could report both the per-family ablation and the share of the k-best oracle gap closed, so that feature bloat and headroom stay visible.
  • I5 (learn): combining witness parsers through a learned meta-scorer may pay off mainly on hard, noisy domains, so its value should be checked per domain rather than assumed.

Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.

 Institutions

🎓Columbia UniversityNew York · United States241 works · 54,053 citations🎓George Washington UniversityWashington · United States51 works · 8,148 citations

 Fields, topics and concepts

📚 Shelves 7The field of science this work belongs to (OpenAlex fields).Computer Science · 16,667 worksArtificial Intelligence · 16,194 worksArts and Humanities · 16,039 worksLanguage and Linguistics · 15,891 worksComputer Vision and Pattern Recognition · 2,869 worksEngineering · 133 worksMedia Technology · 34 works

🧭 Topics 2Research subjects and languages the work is about.Semitic languages · 2,908 worksInformation retrieval & search · 473 works

Cite

Cite as Nadi Tomeh, Nizar Habash, Ryan Roth, Noura Farra, Pradeep Dasigi and Mona Diab. 2013. Reranking with Linguistic and Semantic Features for Arabic Optical Character Recognition. Proceedings of the 51st Annual Meeting of the Association for Computational Linguistics (Volume 2: Short Papers).

The paper itself, not this page. BibTeX from the library's record of the work.

@inproceedings{tomeh2013reranking,
  title = {{Reranking with Linguistic and Semantic Features for Arabic Optical Character Recognition}},
  author = {Nadi Tomeh, Nizar Habash, Ryan Roth, Noura Farra, Pradeep Dasigi and Mona Diab},
  booktitle = {Proceedings of the 51st Annual Meeting of the Association for Computational Linguistics (Volume 2: Short Papers)},
  year = {2013},
  url = {https://aclanthology.org/P13-2098/}
}

References

31 references, 4 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

  1. Yunbo Cao, Jun Xu, Tie-Yan Liu, Hang Li, Yalou Huang, and Hsiao-Wuen Hon. 2006. Adapting ranking SVM to document retrieval. In Proceedings of the 29th annual international ACM SIGIR conference on Research and development in information retrieval, SIGIR ’06, pages 186–193. find
  2. Zhe Cao, Tao Qin, Tie-Yan Liu, Ming-Feng Tsai, and Hang Li. 2007. Learning to rank: from pairwise approach to listwise approach. In Proceedings of the 24th international conference on Machine learning, ICML ’07, pages 129–136. find
  3. Philip K. Chan and Salvatore J. Stolfo. 1993. Experiments on multistrategy learning by meta-learning. In Proceedings of the second international conference on Information and knowledge management, CIKM ’93, pages 314–323. find
  4. Michael Collins and Terry Koo. 2005. Discriminative Reranking for Natural Language Parsing. Comput. Linguist., 31(1):25–70, March. ↗
  5. Jacob Devlin, Matin Kamali, Krishna Subramanian, Rohit Prasad, and Prem Natarajan. 2012. Statistical Machine Translation as a Language Model for Handwriting Recognition. In ICFHR, pages 291– 296. find
  6. Marco Dinarelli, Alessandro Moschitti, and Giuseppe Riccardi. 2012. Discriminative Reranking for Spoken Language Understanding. IEEE Transactions on Audio, Speech & Language Processing, 20(2):526–539. ↗
  7. Yoav Freund, Raj Iyer, Robert E. Schapire, and Yoram Singer. 2003. An efficient boosting algorithm for combining preferences. J. Mach. Learn. Res., 4:933–969, December. find
  8. Jerome H. Friedman. 2000. Greedy Function Approximation: A Gradient Boosting Machine. Annals of Statistics, 29:1189–1232. ↗
  9. Ruifang Ge and Raymond J. Mooney. 2006. Discriminative Reranking for Semantic Parsing. In ACL. find
  10. David Graff. 2007. Arabic Gigaword 3, LDC Catalog No.: LDC2003T40. Linguistic Data Consortium, University of Pennsylvania. find
  11. Nizar Habash and Owen Rambow. 2005. Arabic Tokenization, Part-of-Speech Tagging and Morphological Disambiguation in One Fell Swoop. In Proceedings of the 43rd Annual Meeting of the Association for Computational Linguistics (ACL’05), pages 573– 580, Ann Arbor, Michigan, June.
  12. Nizar Habash and Ryan M. Roth. 2011. Using deep morphology to improve automatic error detection in Arabic handwriting recognition. In Proceedings of the 49th Annual Meeting of the Association for Computational Linguistics: Human Language Technologies - Volume 1, HLT ’11, pages 875–884.
  13. Nizar Habash, Owen Rambow, and Ryan Roth. 2009. MADA+TOKAN: A toolkit for Arabic tokenization, diacritization, morphological disambiguation, POS tagging, stemming and lemmatization. In Khalid Choukri and Bente Maegaard, editors, Proceedings of the Second International Conference on Arabic Language Resources and Tools. The MEDAR Consortium, April. find
  14. Nizar Habash. 2010. Introduction to Arabic Natural Language Processing. Morgan & Claypool Publishers. ↗
  15. Thorsten Joachims. 2002. Optimizing search engines using clickthrough data. In Proceedings of the eighth ACM SIGKDD international conference on Knowledge discovery and data mining, KDD ’02, pages 133–142. find
  16. Peter Kolb. 2009. Experiments on the difference between semantic similarity and relatedness. In Proceedings of the 17th Nordic Conference of Computational Linguistics, NEALT Proceedings Series Vol. 4. find
  17. Joseph Le Roux, Benoit Favre, Alexis Nasr, and Seyed Abolghasem Mirroshandel. 2012. Generative Constituent Parsing and Discriminative Dependency Reranking: Experiments on English and French. In SP-SEM-MRL 2012. find
  18. Dekang Lin. 1997. Using syntactic dependency as local context to resolve word sense ambiguity. In Proceedings of the eighth conference on European chapter of the Association for Computational Linguistics, EACL ’97, Stroudsburg, PA, USA. Association for Computational Linguistics. find
  19. Dekang Lin. 1998. Automatic retrieval and clustering of similar words. In Proceedings of the 36th Annual Meeting of the Association for Computational Linguistics and 17th International Conference on Computational Linguistics - Volume 2, ACL ’98.
  20. Tie-Yan Liu. 2009. Learning to Rank for Information Retrieval. Now Publishers Inc., Hanover, MA, USA. ↗
  21. Volker Märgner and Haikal El Abed. 2009. Arabic Word and Text Recognition - Current Developments. In Khalid Choukri and Bente Maegaard, editors, Proceedings of the Second International Conference on Arabic Language Resources and Tools, Cairo, Egypt, April. The MEDAR Consortium. find
  22. Donald Metzler and W. Bruce Croft. 2007. Linear feature-based models for information retrieval. Inf. Retr., 10(3):257–274, June. ↗
  23. Premkumar Natajan, Zhidong Lu, Richard Schwartz, Issam Bazzi, and John Makhoul. 2002. Hidden Markov models. chapter Multilingual machine printed OCR, pages 43–63. World Scientific Publishing Co., Inc., River Edge, NJ, USA. find
  24. Rohit Prasad, Shirin Saleem, Matin Kamali, Ralf Meermeier, and Premkumar Natarajan. 2008. Improvements in hidden Markov model based Arabic OCR. In Proceedings of International Conference on Pattern Recognition (ICPR), pages 1–4. find
  25. Shirin Saleem, Huaigu Cao, Krishna Subramanian, Matin Kamali, Rohit Prasad, and Prem Natarajan. 2009. Improvements in BBN’s HMM-Based Offline Arabic Handwriting Recognition System. In Proceedings of the 2009 10th International Conference on Document Analysis and Recognition, IC-DAR ’09, pages 773–777. find
  26. D. Sculley. 2009. Large scale learning to rank. In NIPS 2009 Workshop on Advances in Ranking. find
  27. Libin Shen, Anoop Sarkar, and Franz Josef Och. 2004. Discriminative Reranking for Machine Translation. In Daniel Marcu Susan Dumais and Salim Roukos, editors, HLT-NAACL 2004: Main Proceedings, pages 177–184, Boston, Massachusetts, USA, May. find
  28. Andreas Stolcke. 2002. SRILM - an Extensible Language Modeling Toolkit. In Proceedings of the International Conference on Spoken Language Processing (ICSLP), volume 2, pages 901–904, Denver, CO. find
  29. Buzhou Tang, Qingcai Chen, Xuan Wang, and Xiaolong Wang. 2010. Reranking for stacking ensemble learning. In Proceedings of the 17th international conference on Neural information processing: theory and algorithms - Volume Part I, ICONIP’10, pages 575–584. find
  30. David H. Wolpert. 1992. Original Contribution: Stacked generalization. Neural Netw., 5(2):241– 259, February. find
  31. Jun Xu and Hang Li. 2007. AdaRank: a boosting algorithm for information retrieval. In Proceedings of the 30th annual international ACM SIGIR conference on Research and development in information retrieval, SIGIR ’07, pages 391–398. find

Updated

↑ back to top

This page in Markdown → https://md.research.elamur.ai/free/en/papers/tomeh-2013-reranking-linguistic-semantic-features-arabic/