Elamur Research⌕
☰
home / papers / 2006 / Building a Heterogeneous Information Retrieval Collection of Printed Arabic Documents

Building a Heterogeneous Information Retrieval Collection of Printed Arabic Documents

Paper · 2006 · Abdelrahim Abdelsapor, Noha Adly, Kareem Darwish et al.

Summary

The paper addresses a gap in Arabic search research. There was no large, realistic benchmark for retrieving Arabic text produced by OCR, which is often error-ridden. The authors drew on the Bibliotheca Alexandrina's part of the Million Book Project (75,000 Arabic books). They sampled about 25 pages from each of 1,378 books, giving 34,651 scanned pages, and ran these through Sakhr's Automatic Reader. The resulting pages vary in subject, printing era, typeface and how badly the OCR degraded them. The team wrote 25 topics and gathered relevance judgments with a custom interactive search-and-judge web tool.

The baseline experiments tested four index term types (whole words, lightly stemmed forms, and overlapping character sequences of length three and of length four), each run both with and without blind relevance feedback. Character 4-grams were the strongest. Blind feedback changed results only slightly and not significantly. At the time, this was the largest Arabic document-image collection to come with OCR output, topics and judgments. That makes it a realistic testbed for noisy-text retrieval, because it does not depend on simulated corruption.

Key points

  • Collection size: 34,651 page images from 1,378 books, with print dates from the early 1920s to the present. Topics span history, philosophy, culture and politics.
  • Scanning and OCR: Pages were scanned in black and white at 300×300 dpi on a Minolta PS 3000 and processed with Sakhr's Automatic Reader v6.
    • Character error rate per book ranged from 1% to 21%, estimated by manually checking one random page per book.
    • Typefaces were grouped into 12 common font classes plus a 13th class for rare fonts.
  • Topics: There are 25 topics, each with a short title (similar to a web query) and a natural-language description. Relevant documents per topic range from 4 to 95.
  • Judging tool: It ran on the Lemur toolkit (v3.1) with Okapi BM25 weighting and character 4-gram indexing. Judges could reformulate queries, apply optional automatic expansion, keep track of their judgments and view page images with query terms highlighted.
  • Results:
    • Without feedback, 4-grams beat the other index terms with statistical significance on every metric. The one exception was light stems under precision@10.
    • Blind relevance feedback did not significantly help any index term type.

Interesting ideas

  • I1. Using genuinely degraded inputs instead of corruption simulated by a confusion model captures failure modes that synthetic noise misses, such as page skew, odd fonts and damaged pages.
  • I2. Sampling a fixed small number of pages from many different sources spreads variation in genre, font and quality across the set, so no single source dominates.
  • I3. Quality was estimated cheaply by hand-checking one random page per book, giving a per-source error estimate without full annotation.
  • I4. Character n-grams survived OCR damage better than words or stems, because a few broken characters still leave many intact n-grams to match.
  • I5. A judging tool that combined repeated searching, optional query expansion and tracked decisions made relevance labeling feasible without pooling many systems.

For Mova

How these ideas could serve Mova, our small open language model — its files, stages and tests:

  • I1 (learn): Build part of the evaluation set from real messy English text, such as OCR'd scans or forum posts, so parser and reader robustness is measured on actual noise rather than on noise we generated ourselves.
  • I2 (learn): When building silver or test sets, cap the sentences taken from each document or source so treebank-style evaluations are not skewed by a few long texts.
  • I3 (learn): Spot-check one random parsed sentence per source to estimate a per-source error rate, then use that rate to weight or filter silver data.
  • I4 (tokens): Add character n-gram features as a fallback in the tagger or lemmatizer for misspelled or out-of-vocabulary tokens, so a single corrupted letter does not wreck the analysis.
  • I5 (learn): A similar interactive tool for checking frames or answers could let a human or LLM teacher iterate quickly while every accept or reject decision is logged as explainable training signal.

Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.

 Institutions

🎓Cairo UniversityGiza · Egypt12 works · 730 citations🔬Bibliotheca AlexandrinaAlexandria · Egypt2 works · 8 citations🏢IBM (Egypt)Giza · Egypt2 works · 3 citations

📚 Shelves 4The field of science this work belongs to (OpenAlex fields).Computer Science · 9,086 worksArtificial Intelligence · 8,889 worksComputer Vision and Pattern Recognition · 1,282 worksInformation Systems · 523 works

🧭 Topics 2Research subjects and languages the work is about.Semitic languages · 1,484 worksInformation retrieval & search · 145 works

Cite

Cite as Abdelrahim Abdelsapor, Noha Adly, Kareem Darwish, Ossama Emam, Walid Magdy and Magdi Nagi. 2006. Building a Heterogeneous Information Retrieval Collection of Printed Arabic Documents. Proceedings of the Fifth International Conference on Language Resources and Evaluation (LREC'06).

The paper itself, not this page. BibTeX from the library's record of the work.

@inproceedings{abdelsapor2006building,
  title = {{Building a Heterogeneous Information Retrieval Collection of Printed Arabic Documents}},
  author = {Abdelrahim Abdelsapor, Noha Adly, Kareem Darwish, Ossama Emam, Walid Magdy and Magdi Nagi},
  booktitle = {Proceedings of the Fifth International Conference on Language Resources and Evaluation (LREC'06)},
  year = {2006},
  doi = {10.63317/4tn9y2k2se88},
  url = {https://doi.org/10.63317/4tn9y2k2se88}
}

References

27 references, 0 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

  1. Ahmed, M. A large-scale computational processor of the Arabic morphology, and Applications. Master’s Thesis, Faculty of Engineering, Cairo University, Cairo, Egypt, 2000. find
  2. Aljlayl, M., S. Beitzel, E. Jensen, A. Chowdhury, D. Holmes, M. Lee, D. Grossman, and O. Frieder. IIT at TREC-10. TREC-2001, 2001. find
  3. Al-Kharashi, I. and M. Evens. Comparing words, stems, and roots as index terms in an Arabic information retrieval system. JASIS. 45 (8): 548-560, 1994. ↗
  4. Baird, H. Document image defects models and their uses. Proceedings of the Second International Conference on Document Analysis and Recognition (ICDAR), 62-67, 1993. find
  5. Baird, H. State of the art of document image degradation modelling. Proceedings of the 4th IAPR Workshop on Document Analysis Systems (DAS 2000), 2000. find
  6. Chen, A. and F. Gey. Translation term weighting and combining translation resources in cross-language retrieval. TREC-2001, 2001. find
  7. Cormack, G., C. Palmer, and C. Clarke. Efficient Construction of Large Test Collections. In the Proceedings of the 21st ACM SIGIR Conference, 282-289, 1998. find
  8. Darwish, K. Building a shallow morphological analyzer in one day. ACL 2002 Workshop on Computational Approaches to Semitic Languages, July 11, 2002. find
  9. Darwish, K. and O. Emam. The Effect of Blind Relevance Feedback on a New Arabic OCR Degraded Text Collection. In International Conference on Machine Intelligence: Special Session on Arabic Document Image Analysis, 2005. find
  10. Darwish, K., D. Doermann, R. Jones, D. Oard, and M. Rautiainen. TREC-10 experiments at Maryland: CLIR and video. TREC-2001, 2001. find
  11. Darwish, K., D. Oard, Term selection for searching printed Arabic. In the Proceedings of the 25th ACM SIGIR Conference, page 261 - 268, 2002. find
  12. Darwish, K., Probabilistic Methods for Search OCR Degraded Arabic Text. Ph.D. dissertation, University of Maryland, College Park, 2003. find
  13. Harding, S., W. Croft, and C. Weir. Probabilistic retrieval of OCR degraded text using n-grams. European Conference on Digital Libraries, 1997 find
  14. Hmeidi, I., G. Kanaan, and M. Evens. Design and implementation of automatic indexing for information retrieval with Arabic documents. JASIS. 48 (10): 867-881, 1997. ↗
  15. Jones, K. S. and C. J. Van Rijsbergen. Report on the need for and previous of ‘ideal’ test collection, TR #365, University Computer Laboratory, Cambridge, 1975. find
  16. Kanungo, T. Document degradation models and methodology for degradation model validation. Ph.D. Thesis, Electrical Engineering Department, University of Washington, 1996. find
  17. Kanungo, T., G. Marton, and O. Bulbul. OmniPage vs. Sakhr: paired model evaluation of two Arabic OCR products. Proceedings of SPIE Conference on Document Recognition and Retrieval (VI), Vol. 3651, San Jose, California, Jan. 27-28, 1999. find
  18. Larkey, L., L. Ballesteros, and M. Connell. Improving stemming for Arabic information retrieval: light stemming and co-occurrence analysis. In SIGIR, 2002, Tampere, Finland. pp. 275-282 find
  19. Mayfield, J., P. McNamee, C. Costello, C. Piatko, and A. Banerjee. JHU/APL at TREC 2001: experiments in filtering and in Arabic, video, and web retrieval. TREC-2001, 2001. find
  20. Oard, D. and F. Gey. The TREC-2002 Arabic/English CLIR Track. TREC-2002, 2002. find
  21. Sanderson, M. and H. Joho. Forming Test Collection with no System Pooling. In the Proceedings of the 27 th ACM SIGIR Conference, page 33-40, 2004 find
  22. Singhal, A., G. Salton, and C. Buckley. Length normalization in degraded text collections. Proceedings of 5th Annual Symposium on Document Analysis and Information Retrieval, 149-162, April 15-17, 1996. find
  23. Taghva, K., J. Borasack, A. Condit, and J. Gilbreth. Results and implications of the noisy data projects. Technical Report 94-01, Information Science Research Institute, University of Nevada, Las Vegas, 1994. find
  24. Taghva, K., J. Borasack, A. Condit, and P. Inaparthy. Querying short OCR’d documents. Technical Report 94-10, Information Science Research Institute 1995. find
  25. Trenkle, J., A. Gillies, E. Erlandson, S. Schlosser, and S. Cavin. Advances in Arabic text recognition. Proceeding of Symposium on Document Image Understanding Technology, Columbia, Maryland, April 23-25, 2001. find
  26. Tseng, Y. and D. Oard. Document image retrieval techniques for Chinese. Proceeding of Symposium on Document Image Understanding Technology, Columbia, Maryland, April 23-25, 2001. find
  27. Xu, J., A. Fraser, and R. Weischedel. TREC 2001 cross-lingual retrieval at BBN. TREC-2001, 2001. find

Updated

↑ back to top

This page in Markdown → https://md.research.elamur.ai/free/en/papers/abdelsapor-2006-building-heterogeneous-information-retrieval-collection/