Elamur Research⌕
☰
home / papers / 2025 / Baseer: A Vision-Language Model for Arabic Document-to-Markdown OCR

Baseer: A Vision-Language Model for Arabic Document-to-Markdown OCR

Summary

Arabic page OCR remains difficult for current multimodal models. The script is cursive, letter shapes change with context, ligatures are common, diacritics carry meaning, text runs right to left, and fonts vary widely. The authors adapt Qwen2.5-VL-3B-Instruct into Baseer, a model that takes a page image and outputs Markdown, using HTML for tables and dedicated tags for watermarks, page numbers and embedded images. The training set has 500k image–text pairs. Of these, 300k are synthetic pages built from filtered Common Crawl Markdown and passed through an HTML→Word→PDF→image chain with randomized layouts and augmentations; the other 200k are real documents. Only the language decoder is fine-tuned, so the pretrained visual features stay intact. The authors also release Misraj-DocOCR, a 400-page expert-checked benchmark, and a corrected version of the KITAB-Bench PDF-to-Markdown split. On Misraj-DocOCR, Baseer reaches a WER of 0.25 and the best TEDS and MARS scores, ahead of Gemini-2.5-pro, Azure and GPT-5. The result shows that a small, targeted adaptation of a general model can outperform much larger systems on a hard script.

Key points

  • Baseer is based on Qwen2.5-VL-3B-Instruct with decoder-only fine-tuning. Its 500k training pairs comprise 150k clean synthetic pages, 150k augmented synthetic pages and 200k real-world pages.
  • The synthetic source text was filtered in two ways: a KenLM perplexity filter, and removal of documents whose Markdown tables had more than 25% empty cells. Rendering varied 39 Arabic fonts, page sizes, colours, alignment and columns. The augmentation pool has 29 transforms in eight categories, applied 1, 2 or 3 at a time to three 50k subsets.
  • Results on Misraj-DocOCR:
    • Baseer scored WER 0.25, TEDS 66 and MARS 76.885.
    • Gemini-2.5-pro scored 0.37, 52 and 70.775 on the same metrics.
    • Baseer's CER of 0.53 is worse than Azure (0.27) and Gemini (0.31), and Gemini leads on BLEU and CHRF.
  • On the reviewed KITAB-Bench split (only 30 samples, open-source models only), Dots.ocr wins the text metrics (WER 0.39 vs 0.61). Baseer has the best TEDS (56) and MARS (68.13).
  • Both benchmarks are released on Hugging Face as Misraj/Misraj-DocOCR and Misraj/KITAB_pdf_to_markdown_reviewed.

Interesting ideas

  • I1. Instead of annotating images, the authors start from text whose structure is already known and render it into images, so the gold labels are exact by construction. — For Mova (learn): Sentences generated from known meaning graphs could serve as training pairs whose trees and frames are correct by design, complementing silver data from witness parsers.
  • I2. Corruption is applied in graded doses (one, two or three transforms per subset), and the clean originals of those samples are dropped to avoid duplicates. — For Mova (tokens): The tokenizer and tagger could be stress-tested on text with stepwise added noise (typos, casing, broken punctuation) to see at which noise level accuracy collapses.
  • I3. Before scoring, all system outputs are normalized so that equivalent but differently formatted answers are not penalized. — For Mova (tree): Equivalent UD dialect conventions should be canonicalized before comparing LAS across witness parsers, so disagreements reflect real structure rather than annotation style.
  • I4. The authors audited and corrected an existing benchmark and warned that its small size (30 items) makes scores volatile. — For Mova (answer): School-test answer keys and fable QA sets should be re-verified, and their size reported next to accuracy, so a few errors are not mistaken for real progress.
  • I5. The perception component is frozen and only the generating part is tuned, which keeps general features while specializing the output. — For Mova (tree): In gparse, the pretrained transformer encoder could be frozen while only the parsing head is trained on new treebanks, reducing cost and the risk of forgetting.

Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.

Fields: Linguistics & languages

Cited by

3 papers of our library cite it (of the 140884 whose bibliographies we read); 1 of them have a page here, 2 not yet.

Cite

The paper itself, not this page. BibTeX from the library's record of the work. DOI 10.48550/arxiv.2509.18174 · arXiv 2509.18174

@misc{hennara2025baseer,
  title = {{Baseer: A Vision-Language Model for Arabic Document-to-Markdown OCR}},
  author = {Hennara, Khalil and Hreden, Muhammad and Hamed, Mohamed Motasim and Bastati, Ahmad and Aldallal, Zeina and Chrouf, Sara and AlModhayan, Safwan},
  year = {2025},
  doi = {10.48550/arxiv.2509.18174},
  eprint = {2509.18174},
  archivePrefix = {arXiv},
  url = {https://doi.org/10.48550/arxiv.2509.18174}
}

References

20 references, 9 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

  1. Shuai Bai, Keqin Chen, Xuejing Liu, Jialin Wang, Wenbin Ge, Sibo Song, Kai Dang, Peng Wang, Shijie Wang, Jun Tang, et al. Qwen2. 5-vl technical report. arXiv preprint arXiv:2502.13923, 2025. ↗ our card
  2. Zhe Chen, Weiyun Wang, Yue Cao, Yangzhou Liu, Zhangwei Gao, Erfei Cui, Jinguo Zhu, Shenglong Ye, Hao Tian, Zhaoyang Liu, et al. Expanding performance boundaries of open-source multimodal models with model, data, and test-time scaling. arXiv preprint arXiv:2412.05271, 2024. ↗ our card
  3. Gheorghe Comanici, Eric Bieber, Mike Schaekermann, Ice Pasupat, Noveen Sachdeva, Inderjit Dhillon, Marcel Blistein, Ori Ram, Dan Zhang, Evan Rosen, et al. Gemini 2.5: Pushing the frontier with advanced reasoning, multimodality, long context, and next generation agentic capabilities. arXiv preprint arXiv:2507.06261, 2025. ↗ our card
  4. Saurabh Dash, Yiyang Nan, John Dang, Arash Ahmadian, Shivalika Singh, Madeline Smith, Bharat Venkitesh, Vlad Shmyhlo, Viraat Aryabumi, Walter Beller-Morales, et al. Aya vision: Advancing the frontier of multilingual multimodality. arXiv preprint arXiv:2505.08751, 2025. ↗
  5. Kenneth Heafield. KenLM: Faster and smaller language model queries. In Chris Callison-Burch, Philipp Koehn, Christof Monz, and Omar F. Zaidan (eds.), Proceedings of the Sixth Workshop on Statistical Machine Translation, pp. 187–197, Edinburgh, Scotland, July 2011. Association for Computational Linguistics. URL https://aclanthology.org/W11-2123/. ↗ our card
  6. Ahmed Heakl, Abdullah Sohail, Mukul Ranjan, Rania Hossam, Ghazi Shazan Ahmad, Mohamed El-Geish, Omar Maher, Zhiqiang Shen, Fahad Khan, and Salman Khan. Kitab-bench: A comprehensive multi-domain benchmark for arabic ocr and document understanding. arXiv preprint arXiv:2502.14949, 2025. ↗
  7. Aaron Hurst, Adam Lerer, Adam P Goucher, Adam Perelman, Aditya Ramesh, Aidan Clark, AJ Ostrow, Akila Welihinda, Alan Hayes, Alec Radford, et al. Gpt-4o system card. arXiv preprint arXiv:2410.21276, 2024. ↗ our card
  8. Hugo Laurençon, Andrés Marafioti, Victor Sanh, and Léo Tronchon. Building and better understanding vision-language models: insights and future directions. arXiv preprint arXiv:2408.12637, 2024a. ↗
  9. Hugo Laurençon, Léo Tronchon, Matthieu Cord, and Victor Sanh. What matters when building vision-language models? Advances in Neural Information Processing Systems, 37:87874–87907, 2024b. find our card
  10. Feng Li, Renrui Zhang, Hao Zhang, Yuanhan Zhang, Bo Li, Wei Li, Zejun Ma, and Chunyuan Li. Llava-next-interleave: Tackling multi-image, video, and 3d in large multimodal models. arXiv preprint arXiv:2407.07895, 2024. ↗
  11. Zhang Li, Yuliang Liu, Qiang Liu, Zhiyin Ma, Ziyang Zhang, Shuo Zhang, Zidun Guo, Jiarui Zhang, Xinyu Wang, and Xiang Bai. Monkeyocr: Document parsing with a structure-recognition-relation triplet paradigm. arXiv preprint arXiv:2506.05218, 2025. ↗
  12. Haotian Liu, Chunyuan Li, Qingyang Wu, and Yong Jae Lee. Visual instruction tuning. Advances in neural information processing systems, 36:34892–34916, 2023. find our card
  13. Haotian Liu, Chunyuan Li, Yuheng Li, and Yong Jae Lee. Improved baselines with visual instruction tuning. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pp. 26296–26306, 2024. find our card
  14. Souvik Mandal, Ashish Talewar, Paras Ahuja, and Prathamesh Juvatkar. Nanonets-ocr-s: A model for transforming documents into structured markdown with intelligent content recognition and semantic tagging, 2025. find
  15. Andrés Marafioti, Orr Zohar, Miquel Farré, Merve Noyan, Elie Bakouch, Pedro Cuenca, Cyril Zakka, Loubna Ben Allal, Anton Lozhkov, Nouamane Tazi, et al. Smolvlm: Redefining small and efficient multimodal models. arXiv preprint arXiv:2504.05299, 2025. ↗
  16. Ahmed Nassar, Andres Marafioti, Matteo Omenetti, Maksym Lysak, Nikolaos Livathinos, Christoph Auer, Lucas Morin, Rafael Teixeira de Lima, Yusik Kim, A Said Gurbuz, et al. Smoldocling: An ultra-compact vision-language model for end-to-end multi-modal document conversion. arXiv preprint arXiv:2503.11576, 2025. ↗
  17. Andreas Steiner, André Susano Pinto, Michael Tschannen, Daniel Keysers, Xiao Wang, Yonatan Bitton, Alexey Gritsenko, Matthias Minderer, Anthony Sherbondy, Shangbang Long, et al. Paligemma 2: A family of versatile vlms for transfer. arXiv preprint arXiv:2412.03555, 2024. ↗
  18. Gemma Team, Aishwarya Kamath, Johan Ferret, Shreya Pathak, Nino Vieillard, Ramona Merhej, Sarah Perrin, Tatiana Matejovicova, Alexandre Ramé, Morgane Rivière, et al. Gemma 3 technical report. arXiv preprint arXiv:2503.19786, 2025. ↗
  19. Ahmed Wasfy, Omer Nacar, Abdelakreem Elkhateb, Mahmoud Reda, Omar Elshehy, Adel Ammar, and Wadii Boulila. Qari-ocr: High-fidelity arabic text recognition through multimodal large language model adaptation. arXiv preprint arXiv:2506.02295, 2025. ↗
  20. Jinguo Zhu, Weiyun Wang, Zhe Chen, Zhaoyang Liu, Shenglong Ye, Lixin Gu, Hao Tian, Yuchen Duan, Weijie Su, Jie Shao, et al. Internvl3: Exploring advanced training and test-time recipes for open-source multimodal models. arXiv preprint arXiv:2504.10479, 2025. ↗ our card

↑ back to top

This page in Markdown → https://md.research.elamur.ai/free/en/papers/hennara-2025-baseer-vision-language-model-arabic/