Elamur Research⌕
☰
home / papers / 2026 / Compact Multimodal Language Models as Robust OCR Alternatives for Noisy Textual Clinical Reports

Compact Multimodal Language Models as Robust OCR Alternatives for Noisy Textual Clinical Reports

Paper · 2026 · Nikita Neveditsin, Pawan Lingras, Salil Patil et al.

Summary

Hospitals often digitize printed medical reports by photographing them with smartphones. The resulting images suffer from blur, shadows, rotation, bleed-through and background clutter, and conventional OCR handles these poorly. This study asks whether small multimodal language models (up to 14B parameters) that can run locally can replace OCR engines for such material while keeping patient data on-premises.

The authors used a private collection of 340 anonymized obstetric ultrasound reports from an Indian clinic, written in a regional style of medical English. They manually transcribed and noise-annotated 60 of them. On these, they benchmarked eight systems covering four families:

  • classical OCR
  • modular neural OCR
  • end-to-end neural OCR
  • compact MLLMs

Each system was scored on CER, WER, numeric accuracy, noise sensitivity, runtime and memory use.

The compact MLLMs clearly made the fewest errors and stayed robust to image degradation, although they were slower and needed more GPU memory. Because some of them fit on consumer GPUs, privacy-preserving, high-quality digitization of clinical archives becomes realistic.

Key points

  • Systems compared: Tesseract, docTR, PaddleOCR, Surya, GOT-OCR 2.0, Qwen-2.5-VL (7B), Phi-4 MM (14B) and InternVL3.5 (4B). All received raw whole-page images, and the MLLMs were given a simple verbatim-transcription prompt.
  • Accuracy: Qwen-2.5-VL reached the best CER (0.031; WER 0.078), and Phi-4 MM the best WER (0.075). Tesseract scored a CER of 0.189 and a WER of 0.276. GOT-OCR 2.0 had a low CER but an inflated WER (0.395) because it handled spaces inconsistently.
  • Numeric accuracy: the MLLMs kept numeric accuracy above 92%, and it did not decline as overall errors grew. For the non-MLLM systems, numeric accuracy fell together with overall transcription quality.
  • Noise sensitivity: Tesseract was strongly affected by blur, shadows and NR-IQA scores (BRISQUE, NIQE, PIQE), while the MLLMs were barely affected. NR-IQA metrics did not capture bleed-through, rotation or background text. Inter-annotator agreement on noise labels (Krippendorff's α) ranged from 0.62 to 0.85.
  • Cost: the MLLMs took about 11–67 s per image.
    • Qwen-2.5-VL and InternVL needed roughly 17–18 GiB of GPU memory, so they fit on GPUs with about 20 GB.
    • Phi-4 MM peaked at about 47 GiB.
    • docTR and Tesseract each needed under 1 s per image.
    • The code is released at github.com/neveditsin/eacl_ind_ocr.

Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.

 Fields, topics and concepts

📚 Shelves 5The field of science this work belongs to (OpenAlex fields).Computer Science · 16,667 worksArtificial Intelligence · 16,194 worksArts and Humanities · 16,039 worksLanguage and Linguistics · 15,891 worksComputer Vision and Pattern Recognition · 2,869 works

🧭 Topics 1Research subjects and languages the work is about.Vision & language · 657 works

Cite

Cite as Nikita Neveditsin, Pawan Lingras, Salil Patil, Swarup Patil and Vijay Kumar Mago. 2026. Compact Multimodal Language Models as Robust OCR Alternatives for Noisy Textual Clinical Reports. Proceedings of the 19th Conference of the European Chapter of the Association for Computational Linguistics (Volume 5: Industry Track).

The paper itself, not this page. BibTeX from the library's record of the work. DOI 10.18653/v1/2026.eacl-industry.4

@inproceedings{neveditsin2026compact,
  title = {{Compact Multimodal Language Models as Robust OCR Alternatives for Noisy Textual Clinical Reports}},
  author = {Nikita Neveditsin, Pawan Lingras, Salil Patil, Swarup Patil and Vijay Kumar Mago},
  booktitle = {Proceedings of the 19th Conference of the European Chapter of the Association for Computational Linguistics (Volume 5: Industry Track)},
  year = {2026},
  doi = {10.18653/v1/2026.eacl-industry.4},
  url = {https://doi.org/10.18653/v1/2026.eacl-industry.4}
}

References

15 references, 2 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

  1. Jinze Bai, Shuai Bai, Shusheng Yang, Shijie Wang, Sinan Tan, Peng Wang, Junyang Lin, Chang Zhou, and Jingren Zhou. 2023. Qwen-VL: A versatile vision-language model for understanding, localization, text reading, and beyond. Preprint, arXiv:2308.12966.
  2. John F. Nettrour, M. Benjamin Burch, and B. Sonny Bal. 2019. Patients, pictures, and privacy: managing clinical photographs in the smartphone era. Arthroplasty Today, 5(1):57–60. find
  3. Nikita Neveditsin, Pawan Lingras, and Vijay Mago. 2025a. Clinical insights: A comprehensive review of language models in medicine. PLOS Digital Health, 4(5):e0000800. find
  4. JC. Burie, J. Chazalon, M. Coustaty, S. Eskenazi, M.M. Luqman, M. Mehri, N. Nayef, JM. Ogier, S. Prum, and M. Rusiñol. 2015. ICDAR2015 competition on smartphone document capture and OCR (smart-doc). In 2015 13th International Conference on Document Analysis and Recognition (ICDAR), pages 1161–1165. find
  5. Nikita Neveditsin, Pawan Lingras, and Vijay Kumar Mago. 2025b. Evaluating structured output robustness of small language models for open attribute-value extraction from clinical notes. In Proceedings of the 63rd Annual Meeting of the Association for Computational Linguistics (Volume 4: Student Resea… find
  6. Cheng Cui, Ting Sun, Manhui Lin, Tingquan Gao, Yubo Zhang, Jiaxuan Liu, Xueqing Wang, Zelun Zhang, Changda Zhou, Hongen Liu, Yue Zhang, Wenyu Lv, Kui Huang, Yichao Zhang, Jing Zhang, Jun Zhang, Yi Liu, Dianhai Yu, and Yanjun Ma. 2025. PaddleOCR 3.0 technical report. Preprint, arXiv:2507.05595. — [arXiv:2507.05595](https://arxiv.org/abs/2507.05595) find
  7. Vikas Paruchuri and Datalab Team. 2025. Surya: A lightweight document ocr and analysis toolkit. https://github.com/VikParuchuri/surya. GitHub repository. ↗
  8. Andrew Fisher, Karthik Srinivasan, Sean Hillier, and Vijay Mago. 2025. Heal-summ: a lightweight and ethical framework for accessible summarization of health information. Frontiers in Public Health, 13:1619274. find
  9. Ray Smith. 2007. An overview of the tesseract ocr engine. In Proceedings of the Ninth International Conference on Document Analysis and Recognition (ICDAR 2007), pages 629–633. find
  10. Junjie Gao, Runze Liu, Yingzhe Peng, Shujian Yang, Jin Zhang, Kai Yang, and Zhiyuan You. 2025. find
  11. Deqa-Adnan Ul-Hasan, Syed Saqib Bukhari, Faisal Shafait, and Andreas Dengel. 2016. High performance ocr for camera-captured blurred documents with lstm networks. In Proceedings of the 12th IAPR Workshop on Document Analysis Systems (DAS), pages 7–12. find
  12. N. Venkatanath, D. Praneeth, Maruthi Bhargav Chandrasekhar, Sumohana S. Channappayya, and Sharath S. Medasani. 2015. Blind image quality evaluation using perception based features. In Proceedings of the 21st National Conference on Communications (NCC), pages 1–6. find
  13. Sam Walters, Benjamin Metcalfe, Martin Twiste, Elena Seminati, and Nicola Y Bailey. 2024. Smartphone scanning is a reliable and accurate alternative to contemporary residual limb measurement techniques. PLOS ONE, 19(12):e0313542. find
  14. Weiyun Wang, Zhangwei Gao, Lixin Gu, Hengjun Pu, Long Cui, Xingguang Wei, Zhaoyang Liu, Linglin Jing, Shenglong Ye, Jie Shao, Zhaokai Wang, Zhe Chen, Hongjie Zhang, Ganlin Yang, Haomin Wang, Qi Wei, Jinhui Yin, Wenhao Li, Erfei Cui, and 56 others. 2025. Internvl3.5: Advancing open-source multimodal…
  15. Haoran Wei, Chenglong Liu, Jinyue Chen, Jia Wang, Lingyu Kong, Yanming Xu, Zheng Ge, Liang Zhao, Jianjian Sun, Yuang Peng, Chunrui Han, and Xiangyu Zhang. 2024. General ocr theory: Towards ocr-2.0 via a unified end-to-end model. Preprint, arXiv:2409.01704. — [arXiv:2409.01704](https://arxiv.org/abs/2409.01704) find

Updated

↑ back to top

This page in Markdown → https://md.research.elamur.ai/free/en/papers/neveditsin-2026-compact-multimodal-language-models-as/