Elamur Research⌕
☰
home / papers / 2026 / Scaling Vision–Language Models for Pharmaceutical Long-Form Video Reasoning on Industrial GenAI Platform

Scaling Vision–Language Models for Pharmaceutical Long-Form Video Reasoning on Industrial GenAI Platform

Paper · 2026 · Suyash Mishra, Qiang Li, Srikanth Patil et al.

Summary

Most benchmarks for vision–language models (VLMs) use short clips and assume ample compute. Pharmaceutical work is different: long trial recordings, lectures and patient-education videos must be processed under tight GPU, latency and compliance limits. The authors (Roche, Accenture, Involead) describe RICI, an industrial GenAI platform for natural-language search. It combines LLMs, audio language models (ALMs) and VLMs to ingest, caption, tag and vectorize content.

Rather than proposing a new architecture, they benchmark over 40 existing VLMs on Video-MME, MMBench and an internal pharma video collection, comparing strong and weaker GPUs. They also add summarization and keyframe subtasks to Video-MME, scored with a knowledge-graph schema.

Four lessons emerge:

  • Adding transcripts and metadata helps most task types, especially for long videos, though temporal reasoning may suffer.
  • The attention implementation should be matched to the hardware.
  • Both open and proprietary models are weak at timestamps and keyframes.
  • Splitting or compressing long videos tends to add errors rather than save time.

The result is practical guidance for deploying VLMs in regulated industries.

Key points

  • The platform ingested 200,000+ PDFs, 25,326 videos in 8 formats and 888 audio files covering 20+ languages and 14 disease areas. The videos span less than 2 minutes up to more than 3 hours in duration.
  • 42 VLMs/ALMs were tested, including GPT-4o, Gemini 1.5 Pro, Qwen2-VL, InternVL and Whisper, on NVIDIA A100 and A10G GPUs.
  • SDPA attention delivered 3–8× efficiency gains on commodity GPUs. Multimodal input improved results in 8 of 12 task domains.
  • End-to-end processing averaged about 2.2 minutes per long Video-MME video. This cut manual effort by 94.4% for long videos and 88% overall.
  • In production pilots, the time needed to create patient-facing materials dropped by 66%.

Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.

 Fields, topics and concepts

📚 Shelves 5The field of science this work belongs to (OpenAlex fields).Computer Science · 16,667 worksArtificial Intelligence · 16,194 worksArts and Humanities · 16,039 worksLanguage and Linguistics · 15,891 worksComputer Vision and Pattern Recognition · 2,869 works

🧭 Topics 3Research subjects and languages the work is about.Language models & pretraining · 3,320 worksReasoning · 1,426 worksVision & language · 657 works

Cite

Cite as Suyash Mishra, Qiang Li, Srikanth Patil, Satyanarayan Pati and Baddu Narendra. 2026. Scaling Vision–Language Models for Pharmaceutical Long-Form Video Reasoning on Industrial GenAI Platform. Proceedings of the 4th Workshop on Advances in Language and Vision Research (ALVR).

The paper itself, not this page. BibTeX from the library's record of the work. DOI 10.18653/v1/2026.alvr-main.7

@inproceedings{mishra2026scaling,
  title = {{Scaling Vision–Language Models for Pharmaceutical Long-Form Video Reasoning on Industrial GenAI Platform}},
  author = {Suyash Mishra, Qiang Li, Srikanth Patil, Satyanarayan Pati and Baddu Narendra},
  booktitle = {Proceedings of the 4th Workshop on Advances in Language and Vision Research (ALVR)},
  year = {2026},
  doi = {10.18653/v1/2026.alvr-main.7},
  url = {https://doi.org/10.18653/v1/2026.alvr-main.7}
}

References

48 references, 12 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

  1. Ai central (american college of radiology) – fda-cleared imaging ai directory. https://aicentral.acrdsi.org/. Over 200 FDA-cleared imaging AI products cataloged; Accessed 2025-10-07. ↗
  2. Hugging Face. 2024. Transformers performance inference on gpu. Accessed: 2024-03-10. find
  3. Artificial intelligence (ai/ml)-enabled medical devices. https://www.fda.gov/medical-devices/ Ghazala Fatima, Imran Khan, Saeed Farooq, et al. 2024. software-medical-device-samd/ A comprehensive review of advances in digital pathol-artificial-intelligence-enabled-medical-devices. ogy. Diagnostics. ↗
  4. Chaoyou Fu, Peixian Chen, Yunhang Shen, Yulei Qin, Mengdan Zhang, Xu Lin, Jinrui Yang, Xiawu Zheng, Ke Li, Xing Sun, et al. 2023. Mme: A comprehensive evaluation benchmark for multimodal large language models. arXiv preprint arXiv:2306.13394.
  5. Zongxia Li, Xiyang Wu, Hongyang Du, Huy Nghiem, and Guangyao Shi. 2025b. Benchmark evaluations, applications, and challenges of large vision language models: A survey. Preprint, arXiv:2501.02189. — [arXiv:2501.02189](https://arxiv.org/abs/2501.02189) find
  6. Yuan Liu, Haodong Duan, Yuanhan Zhang, Bo Li, Songyang Zhang, Wangbo Zhao, Yike Yuan, Jiaqi Wang, Conghui He, Ziwei Liu, Kai Chen, and Dahua Lin. 2023. Mmbench: Is your multi-modal model an all-around player? arXiv:2307.06281.
  7. Chaoyou Fu, Yuhan Dai, Yondong Luo, Lei Li, Shuhuai Ren, Renrui Zhang, Zihan Wang, Chenyu Zhou, Yunhang Shen, Mengdan Zhang, et al. 2024a. Video-mme: The first-ever comprehensive evaluation benchmark of multi-modal llms in video analysis. arXiv preprint arXiv:2405.21075.
  8. Ismini Lourentzou, Chen Chen, and ChengXiang Zhai. 2021. Adarenet: Adaptive reweighted semi-supervised active learning. In Proceedings of the 14th ACM International Conference on PErvasive Technologies Related to Assistive Environments (PE-TRA), pages 1–8. find
  9. Chaoyou Fu, Yi-Fan Zhang, Shukang Yin, Bo Li, Xinyu Fang, Sirui Zhao, Haodong Duan, Xing Sun, Ziwei Liu, Liang Wang, et al. 2024b. Mme-survey: A comprehensive survey on evaluation of multimodal llms. arXiv preprint arXiv:2411.15296. — [arXiv:2411.15296](https://arxiv.org/abs/2411.15296) find
  10. O. Mahmoud and et al. 2024. Catstep: Automated cataract surgical phase classification using deep learning. Ophthalmology Science. find
  11. Gemini Team, Google: Petko Georgiev and 1135 other authors. 2024. Gemini 1.5: Unlocking multimodal understanding across millions of tokens of context. arXiv preprint arXiv:2403.05530.
  12. Suyash Mishra, Qiang Li, Srikanth Patil, and Anubhav Girdhar. 2026. From understanding to engagement: Personalized pharmacy video clips via vision language models (vlms). Preprint, arXiv:2601.05059.
  13. N. Ghamsarian and et al. 2024. Cataract-1k: A large-scale cataract surgery video dataset for surgical workflow analysis. Scientific Data. find
  14. R. et al. Najjar. 2023. Redefining radiology: A review of artificial intelligence. PMC. find
  15. Md. Haque et al. 2025. Demographic bias in public remote photoplethysmography datasets. npj Digital Medicine. find
  16. OpenAI. 2023. Chatgpt for intelligence briefing: A case study. Accessed: March 2025. find
  17. Bin Huang et al. 2023. Challenges and prospects of visual contactless physiological monitoring. npj Digital Medicine. find
  18. OpenAI. 2024. Chatgpt: Optimizing language models for dialogue. Accessed: March 2025. find
  19. T. Oshika and et al. 2024. Artificial intelligence applications in ophthalmology. find
  20. Patrick Lewis, Ethan Perez, Aleksandra Piktus, Fabio Petroni, Vladimir Karpukhin, Naman Goyal, Heinrich Küttler, Mike Lewis, Wen tau Yih, Tim Rocktäschel, Sebastian Riedel, and Douwe Kiela. 2021. Retrieval-augmented generation for knowledge-intensive nlp tasks. Preprint, arXiv:2005.11401.
  21. Long Ouyang, Jeffrey Wu, Xu Jiang, Diogo Almeida, Carroll Wainwright, Pamela Mishkin, Chong Zhang, Sandhini Agarwal, Katarina Slama, Alex Ray, et al. 2022. Training language models to follow instructions with human feedback. In Advances in Neural Information Processing Systems, volume 35, pages 2773…
  22. Johann Li, Guangming Zhu, and Cong Hua. 2021. A systematic collection of medical image datasets for deep learning. arXiv preprint arXiv:2106.12864. — [arXiv:2106.12864](https://arxiv.org/abs/2106.12864) find
  23. Evrim Ozmermer and Qiang Li. 2023. Self-supervised learning with temporary exact solutions: Linear projection. In 2023 IEEE 21st International Conference on Industrial Informatics (INDIN), pages 1–7. find
  24. Junnan Li, Ramprasaath R. Selvaraju, Akhilesh D. Gotmare, Shafiq Joty, Caiming Xiong, and Steven C.H. Hoi. 2022. Blip: Bootstrapped language-image pre-training for unified vision-language understanding and generation. In Proceedings of the 39th International Conference on Machine Learning (ICML), pa… find
  25. L. et al. Pinto-Coelho. 2023. How artificial intelligence is shaping medical imaging. PMC. find
  26. Xiaoye Qu, Jiashuo Sun, Wei Wei, Daizong Liu, Jianfeng Dong, and Yu Cheng. 2025. Look, compare, decide: Alleviating hallucination in large vision-language models via multi-view multi-path reasoning. In Proceedings of the 31st International Conference on Computational Linguistics. Association for Com… find
  27. Qiang Li, Mingkun Tan, Xun Zhao, Dan Zhang, Daoan Zhang, Shengzhao Lei, Anderson S. Chu, Lujun Li, and Porawit Kamnoedboon. 2025a. How LLMs react to industrial spatio-temporal data? assessing hallucination with a novel traffic incident benchmark dataset. In Proceedings of the 2025 Conference of the… find
  28. Alec Radford, Jong Wook Kim, Chris Hallacy, Aditya Ramesh, Gabriel Goh, Sandhini Agarwal, Girish Sastry, Amanda Askell, Pamela Mishkin, Jack Clark, Gretchen Krueger, and Ilya Sutskever. 2021. Learning transferable visual models from natural language supervision. In Proceedings of the 38th Internatio…
  29. X. Xu and et al. 2024. The application of artificial intelligence in diabetic retinopathy: progress and challenges. find
  30. Kaixun Yang, Mladen Raković, Zhiping Liang, Lixiang Yan, Zijie Zeng, Yizhou Fan, Dragan Gašević, and Guanliang Chen. 2024. Modifying ai, enhancing essays: How active engagement with generative ai boosts writing quality. Preprint, arXiv:2412.07200. — [arXiv:2412.07200](https://arxiv.org/abs/2412.07200) find
  31. Manuel Sebastián Ríos and et al. 2023. Cholec80-cvs: An open dataset with an evaluation of strasberg’s critical view of safety for ai. Scientific Data. find
  32. Moran Yanuka, Assaf Ben-Kish, Yonatan Bitton, Idan Szpektor, and Raja Giryes. 2025. Bridging the visual gap: Fine-tuning multimodal models with knowledge-adapted captions. In Proceedings of the 2025 Conference of the Nations of the Americas Chapter of the Association for Computational Linguistics: H… find
  33. H. Saeidi and et al. 2022. Autonomous robotic laparoscopic surgery for intestinal anastomosis. Science Robotics. find
  34. Amanpreet Singh, Ronghang Hu, Vedanuj Goswami, Dhruv Mahajan, Xinlei Wu, Christoph Feichtenhofer, Trevor Darrell, and Ross Girshick. 2022. Flava: A foundational vision and language model for generalized multimodal learning. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Rec… find
  35. A. Yilmaz and et al. 2024. Derm12345: A large, multi-source dermatoscopic skin lesion dataset. Scientific Data. find
  36. Karan Singhal, Shekoofeh Azizi, Tao Tu, S Sara Mahdavi, Jason Wei, Hyung Won Chung, Nathan Scales, Ajay Tanwani, Heather Cole-Lewis, Stephen Pfohl, et al. 2022. Large language models encode clinical knowledge. arXiv preprint arXiv:2212.13138.
  37. Dan Zhang, Fangfang Zhou, Yuwen Jiang, Felix Albu, Yuanzhou Wei, Xiao Yang, and Qiang Li. 2026. Unleashing the power of self-supervised image denoising: A comprehensive review. Computer Vision and Image Understanding, 269:104786. find
  38. K. et al. Strz˛epek. 2023. Quantitative and qualitative analysis of agricultural fields with uavs. PMC. find
  39. Dan Zhang, Fangfang Zhou, Yuwen Jiang, and Zhengming Fu. 2023. Mm-bsn: Self-supervised image denoising for real-world with multi-mask based on blind-spot network. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) Workshops, pages 4189–4198. find
  40. Amanda Tan-Garcia et al. 2025. Computational pathology in the age of artificial intelligence. Histopathology. find
  41. Romal Thoppilan, Daniel De Freitas, Jamie Hall, Noam Shazeer, Apoorv Kulshreshtha, Heng-Tze Cheng, Alicia Jin, Taylor Bos, Leslie Baker, Yu Du, et al. 2022. Lamda: Language models for dialog applications. arXiv preprint arXiv:2201.08239.
  42. Yi-Fan Zhang, Huanyu Zhang, Haochen Tian, Chaoyou Fu, Shuangqing Zhang, Junfei Wu, Feng Li, Kun Wang, Qingsong Wen, Zhang Zhang, et al. 2024a. Mme-realworld: Could your multimodal llm challenge high-resolution real-world scenarios that are difficult for humans? arXiv preprint arXiv:2408.13257. — [arXiv:2408.13257](https://arxiv.org/abs/2408.13257) find
  43. Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N. Gomez, Lukasz Kaiser, and Illia Polosukhin. 2017. Attention is all you need. In Advances in Neural Information Processing Systems.
  44. Yifu Zhang, Peize Sun, Yi Jiang, Dongdong Yu, Fucheng Weng, Zehuan Yuan, Ping Luo, Wenyu Liu, and Xinggang Wang. 2022. Bytetrack: Multi-object tracking by associating every detection box. In Computer Vision – ECCV 2022, volume 13682 of Lecture Notes in Computer Science, pages 1–21. Springer, Cham. find
  45. D. et al. Wang. 2025. A survey of unmanned aerial vehicles and deep learning in precision agriculture. ScienceDirect. find
  46. Peng Wang, An Yang, Rui Men, Ming Zhou, Yinan Zhang, Junyang Lin, Xu Sun, Shuai Li, Houqiang Wang, and Luo Si. 2022. Ofa: Unifying architectures, tasks, and modalities through a simple sequence-to-sequence learning framework. arXiv preprint arXiv:2202.03052.
  47. Yuanhan Zhang, Bo Li, Haotian Liu, Yong Jae Lee, Liangke Gui, Di Fu, Jiashi Feng, Ziwei Liu, and Chunyuan Li. 2024b. Llava-next: A strong zero-shot video understanding model. find
  48. Jing Wei, Zhanqing Li, Alexei Lyapustin, Jun Wang, Oleg Dubovik, Joel Schwartz, Lin Sun, Chi Li, Song Liu, Tong Zhu, et al. 2023. First close insight into global daily gapless 1 km pm 2.5 pollution, variability, and health impact. Nature Communications, 14:8349. find

Updated

↑ back to top

This page in Markdown → https://md.research.elamur.ai/free/en/papers/mishra-2026-scaling-vision-language-models-pharmaceutical/