Elamur Research⌕
☰
home / papers / 2026 / Thinking Like a Botanist: Challenging Multimodal Language Models with Intent Driven Chain-of-Inquiry

Thinking Like a Botanist: Challenging Multimodal Language Models with Intent Driven Chain-of-Inquiry

Paper · 2026 · Syed Nazmus Sakib, Nafiul Haque, Shahrear Bin Amin et al.

Summary

Vision-language models are usually tested on isolated, single-turn questions. Plant pathologists work differently: they inspect a leaf, spot visual cues, and choose follow-up questions according to species, symptoms and severity. The authors capture this with a Chain-of-Inquiry (CoI) framework, in which a diagnosis is an ordered sequence of question-answer steps tied to grounded visual evidence and an explicit epistemic intent. For example, early ambiguous cases focus on differential diagnosis, while severe cases shift to prognosis and management.

On this basis they build PlantInquiryVQA, a benchmark of expert-curated leaf images. Visual-cue descriptions were extracted automatically and then checked by domain experts. Botanist-compiled question templates were then turned into adaptive QA chains by an LLM pipeline. Open-source and proprietary MLLMs perceive symptoms reasonably well but fall short on safe clinical reasoning and correct diagnosis. Guiding models with intent-driven question sequences improves correctness, reduces hallucination and makes answers more efficient. The resource moves agricultural AI beyond static classification toward expert-like diagnostic dialogue.

Key points

  • Dataset scale and availability: 24,950 leaf images across 34 crop species and 138,068 QA pairs, with visual grounding and severity labels. Code is on GitHub (MIT) and data on Hugging Face (CC BY 4.0).
  • Taxonomy: 12 reasoning templates grouped into 7 cognitive categories, such as Etiological Reasoning, Differential Diagnosis and Counterfactual Analysis.
  • Annotation pipeline:
    • Qwen3-VL-4B, the best of three VLMs tested on 250 images, extracted visual cues.
    • GPT-4V cross-checked the cues and flagged divergent outputs.
    • Experts reviewed all flagged items plus 5,000 random images, giving a 93.8% Factuality Score.
  • Main results: Gemini-3-Flash scored best overall but reached only 0.188 Clinical Utility and 0.147 Safety. Grok-4.1-Fast described symptoms more accurately than it diagnosed them.
  • Evaluation and limitations: Lexical metrics were supplemented by LLM-as-judge scoring (GPT-5, Gemini-3-Pro) on the expert-verified subset. The benchmark is English-only and limited to single static images.

Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.

 Institutions

Also on the byline: University of Dhaka · Gazipur Agricultural University.

 Fields, topics and concepts

📚 Shelves 5The field of science this work belongs to (OpenAlex fields).Computer Science · 16,667 worksArtificial Intelligence · 16,194 worksArts and Humanities · 16,039 worksLanguage and Linguistics · 15,891 worksComputer Vision and Pattern Recognition · 2,869 works

🧭 Topics 2Research subjects and languages the work is about.Language models & pretraining · 3,320 worksVision & language · 657 works

Cite

Cite as Syed Nazmus Sakib, Nafiul Haque, Shahrear Bin Amin, Hasan Muhammad Abdullah, Md Mehedi Hasan, Mohammad Zabed Hossain and Shifat E. Arman. 2026. Thinking Like a Botanist: Challenging Multimodal Language Models with Intent Driven Chain-of-Inquiry. Findings of the Association for Computational Linguistics: ACL 2026.

The paper itself, not this page. BibTeX from the library's record of the work. DOI 10.18653/v1/2026.findings-acl.1741

@inproceedings{sakib2026thinking,
  title = {{Thinking Like a Botanist: Challenging Multimodal Language Models with Intent Driven Chain-of-Inquiry}},
  author = {Syed Nazmus Sakib, Nafiul Haque, Shahrear {Bin Amin}, Hasan Muhammad Abdullah, Md Mehedi Hasan, Mohammad Zabed Hossain and Shifat E. Arman},
  booktitle = {Findings of the Association for Computational Linguistics: ACL 2026},
  year = {2026},
  doi = {10.18653/v1/2026.findings-acl.1741},
  url = {https://doi.org/10.18653/v1/2026.findings-acl.1741}
}

References

73 references, 7 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

  1. Asma Ben Abacha, Sadid A. Hasan, Vivek Datla, Joey Liu, Dina Demner-Fushman, and Henning Müller. 2019. VQA-Med: Overview of the medical visual question answering task at ImageCLEF 2019. In CLEF 2019 Working Notes. find
  2. Swami Nisha Bhagirath, Vaibhav Bhatnagar, and Linesh Raja. 2023. Winter wheat leaf images dataset. find
  3. Md Masum Billah, Saifuddin Sagor, and Mohammad Shorif Uddin. 2025a. Hibiscus leaf diseases classification dataset. find
  4. Joshua Achiam, Steven Adler, Sandhini Agarwal, Lama Ahmad, Ilge Akkaya, Florencia Aleman, Diogo Almeida, Janko Altenschmidt, Sam Altman, Shubham Anadkat, and 1 others. 2023. Gpt-4 technical report. arXiv preprint arXiv:2303.08774.
  5. Md Masum Billah, Saifuddin Sagor, and Mohammad Shorif Uddin. 2025b. A real-world hibiscus and tea leaf image dataset for classification. find
  6. Prayma Bishshash, Md Asraful Sharker Nirob, Md Habibur Shikder, and Afjal Sarower. 2024. Sar-cld-2024: A comprehensive dataset for cotton leaf disease detection. find
  7. Md. Zahid Hasan, Abu Kowshir Bitto, and Md Hasan Imam Bijoy. 2025b. Brinjalfruitx: A field-collected image dataset for machine learning and deep learning-based disease identification in brinjal fruits. find
  8. Abhishek Das, Satwik Kottur, Khushi Gupta, Avi Singh, Deshraj Yadav, José MF Moura, Devi Parikh, and Dhruv Batra. 2017. Visual dialog. In Proceedings of the IEEE conference on computer vision and pattern recognition, pages 326–335. find
  9. Mehedi Hasan, Sonia Khatun, Md Abu Raihan, and Abdul Hasib Uddin. 2023. Rice leaf bacterial and fungal disease dataset. find
  10. Rakib Hasan, Sanimun Hossain Sanzit, Md Mamun Hosen, Ferdous Hasan, Md Mehedi Hasan Topu, and Muksitul Islam. 2025c. High-resolution eggplant leaf image dataset for plant disease classification and detection. find
  11. Utsab Das, Showrov Azam, and Md Abdullah Al Kafi. 2025. Banana and banana leaf dataset for classification and disease detection. find
  12. Sadib Hassan, Md Zahid Akon, Partho Sarathi Sarker, Rahat Hossain, Jannatul Ferdous, Md Mahadi Hasan Shaon, and Md Abdur Razzak. 2025. Ugv: Guava leaves disease dataset bangladesh. find
  13. Pulak Deb Nath. 2025. Citrusleafvision: A diverse dataset for lemon leaf disease detection. find
  14. Pulak Deb Nath, Faruk Ahmed, and Belal Uddin. 2025. Bdrubberleaf: A comprehensive dataset of rubber tree leaf diseases from bangladesh for agricultural research. find
  15. Xuehai He, Yichen Zhang, Luntian Mou, Eric Xing, and Pengtao Xie. 2020. Pathvqa: 30000+ questions for medical visual question answering. arXiv preprint arXiv:2003.10286. — [arXiv:2003.10286](https://arxiv.org/abs/2003.10286) find
  16. Emerson M Del Ponte, Sarah J Pethybridge, Clive H Bock, Sami J Michereff, Franklin J Machado, and Piérri Spolti. 2017. Standard area diagrams for aiding severity estimation: scientometrics, pathosystems, and methodological trends in the last 25 years. Phytopathology, 107(10):1161–1174. find
  17. Ismail Hossain, Khandaker Rezoanul Haque, Abdullah Al Rafi, and Md Zahidul Islam Talukder. 2025a. Tomato leaf diseases. find
  18. Md Riyad Hossain, Mohammad Rifat Ahmmad Rashid, Tasfia Binte Jahangir, Md Samir Hossain, Md Ma-hamudur Rahman, Raiyan Gani, Jubaer Ahmed, Raihan Ul Islam, and M Saddam Hossain Khan. 2025b. Image dataset for turmeric plant leaf disease detection. find
  19. Madhuri Dharrao, Deepak Dharrao, Rakesh Sonawane, and Nilima Zade. 2025. Niphad grape leaf disease dataset (ngld). find
  20. Yue Fan, Jing Gu, Kaiwen Zhou, Qianqi Yan, Shan Jiang, Ching-Chen Kuo, Yang Zhao, Xinze Guan, and Xin Wang. 2024. Muffin or chihuahua? challenging multimodal large language models with multipanel vqa. In Proceedings of the 62nd Annual Meeting of the Association for Computational Linguistics (Volume… find
  21. Shahariar Hossain, Saifuddin Sagor, and Faruk Ahmed. 2025c. High-resolution images of lychee plant diseases for classification and detection. find
  22. Md Faysal Hossan and Md Zamirul Islam Reyad. 2025. Comprehensive mango leaf images dataset for multi-class disease classification and automated plant disease detection. find
  23. Alvan R. Feinstein and Domenic V. Cicchetti. 1990. High agreement but low kappa: I. The problems of two paradoxes. Journal of Clinical Epidemiology, 43(6):543–549. find
  24. Rashidul Hasan Hridoy, Md Tarek Habib, Imran Mahmud, Aminul Haque, and Md Abdulla Al Mamun. 2025. Black gram leaf image dataset for disease detection in field conditions. find
  25. Yunye Gong, Robik Shrestha, Jared Claypoole, Michael Cogswell, Arijit Ray, Christopher Kanan, and Ajay Divakaran. 2024. Bloomvqa: Assessing hierarchical multi-modal comprehension. In Findings of the Association for Computational Linguistics: ACL 2024, pages 14905–14918. find
  26. David P. Hughes and Marcel Salathé. 2015. An open access repository of images on plant health to enable the development of mobile disease diagnostics through machine learning and crowdsourcing. CoRR, abs/1511.08060. — [arXiv:1511.08060](https://arxiv.org/abs/1511.08060) find
  27. Rezwan Huq, Farzia Hossain, Shahida Begum, Raiyan Gani, and Jubaer Ahmed. 2025. Image datasets for jackfruit plant leaf disease. find
  28. Md Ehsanul Haque and Md Al-Imran. 2025. Agrileafnet: Fruit tree leaf dataset for agricultural research. find
  29. ICAR-CITH. 2024. Apple leaf diseases image dataset of ICAR-CITH. find
  30. Md. Taufiq Hasan, Sabbir Hossain Durjoy, Md Emon Shikder, Md. Safaet Zahangir, Md Muntasir Uddin, Md Mehedi Hasan Shoib, and Md Hasan Imam Bijoy. 2025a. Litchileaf4001: A comprehensive dataset of lychee leaf diseases for ai-based visual diagnosis. find
  31. Ahmed Imtiaz, Fahad Bin Islam Swapnil, Syed Rayhan Masud, and Debajoyti Karmaker. 2024. Tomato leaf dataset: A dataset for multiclass disease detection and classification. find
  32. Md Forhadul Islam, Imon Sutradhar, and Md Mizanur Rahman. 2025. Leaf image dataset for disease detection in bitter gourd, okra, pumpkin, and ridge gourd. find
  33. Md Asraful Sharker Nirob, A K M Fazlul Kobir Siam, Prayma Bishshash, and Md Assaduzzaman. 2025b. Chili plant leaf disease and growth stage dataset from bangladesh. find
  34. Abid Khan, Sabit Ahamed Preanto, Tapon Paul, and Md Hasan Imam Bijoy. 2025. Moringaleafnet: A multi-class leaf disease dataset for precision agriculture and deep learning research. find
  35. Judea Pearl and Dana Mackenzie. 2018. The book of why: the new science of cause and effect. Basic books. find
  36. Takeshi Kojima, Shixiang Shane Gu, Machel Reid, Yutaka Matsuo, and Yusuke Iwasawa. 2022. Large language models are zero-shot reasoners. Advances in neural information processing systems, 35:22199– 22213.
  37. Md Istiak Rahman Radowan and Rokonozzaman Ayon. 2025. Disease dataset of wheat: Original, augmented, and balanced for deep learning. find
  38. Maruful Islam Rafe, Farhan Masud Nayem, Shanto Babu Sarker, and Abdullah Al Shiam. 2025. Eggplant_leaf_disease_dataset. find
  39. Jason J Lau, Soumya Gayen, Asma Ben Abacha, and Dina Demner-Fushman. 2018. A dataset of clinically generated visual questions and answers about radiology images. Scientific data, 5(1):1–10. find
  40. Salman Af Rahman, Md Nafiz Imtiaz, Naima Ahmed, and Md Hasan Imam Bijoy. 2025. Burmese grape leaf disease dataset for computer vision-based plant health diagnosis. find
  41. Tony Lee, Haoqin Tu, Chi H Wong, Wenhao Zheng, Yiyang Zhou, Yifan Mai, Josselin S Roberts, Michihiro Yasunaga, Huaxiu Yao, Cihang Xie, and 1 others. 2024. Vhelm: A holistic evaluation of vision language models. Advances in Neural Information Processing Systems, 37:140632–140666. find
  42. Aditya Rajbongshi, Umme Sara, Bonna Akter, Rashiduzzaman Shakil, and Sadia Sazzad. 2022. Sun flower fruits and leaves dataset for sunflower disease classification through machine learning and deep learning. find
  43. Bo Liu, Li-Ming Zhan, Li Xu, Lin Ma, Yan Yang, and Xiao-Ming Wu. 2021. Slake: A semantically-labeled knowledge-enhanced dataset for medical visual question answering. In 2021 IEEE 18th international symposium on biomedical imaging (ISBI), pages 1650– 1654. IEEE. find
  44. Mohammad Rifat Ahmmad Rashid, Raiyan Gani, Jubaer Ahmed, Maherun Nessa Isty, and Sawkat Ali. 2024. Healthy and unhealthy papaya leaf images from bangladeshi orchards. find
  45. Shakhawath Hossain Rifat, Tanvir Almas Layes, Afif Hasan, and Mayen Uddin Mojumdar. 2024. Rice leaf disease and pest dataset overview. find
  46. Jiaxiang Liu, Yuan Wang, Jiawei Du, Joey Tianyi Zhou, and Zuozhu Liu. 2024. Medcot: Medical chain of thought via hierarchical expert. arXiv preprint arXiv:2412.13736. find
  47. Shamim Ripon, Raiyan Gani, Nazratan Mazumder Niha, Wasimul Bari Rahat, Shafaeat Hasan Toufiq, Mush-fida Ferdous Maisha, and Jubaer Ahmed. 2025. Cotton leaf image dataset for disease classification. find
  48. Yongbo Liu. 2025. Tomato disease dataset. find
  49. Laurence V Madden, Gareth Hughes, and F van den Bosch. 2007. The study of plant disease epidemics. find
  50. Saifuddin Sagor, Md Faysal Hossan, Faruk Ahmed, and Md Zamirul Islam Reyad. 2025. Sunflower plant health and growth stage image dataset for agricultural machine learning applications. find
  51. Eram Mahamud and Md Assaduzzaman Tapos. 2024. Lentil plant disease image dataset (4 class). find
  52. Md Mafiul Hasan Matin, Mehedi Hasan Hasan, and Sabik Ur Rahman. 2025. Cauliflower diseases identification image dataset. find
  53. Christoph Schuhmann, Romain Beaumont, Richard Vencu, Cade Gordon, Ross Wightman, Mehdi Cherti, Theo Coombes, Aarush Katta, Clayton Mullis, Mitchell Wortsman, and 1 others. 2022. Laion-5b: An open large-scale dataset for training next generation image-text models. Advances in neural information proce…
  54. Sharada P Mohanty, David P Hughes, and Marcel Salathé. 2016. Using deep learning for image-based plant disease detection. Frontiers in plant science, 7:215232. find
  55. Gail Lynn Schumann and 1 others. 1991. Plant diseases: their biology and social impact. APS Press, American Phytopathological Society St. Paul, MN, USA. find
  56. Md Asraful Sharker Nirob, Prayma Bishshash, Mariyam Bin Ayan, Tania Khatun, and Mohammad Shorif Uddin. 2024. Eggplant dataset: A comprehensive dataset for agricultural research and disease detection. find
  57. Davinder Singh, Naman Jain, Pranjali Jain, Pratik Kayal, Sudhakar Kumawat, and Nipun Batra. 2020. Plantdoc: A dataset for visual plant disease detection. In Proceedings of the 7th ACM IKDD CoDS and 25th COMAD, pages 249–253. find
  58. Md Asraful Sharker Nirob, Prayma Bishshash, A K M Fazlul Kobir Siam, and Mayen Uddin Mojumdar. 2025a. Cair-bgd-2025: Annotated dataset for bottle gourd disease & growth stages. find
  59. Richard N Strange. 2003. Introduction to plant pathology. John Wiley & Sons. rethinking of post-hoc debiasing in vision-language models. arXiv preprint arXiv:2511.18123. — [arXiv:2511.18123](https://arxiv.org/abs/2511.18123) find
  60. Rubert B Streets. 1972. The diagnosis of plant diseases: a field and laboratory manual emphasizing the most practical methods for rapid identification. University of Arizona Press. find
  61. Lianmin Zheng, Wei-Lin Chiang, Ying Sheng, Siyuan Zhuang, Zhanghao Wu, Yonghao Zhuang, Zi Lin, Zhuohan Li, Dacheng Li, Eric Xing, and 1 others. 2023. Judging llm-as-a-judge with mt-bench and chatbot arena. Advances in neural information processing systems, 36:46595–46623.
  62. Nusrat Sultana, Sumaita Binte Shorif, Morium Akter, and Mohammad Shorif Uddin. 2022. Cucumber disease recognition dataset. find
  63. Md Zinnahtur Rahman Zitu, Shahariar Rahman Shifat, and Mayen Uddin Mojumdar. 2024. A benchmark dataset for detecting disease in plant leaves: An essential resource for deep learning models. find
  64. Ranjita Thapa, Kai Zhang, Noah Snavely, Serge Belongie, and Awais Khan. 2020. The plant pathology challenge 2020 data set to classify foliar disease of apples. Applications in Plant Sciences, 8(9):e11390. find
  65. Sandip Thite and Kailas Patil. 2025. Pisum sativum image dataset: Healthy and disease-affected cases. find
  66. Nguyen Thanh Truong, Xuan Linh Nguyen, Pham Dinh Cap Thang, and Tuong Le. 2025. A durian leaf image dataset of common diseases in vietnam for agricultural diagnosis. find
  67. Jason Wei, Xuezhi Wang, Dale Schuurmans, Maarten Bosma, Fei Xia, Ed Chi, Quoc V Le, Denny Zhou, and 1 others. 2022. Chain-of-thought prompting elicits reasoning in large language models. Advances in neural information processing systems, 35:24824– 24837.
  68. Adrian Wolny, Lorenzo Cerrone, Athul Vijayan, Rachele Tofanelli, Amaya Vilches Barro, Marion Louveaux, Christian Wenzl, Sören Strauss, David Wilson-Sánchez, Rena Lymbouridou, and 1 others. 2020. Accurate and versatile 3d segmentation of plant tissues at cellular resolution. Elife, 9:e57613. find
  69. Nahathai Wongpakaran, Tinakon Wongpakaran, Danny Wedding, and Kilem L. Gwet. 2013. A comparison of Cohen’s kappa and Gwet’s AC1 when calculating inter-rater reliability coefficients: a study conducted with personality disorder samples. BMC Medical Research Methodology, 13:61. find
  70. An Yang, Bowen Yu, Chengyuan Li, Dayiheng Liu, Fei Huang, Haoyan Huang, Jiandong Jiang, Jianhong Tu, Jianwei Zhang, Jingren Zhou, and 1 others. 2025. Qwen2. 5-1m technical report. arXiv preprint arXiv:2501.15383. — Qwen2.5-1M Technical Report
  71. Xiaoman Zhang, Chaoyi Wu, Ziheng Zhao, Weixiong Lin, Ya Zhang, Yanfeng Wang, and Weidi Xie. 2023a. Pmc-vqa: Visual instruction tuning for medical visual question answering. arXiv preprint arXiv:2305.10415. — [arXiv:2305.10415](https://arxiv.org/abs/2305.10415) find
  72. Zhuosheng Zhang, Aston Zhang, Mu Li, Hai Zhao, George Karypis, and Alex Smola. 2023b. Multimodal chain-of-thought reasoning in language models. arXiv preprint arXiv:2302.00923.
  73. Dachuan Zhao, Weiyue Li, Zhenda Shen, Yushu Qiu, Bowen Xu, Haoyu Chen, and Yongchao Chen. 2025. Bias is a subspace, not a coordinate: A geometric find

Updated

↑ back to top

This page in Markdown → https://md.research.elamur.ai/free/en/papers/sakib-2026-thinking-like-botanist-challenging-multimodal/