Elamur Research⌕
☰
home / papers / 2026 / Region-Grounded Report Generation for 3D Medical Imaging: A Fine-Grained Dataset and Graph-Enhanced Framework

Region-Grounded Report Generation for 3D Medical Imaging: A Fine-Grained Dataset and Graph-Enhanced Framework

Paper · 2026 · Cong Huy Nguyen, Son Dinh Nguyen, Guanlin Li et al.

Summary

Automatic report writing for volumetric PET/CT scans is hard because the data are large and three-dimensional, annotated corpora are scarce (especially outside English and Chinese), and most systems translate a whole scan directly into text. Radiologists instead inspect individual Regions of Interest (RoIs) before writing a conclusion. To support this region-centred approach, the authors release VietPET-RoI, a Vietnamese dataset in which every finding mentioned in a report is linked to a 3D bounding box and a structured set of clinical attributes. They also present HiRRA, a vision-language model that keeps CT and PET features in separate streams before merging them. HiRRA then uses graph-based relational modules to model how RoIs and their attributes depend on each other. For evaluation, they propose RoI Coverage and RoI Quality Index, two clinically oriented measures computed with LLM assistance. HiRRA outperforms the strongest baseline, M3D-LaMed, on both text-overlap and clinical scores. The work offers a grounded, more interpretable and less hallucination-prone route to PET/CT reporting in a low-resource language.

Key points

  • VietPET-RoI data: 200 cancer patients from one Vietnamese hospital, covering six common cancer types scanned in 2017–2019. Each scan is standardised to 313 slices and split into head-neck, chest and abdomen-pelvis segments with a 15-slice overlap. This yields 600 image–report pairs and 1,960 manually annotated RoIs.
  • Annotation process: Eight nuclear medicine physicians used 3DSlicer to mark each RoI and fill a template with ten attributes, including size, SUVmax, density, FDG uptake, and top-3 diagnoses and follow-up tests. Senior physicians verified all annotations.
  • Main results: Compared with M3D-LaMed, HiRRA improves BLEU-4 by 19.7% and ROUGE-L by 4.7%. It also gains 15.5% and 45.8% on the two clinical metrics.
  • Fusion ablation: Combining CT and PET reaches 36.1 BLEU-4, compared with 31.3 for CT alone and 33.6 for PET alone.
  • Limitations: The data come from a single institution, which may introduce demographic bias. Annotation is costly at about 15–30 minutes per case, and the model has high compute requirements. The dataset is released for non-commercial research only.

Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.

 Institutions

🎓Institut Polytechnique de ParisPalaiseau · France15 works · 1,612 citations🎓Télécom SudParisÉvry-Courcouronnes · France2 works · 305 citations🎓Hanoi University of Science and TechnologyHanoi · Vietnam10 works · 89 citations

Also on the byline: 108 Military Central Hospital · Services répartis, Architectures, MOdélisation, Validation, Administration des Réseaux · Viện Nghiên cứu và Ứng dụng Trí tuệ nhân tạo.

 Fields, topics and concepts

📚 Shelves 7The field of science this work belongs to (OpenAlex fields).Computer Science · 18,614 worksArtificial Intelligence · 17,965 worksArts and Humanities · 17,450 worksLanguage and Linguistics · 17,296 worksComputer Vision and Pattern Recognition · 3,486 worksMedicine · 275 worksHealth Informatics · 214 works

Cite

Cite as Cong Huy Nguyen, Son Dinh Nguyen, Guanlin Li, Tuan Dung Nguyen, Aditya Narayan Sankaran, Mai Huy Thong, Thanh Trung Nguyen, Mai Hong Son, Reza Farahbakhsh, Phi Le Nguyen and Noel Crespi. 2026. Region-Grounded Report Generation for 3D Medical Imaging: A Fine-Grained Dataset and Graph-Enhanced Framework. Proceedings of the 64th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers).

The paper itself, not this page. BibTeX from the library's record of the work. DOI 10.18653/v1/2026.acl-long.903

@inproceedings{nguyen2026region,
  title = {{Region-Grounded Report Generation for 3D Medical Imaging: A Fine-Grained Dataset and Graph-Enhanced Framework}},
  author = {Cong Huy Nguyen, Son Dinh Nguyen, Guanlin Li, Tuan Dung Nguyen, Aditya Narayan Sankaran, Mai Huy Thong, Thanh Trung Nguyen, Mai Hong Son, Reza Farahbakhsh, Phi {Le Nguyen} and Noel Crespi},
  booktitle = {Proceedings of the 64th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers)},
  year = {2026},
  doi = {10.18653/v1/2026.acl-long.903},
  url = {https://doi.org/10.18653/v1/2026.acl-long.903}
}

References

38 references, 5 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

  1. Fan Bai and 1 others. 2024. Advancing 3d medical image analysis with multi-modal large language models. Preprint. M3D-LaMed: Generalist 3D medical vision–language model. find
  2. Thomas Beyer, Gerald Antoch, Stefan Müller, Thomas Egelhof, Lutz S Freudenberg, Jörg Debatin, and Andreas Bockisch. 2004. Acquisition protocol considerations for combined pet/ct imaging. Journal of Nuclear Medicine, 45(1 suppl):25S–35S. find
  3. Benedikt Boecking, Naoto Usuyama, Shruthi Bannur, Daniel C Castro, Anton Schwaighofer, Stephanie Hyland, Maria Wetscherek, Tristan Naumann, Aditya Nori, Javier Alvarez-Valle, and 1 others. 2022. Making the most of text semantics to improve biomedical vision–language processing. In European conferenc… find
  4. Shaked Brody, Uri Alon, and Eran Yahav. 2022. How attentive are graph attention networks? Preprint, arXiv:2105.14491. — [arXiv:2105.14491](https://arxiv.org/abs/2105.14491) find
  5. Kai Chen, Zhihong Chen, Yalun Gu, Wentao Wu, and Wanli Ouyang. 2024a. Towards systematic evaluation of hallucination for large vision language models in the medical context. arXiv preprint arXiv:2408.08472. — [arXiv:2408.08472](https://arxiv.org/abs/2408.08472) find
  6. Z. Chen and 1 others. 2024b. Internvl: Scaling up vision foundation models and aligning for generic visual– linguistic tasks. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pages 24185–24198.
  7. Daniel Coelho de Castro, Aurelia Bustos, Shruthi Bannur, Stephanie L Hyland, Kenza Bouzid, Maria Teodora Wetscherek, Maria Dolores Sánchez-Valverde, Lara Jaques-Pérez, Lourdes Pérez-Rodríguez, Kenji Takeda, and 1 others. 2025. Padchest-gr: A bilingual chest x-ray dataset for grounded radiology repor… find
  8. D Dhouib, A Naït-Ali, C Olivier, and MS Naceur. 2021. Roi-based compression strategy of 3d mri brain datasets for wireless communications. IRBM, 42(3):146–153. find
  9. Sergios Gatidis, Tobias Hepp, Marcel Früh, Christian La Fougère, Konstantin Nikolaou, Christina Pfannen-berg, Bernhard Schölkopf, Thomas Küstner, Clemens Cyran, and Daniel Rubin. 2022. A whole-body fdg-pet/ct dataset with manually annotated tumor lesions. Scientific Data, 9(1):601. find
  10. Akshay Goel. 2025. LangExtract. find
  11. Ibrahim Ethem Hamamci, Sezgin Er, and Bjoern Menze. 2024a. Ct2rep: Automated radiology report generation for 3d medical imaging. Preprint, arXiv:2403.06801. — [arXiv:2403.06801](https://arxiv.org/abs/2403.06801) find
  12. Ibrahim Ethem Hamamci, Sezgin Er, Anjany Sekuboy-ina, Enis Simsar, Alperen Tezcan, Ayse Gulnihan Simsek, Sevval Nil Esirgun, Furkan Almas, Irem Dogan, Muhammed Furkan Dasdelen, Chinmay Prabhakar, Hadrien Reynaud, Sarthak Pati, Christian Bluethgen, Mehmet Kemal Ozdemir, and Bjoern Menze. 2024b. Gener… — [arXiv:2305.16037](https://arxiv.org/abs/2305.16037) find
  13. Edward J. Hu, Yelong Shen, Phillip Wallis, Zeyuan Allen-Zhu, Yuanzhi Li, Shean Wang, Lu Wang, and Weizhu Chen. 2021. Lora: Low-rank adaptation of large language models. Preprint, arXiv:2106.09685.
  14. Young-jun Hu, Da-hye Kang, Jeong Won Hwang, Hwan-Ho Cho, Sang-Keun Lee, Mee-Hye Kim, Won-Jun Jung, and Kyoungjune Yang. 2024. Graph neural network model for prediction of non-small cell lung cancer lymph node metastasis using protein-protein interaction network and 18f-fdg pet/ct radiomics. Internat… find
  15. Wenpei Jiao, Kun Shang, Hui Li, Ke Yan, Jiajin Zhang, Guangjie Yang, Lijuan Guo, Yan Wan, Xing Yang, Dakai Jin, and Zhaoheng Xie. 2025. Vision-language models for automated 3d pet/ct report generation. arXiv preprint arXiv:2511.20145. — [arXiv:2511.20145](https://arxiv.org/abs/2511.20145) find
  16. Paul E Kinahan, Bruce H Hasegawa, and Thomas Beyer. 2003. X-ray-based attenuation correction for positron emission tomography/computed tomography scanners. Seminars in Nuclear Medicine, 33(3):166–179. Key kept as requested, original paper 2003. find
  17. Harold W Kuhn. 1955. The hungarian method for the assignment problem. Naval research logistics quarterly, 2(1-2):83–97. find
  18. Chunyuan Li, Cliff Wong, Sheng Zhang, Naoto Usuyama, Haotian Liu, Jianwei Yang, Tristan Naumann, Hoifung Poon, and Jianfeng Gao. 2023a. Llava-med: Training a large language-and-vision assistant for biomedicine in one day. Advances in Neural Information Processing Systems, 36:28541– 28564.
  19. Junnan Li, Dongxu Li, Silvio Savarese, and Steven Hoi. 2023b. Blip-2: Bootstrapping language-image pre-training with frozen image encoders and large language models. Preprint, arXiv:2301.12597.
  20. Tsung-Yi Lin, Piotr Dollár, Ross Girshick, Kaiming He, Bharath Hariharan, and Serge Belongie. 2017. Feature pyramid networks for object detection. Preprint, arXiv:1612.03144.
  21. Danyal Maqbool, Changhee Lee, Zachary Huemann, Samuel D. Church, Matthew E. Larson, Scott B. Perlman, Tomas A. Romero, Joshua D. Warner, Meghan Lubner, Xin Tie, Jameson Merkow, Junjie Hu, Steve Y. Cho, and Tyler J. Bradshaw. 2025. Petar: Localized findings generation with mask-aware vision-language… — [arXiv:2510.27680](https://arxiv.org/abs/2510.27680) find
  22. Pablo Messina, Pablo Pino, Denis Parra, Alvaro Soto, Cecilia Besa, Sergio Uribe, Marcelo Andía, Cristian Tejos, Claudia Prieto, and Daniel Capurro. 2022. A survey on deep learning and explainability for automatic report generation from medical images. ACM Computing Surveys (CSUR), 54(10s):1–40. find
  23. Peter Muzi, Michelle Wanner, and Paul Kinahan. 2015. Data from rider lung pet-ct. (No Title). find
  24. Huu Tien Nguyen, Dac Thai Nguyen, The Minh Duc Nguyen, Trung Thanh Nguyen, Thao Nguyen Truong, Huy Hieu Pham, Johan Barthelemy, Minh Quan Tran, Thanh Tam Nguyen, Quoc Viet Hung Nguyen, and 1 others. 2025. Toward a vision-language foundation model for medical data: Multimodal dataset and benchmarks f… — [arXiv:2509.24739](https://arxiv.org/abs/2509.24739) find
  25. OpenGVLab. 2025. Internvl3: Advanced multimodal large language models. https://internvl.github.io/blog/2025-04-11-InternVL-3.0/. Accessed 2025-12-01. ↗
  26. Andrés Ortiz, Juan M Górriz, Javier Ramírez, Francisco J Martinez-Murcia, and Alzheimer’s Disease Neuroimaging Initiative. 2014. Automatic roi selection in structural brain mri using som 3d projection. PloS one, 9(4):e93851. find
  27. Chaoyi Wu, Xiaoman Zhang, Ya Zhang, Yanfeng Wang, and Weidi Xie. 2025. Towards generalist foundation model for radiology by leveraging web-scale 2d & 3d medical data. Nature Communications. find
  28. Yunfei Xie, Ce Zhou, Lang Gao, Juncheng Wu, Xianhang Li, Hong-Yu Zhou, Sheng Liu, Lei Xing, James Zou, Cihang Xie, and 1 others. 2024. Medtrinity-25m: A large-scale multimodal dataset with multi-granular annotations for medicine. arXiv preprint arXiv:2408.02900. — [arXiv:2408.02900](https://arxiv.org/abs/2408.02900) find
  29. Yu Xin, Gorkem Can Ates, Kuang Gong, and Wei Shao. 2025. Med3dvlm: An efficient vision–language model for 3d medical image analysis. IEEE Journal of Biomedical and Health Informatics. To appear. find
  30. Junsan Zhang, Ming Cheng, Qiaoqiao Cheng, Xiuxuan Shen, Yao Wan, Jie Zhu, and Mengxuan Liu. 2024. Hierarchical medical image report adversarial generation with hybrid discriminator. Artificial Intelligence in Medicine, 151:102846. find
  31. Yichi Zhang, Wenbo Zhang, Zehui Ling, Gang Feng, Sisi Peng, Deshu Chen, Yuchen Liu, Hongwei Zhang, Shuqi Wang, Lanlan Li, and 1 others. 2025. Pet2rep: Towards vision-language model-drived automated radiology report generation for positron emission tomography. arXiv preprint arXiv:2508.04062. — [arXiv:2508.04062](https://arxiv.org/abs/2508.04062) find
  32. Steve Pieper, Michael Halle, and Ron Kikinis. 2004. 3d slicer. In 2004 2nd IEEE international symposium on biomedical imaging: nano to macro (IEEE Cat No. 04EX821), pages 632–635. IEEE. find
  33. A. Sellergren and 1 others. 2025. Medgemma: A collection of medical vision–language foundation models based on gemma 3. https://developers.google.com/health-ai-developer-foundations/ medgemma. Google Health AI Developer Foundations, accessed 2025-12-01. ↗
  34. Hyeonsoo Seo, Colin Huang, Maxime Bassenne, Ran Xiao, and Lei Xing. 2021. Lymph node graph neural networks for cancer metastasis prediction. arXiv preprint arXiv:2106.01711. — [arXiv:2106.01711](https://arxiv.org/abs/2106.01711) find
  35. Y. Shi and 1 others. 2025. Medm-vl: What makes a good medical lvlm? In Medical Image Computing and Computer-Assisted Intervention – MICCAI, Lecture Notes in Computer Science. Springer. find
  36. David W Townsend. 2008. Multimodality imaging of structure and function. Physics in Medicine & Biology, 53(4):R1. find
  37. Gustav K Von Schulthess, Hans C Steinert, and Thomas F Hany. 2009. Integrated pet/ct: current applications and future directions. Radiology, 251(3):708–736. find
  38. Stephen Waite, Arkadij Grigorian, Robert G Alexander, Stephen L Macknik, Marisa Carrasco, David J Heeger, and Susana Martinez-Conde. 2019. Analysis of perceptual expertise in radiology–current knowledge and a new perspective. Frontiers in human neuroscience, 13:213. find

Updated

↑ back to top

This page in Markdown → https://md.research.elamur.ai/free/en/papers/nguyen-2026-region-grounded-report-generation-3d/