Elamur Research⌕
☰
home / papers / 2026 / RePrompT: Recurrent Prompt Tuning for Integrating Structured EHR Encoders with Large Language Models

RePrompT: Recurrent Prompt Tuning for Integrating Structured EHR Encoders with Large Language Models

Paper · 2026 · Arya Hadizadeh Moghaddam, Drew Ross, Mohsen Nayebi Kerdabadi et al.

Summary

When LLMs are applied to structured electronic health records, two weaknesses appear. Converting visit sequences into flat text blurs the timing between visits and the identity of individual codes. Each patient is also handled on its own, so the model gains nothing from patterns that recur across the cohort.

RePrompT addresses both problems with soft prompt tuning and leaves the base LLM architecture untouched. The pipeline works in three steps:

  • DeepSeek-V3 condenses the noisy discharge notes and visit codes into a short summary for each visit, which serves as the hard prompt.
  • The predictor LLM's hidden state from the previous visit is passed forward as a soft prompt, so information carries over from visit to visit.
  • A cohort-trained EHR encoder (RETAIN) turns the code history into a second set of trainable prompt tokens.

The authors test the framework on readmission and mortality prediction using MIMIC-III and MIMIC-IV. It beats both EHR-based and LLM-based baselines. The work shows that adding structured, population-level signals to LLMs through prompt tuning can make clinical prediction more reliable.

Key points

  • Model components:
    • Summaries come from DeepSeek-V3.
    • The predictor is Llama 3.1 1B, adapted for embedding extraction with LLM2Vec.
    • The structured encoder is RETAIN (Choi et al., 2016).
    • Each prompt-tuning module adds P soft tokens in front of the summary embeddings.
  • MIMIC-IV results:
    • Readmission AUROC/PRAUC reached 0.706/0.728, compared with 0.670/0.690 for RETAIN.
    • Mortality AUROC rose from 0.601 to 0.673.
  • MIMIC-III results:
    • Readmission AUROC reached 0.688, compared with 0.660 for RETAIN.
    • Mortality AUROC/PRAUC reached 0.646/0.152, compared with 0.608/0.134.
  • Ablation without DeepSeek summaries (raw clinical notes fed to Llama):
    • RePrompT still beats RETAIN on AUROC in all four settings.
    • The one exception is MIMIC-IV mortality PRAUC, which drops to 0.027 against 0.031 for RETAIN.
    • The authors conclude that the framework, not only the summarization, drives the gains.
  • Limitations noted by the authors:
    • The method depends on EHR data quality.
    • It may not generalize under domain shift to other healthcare systems.
    • It has so far been tested only on readmission and mortality tasks.

Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.

 Institutions

🎓University of KansasLawrence · United States8 works · 32 citations

 Fields, topics and concepts

📚 Shelves 6The field of science this work belongs to (OpenAlex fields).Computer Science · 18,614 worksArtificial Intelligence · 17,965 worksMedicine · 275 worksHealth Informatics · 214 worksHealth Professions · 75 worksHealth Information Management · 56 works

Cite

Cite as Arya Hadizadeh Moghaddam, Drew Ross, Mohsen Nayebi Kerdabadi, Dongjie Wang and Zijun Yao. 2026. RePrompT: Recurrent Prompt Tuning for Integrating Structured EHR Encoders with Large Language Models. Findings of the Association for Computational Linguistics: ACL 2026.

The paper itself, not this page. BibTeX from the library's record of the work. DOI 10.18653/v1/2026.findings-acl.1334

@inproceedings{moghaddam2026reprompt,
  title = {{RePrompT: Recurrent Prompt Tuning for Integrating Structured EHR Encoders with Large Language Models}},
  author = {Arya Hadizadeh Moghaddam, Drew Ross, Mohsen Nayebi Kerdabadi, Dongjie Wang and Zijun Yao},
  booktitle = {Findings of the Association for Computational Linguistics: ACL 2026},
  year = {2026},
  doi = {10.18653/v1/2026.findings-acl.1334},
  url = {https://doi.org/10.18653/v1/2026.findings-acl.1334}
}

References

41 references, 10 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

  1. Parishad BehnamGhader, Vaibhav Adlakha, Marius Mosbach, Dzmitry Bahdanau, Nicolas Chapados, and Siva Reddy. 2024. Llm2vec: Large language models are secretly powerful text encoders. arXiv preprint arXiv:2404.05961.
  2. Biykem Bozkurt, David Aguilar, Anita Deswal, Sandra B Dunbar, Gary S Francis, Tamara Horwich, Mariell Jessup, Mikhail Kosiborod, Allison M Pritchett, Kumudha Ramasubbu, and 1 others. 2016. Contributory risk and management of comorbidities of hypertension, obesity, diabetes mellitus, hyperlipidemia,… find
  3. Martin Juan José Bucher and Marco Martini. 2024. Fine-tuned’small’llms (still) significantly outperform zero-shot generative ai models in text classification. arXiv preprint arXiv:2406.08660. — [arXiv:2406.08660](https://arxiv.org/abs/2406.08660) find
  4. Chin Wang Cheong, Kejing Yin, William K Cheung, Benjamin CM Fung, and Jonathan Poon. 2023. Adaptive integration of categorical and multi-relational ontologies with ehr data for medical concept embedding. ACM Transactions on Intelligent Systems and Technology, 14(6):1–20. find
  5. Edward Choi, Mohammad Taha Bahadori, Le Song, Walter F Stewart, and Jimeng Sun. 2017. Gram: graph-based attention model for healthcare representation learning. In Proceedings of the 23rd ACM SIGKDD international conference on knowledge discovery and data mining, pages 787–795. find
  6. Edward Choi, Mohammad Taha Bahadori, Jimeng Sun, Joshua Kulas, Andy Schuetz, and Walter Stewart. 2016. Retain: An interpretable predictive model for healthcare using reverse time attention mechanism. Advances in neural information processing systems, 29. find
  7. Edward Choi, Zhen Xu, Yujia Li, Michael Dusenberry, Gerardo Flores, Emily Xue, and Andrew Dai. 2020. Learning the graphical structure of electronic health records with graph convolutional transformer. In Proceedings of the AAAI conference on artificial intelligence, volume 34, pages 606–613. find
  8. Junyi Gao, Cao Xiao, Yasha Wang, Wen Tang, Lucas M Glass, and Jimeng Sun. 2020. Stagenet: Stage-aware neural networks for health risk prediction. In Proceedings of the web conference 2020, pages 530– 540. find
  9. Ze-Feng Gao, Kun Zhou, Peiyu Liu, Wayne Xin Zhao, and Ji-Rong Wen. 2023. Small pre-trained language models can be fine-tuned as large models via overparameterization. In Proceedings of the 61st Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 3819– 3834. find
  10. Senay A Gebreab, Khaled Salah, Raja Jayaraman, Muhammad Habib ur Rehman, and Samer Ellaham. 2024. Llm-based framework for administrative task automation in healthcare. In 2024 12th International Symposium on Digital Forensics and Security (ISDFS), pages 1–7. IEEE. find
  11. Sagar Goyal, Eti Rastogi, Sree Prasanna Rajagopal, Dong Yuan, Fen Zhao, Jai Chintagunta, Gautam Naik, and Jeff Ward. 2024. Healai: A healthcare llm for effective medical documentation. In Proceedings of the 17th ACM International Conference on Web Search and Data Mining, pages 1167–1168. find
  12. Arya Hadizadeh Moghaddam, Mohsen Nayebi Kerdabadi, Mei Liu, and Zijun Yao. 2024. Contrastive learning on medical intents for sequential prescription recommendation. In Proceedings of the 33rd ACM International Conference on Information and Knowledge Management, pages 748–757. find
  13. Shibo Hao, Sainbayar Sukhbaatar, DiJia Su, Xian Li, Zhiting Hu, Jason Weston, and Yuandong Tian. 2024. Training large language models to reason in a continuous latent space. arXiv preprint arXiv:2412.06769. find
  14. Edward J Hu, Yelong Shen, Phillip Wallis, Zeyuan Allen-Zhu, Yuanzhi Li, Shean Wang, Lu Wang, Weizhu Chen, and 1 others. 2022. Lora: Low-rank adaptation of large language models. ICLR, 1(2):3.
  15. Pengcheng Jiang, Cao Xiao, Adam Cross, and Jimeng Sun. 2023. Graphcare: Enhancing healthcare predictions with personalized knowledge graphs. arXiv preprint arXiv:2305.12788. — [arXiv:2305.12788](https://arxiv.org/abs/2305.12788) find
  16. Alistair Johnson, Lucas Bulgarelli, Tom Pollard, Steven Horng, Leo Anthony Celi, and Roger Mark. 2020. Mimic-iv. PhysioNet. Available online at: https://physionet. org/content/mimiciv/1.0/(accessed August 23, 2021), pages 49–55. find
  17. Alistair EW Johnson, Lucas Bulgarelli, Lu Shen, Alvin Gayles, Ayad Shammout, Steven Horng, Tom J Pollard, Sicheng Hao, Benjamin Moody, Brian Gow, and 1 others. 2023. Mimic-iv, a freely accessible electronic health record dataset. Scientific data, 10(1):1.
  18. Alistair EW Johnson, Tom J Pollard, Lu Shen, Liwei H Lehman, Mengling Feng, Mohammad Ghassemi, Benjamin Moody, Peter Szolovits, Leo Anthony Celi, and Roger G Mark. 2016. Mimic-iii, a freely accessible critical care database. Scientific data, 3(1):1–9.
  19. Mohsen Nayebi Kerdabadi, Arya Hadizadeh Moghaddam, Dongjie Wang, and Zijun Yao. 2025. Multi-ontology integration with dual-axis propagation for medical concept representation. Preprint, arXiv:2508.21320. — [arXiv:2508.21320](https://arxiv.org/abs/2508.21320) find
  20. Brian Lester, Rami Al-Rfou, and Noah Constant. 2021. The power of scale for parameter-efficient prompt tuning. arXiv preprint arXiv:2104.08691.
  21. Rumeng Li, Xun Wang, and Hong Yu. 2024. Llamacare: An instruction fine-tuned large language model for clinical nlp. In Proceedings of the 2024 Joint International Conference on Computational Linguistics, Language Resources and Evaluation (LREC-COLING 2024), pages 10632–10641, Torino, Italia. ELRA an… find
  22. Aixin Liu, Bei Feng, Bing Xue, Bingxuan Wang, Bochao Wu, Chengda Lu, Chenggang Zhao, Chengqi Deng, Chenyu Zhang, Chong Ruan, and 1 others. 2024. Deepseek-v3 technical report. arXiv preprint arXiv:2412.19437. find
  23. Yang Liu and Mirella Lapata. 2019. Hierarchical transformers for multi-document summarization. arXiv preprint arXiv:1905.13164.
  24. Ilya Loshchilov and Frank Hutter. 2017. Decoupled weight decay regularization. arXiv preprint arXiv:1711.05101. highly capable multimodal models. arXiv preprint arXiv:2312.11805. find
  25. A Vaswani. 2017. Attention is all you need. Advances in Neural Information Processing Systems.
  26. Tu Vu, Brian Lester, Noah Constant, Rami Al-Rfou, and Daniel Cer. 2021. Spot: Better frozen model adaptation through soft prompt transfer. arXiv preprint arXiv:2110.07904.
  27. Cheng Wang, Gyuri Szarvas, Georges Balazs, Pavel Danchenko, and Patrick Ernst. 2024. Calibrating verbalized probabilities for large language models. arXiv preprint arXiv:2410.06707. — [arXiv:2410.06707](https://arxiv.org/abs/2410.06707) find
  28. Shansong Wang, Mingzhe Hu, Qiang Li, Mojtaba Safari, and Xiaofeng Yang. 2025a. Capabilities of gpt-5 on multimodal medical reasoning. arXiv preprint arXiv:2508.08224. — [arXiv:2508.08224](https://arxiv.org/abs/2508.08224) find
  29. Liantao Ma, Junyi Gao, Yasha Wang, Chaohe Zhang, Jiangtao Wang, Wenjie Ruan, Wen Tang, Xin Gao, and Xinyu Ma. 2020. Adacare: Explainable clinical health status representation learning via scale-adaptive feature extraction and recalibration. In Proceedings of the AAAI Conference on Artificial Intelli… find
  30. Zixiang Wang, Yinghao Zhu, Huiya Zhao, Xiaochen Zheng, Dehao Sui, Tianlong Wang, Wen Tang, Yasha Wang, Ewen Harrison, Chengwei Pan, and 1 others. 2025b. Colacare: Enhancing electronic health record modeling through large language model-driven multi-agent collaboration. In Proceedings of the ACM on W… find
  31. Jenish Maharjan, Anurag Garikipati, Navan Preet Singh, Leo Cyrus, Mayank Sharma, Madalina Ciobanu, Gina Barnes, Rahul Thapa, Qingqing Mao, and Ritankar Das. 2024. Openmedlm: prompt engineering can out-perform fine-tuning in medical question-answering with open-source large language models. Scientifi… find
  32. Thomas Wolf, Lysandre Debut, Victor Sanh, Julien Chaumond, Clement Delangue, Anthony Moi, Pier-ric Cistac, Tim Rault, Rémi Louf, Morgan Funtowicz, and 1 others. 2019. Huggingface’s transformers: State-of-the-art natural language processing. arXiv preprint arXiv:1910.03771.
  33. Bertalan Meskó. 2023. Prompt engineering as an important emerging skill for medical professionals: tutorial. Journal of medical Internet research, 25:e50638. find
  34. Phuoc Nguyen, Truyen Tran, Nilmini Wickramasinghe, and Svetha Venkatesh. 2016. Deepr: a convolutional net for medical records. IEEE journal of biomedical and health informatics, 21(1):22–30. find
  35. Adam Paszke, Sam Gross, Francisco Massa, Adam Lerer, James Bradbury, Gregory Chanan, Trevor Killeen, Zeming Lin, Natalia Gimelshein, Luca Antiga, and 1 others. 2019. Pytorch: An imperative style, high-performance deep learning library. Advances in neural information processing systems, 32.
  36. Mingtian Tan, Mike Merrill, Vinayak Gupta, Tim Althoff, and Tom Hartvigsen. 2024. Are language models actually useful for time series forecasting? Advances in Neural Information Processing Systems, 37:60162–60191. find
  37. Gemini Team, Rohan Anil, Sebastian Borgeaud, Jean-Baptiste Alayrac, Jiahui Yu, Radu Soricut, Johan Schalkwyk, Andrew M Dai, Anja Hauth, Katie Millican, and 1 others. 2023. Gemini: a family of Junda Wu, Tong Yu, Rui Wang, Zhao Song, Ruiyi Zhang, Handong Zhao, Chaochao Lu, Shuai Li, and Ricardo Henao.… find
  38. Ran Xu, Wenqi Shi, Yue Yu, Yuchen Zhuang, Bowen Jin, May Dongmei Wang, Joyce Ho, and Carl Yang. 2024. Ram-ehr: Retrieval augmentation meets clinical predictions on electronic health records. In Proceedings of the 62nd Annual Meeting of the Association for Computational Linguistics (Volume 2: Short P… find
  39. Chaoqi Yang, Zhenbang Wu, Patrick Jiang, Zhen Lin, Junyi Gao, Benjamin Danek, and Jimeng Sun. 2023. PyHealth: A deep learning toolkit for healthcare predictive modeling. In Proceedings of the 27th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining (KDD) 2023. find
  40. Chaoqi Yang, Cao Xiao, Fenglong Ma, Lucas Glass, and Jimeng Sun. 2021. Safedrug: Dual molecular graph encoders for recommending effective and safe drug combinations. arXiv preprint arXiv:2105.02711. — [arXiv:2105.02711](https://arxiv.org/abs/2105.02711) find
  41. Jamil Zaghir, Marco Naguib, Mina Bjelogrlic, Aurélie Névéol, Xavier Tannier, and Christian Lovis. 2024. Prompt engineering paradigms for medical applications: scoping review. Journal of Medical Internet Research, 26:e60501. find

Updated

↑ back to top

This page in Markdown → https://md.research.elamur.ai/free/en/papers/moghaddam-2026-reprompt-recurrent-prompt-tuning-integrating/