Elamur Research⌕
☰
home / papers / 2026 / Automated Detection of Dosing Errors in Clinical Trial Narratives: A Multi-Modal Feature Engineering Approach with LightGBM

Automated Detection of Dosing Errors in Clinical Trial Narratives: A Multi-Modal Feature Engineering Approach with LightGBM

Paper · 2026 · Mohammad AL-Smadi

Summary

Errors in medication dosing during clinical trials endanger participants and can undermine study results, yet such deviations are usually caught only through laborious manual review. This paper frames automatic flagging of dosing errors as binary text classification on the CT-DEB benchmark (CT-DEB'26), which has 42,112 records and only about 5% positives. For every record, the author merges all non-empty text from nine fields (e.g., briefSummary, protocolPdfText, armDescriptions), giving a median input of about 5,400 characters. This text is converted into 3,451 features: word TF-IDF, character n-grams, all-MiniLM-L6-v2 sentence embeddings, rule-based medical patterns, and probability scores from BiomedBERT and DeBERTa-v3. An Optuna-tuned LightGBM model, trained as a 5-fold ensemble, reaches 0.8725 test ROC-AUC. Ablations show that sentence embeddings matter most, lexical features add complementary signal, and transformer scores contribute little. Keeping only the strongest 500–1000 features outperforms the full set. Overall, a well-engineered gradient-boosting pipeline remains a strong and economical option for highly imbalanced clinical text.

Key points

  • Data: CT-DEB, 42,112 records split into 29,478 train, 6,316 validation and 6,318 test, with a positive rate of about 4.6–4.9%. ROC-AUC is the primary metric.
  • Results: cross-validation AUC of 0.8833 ± 0.0091 and test ROC-AUC of 0.8725. The gap between out-of-fold and test scores is only 0.69%. Adjusting the decision threshold yields recall between 26% and 60%.
  • Ablation: dropping the sentence embeddings causes the biggest loss (2.39%), even though they make up only 37.07% of total feature importance.
  • Feature selection: the top 500–1000 features (14–29% of all) reach 0.886–0.887 AUC, compared with 0.879 for all 3,451. The author interprets this as noise reduction acting as regularization.
  • Feature inventory: about 2,000 word TF-IDF unigrams, about 1,000 character n-grams (n=3–7), MiniLM embeddings, medical pattern features, and BiomedBERT/DeBERTa-v3 scores.

Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.

 Institutions

🎓Qatar UniversityDoha · Qatar48 works · 1,026 citations

 Fields, topics and concepts

📚 Shelves 4The field of science this work belongs to (OpenAlex fields).Computer Science · 16,667 worksArtificial Intelligence · 16,194 worksBiochemistry, Genetics and Molecular Biology · 234 worksMolecular Biology · 231 works

🧭 Topics 1Research subjects and languages the work is about.Vision & language · 657 works

Cite

Cite as Mohammad AL-Smadi. 2026. Automated Detection of Dosing Errors in Clinical Trial Narratives: A Multi-Modal Feature Engineering Approach with LightGBM. Proceedings of the Third Workshop on Patient-Oriented Language Processing (CL4Health) @ LREC 2026.

The paper itself, not this page. BibTeX from the library's record of the work. DOI 10.63317/2kuc94aqpqj8

@inproceedings{alsmadi2026automated,
  title = {{Automated Detection of Dosing Errors in Clinical Trial Narratives: A Multi-Modal Feature Engineering Approach with LightGBM}},
  author = {Mohammad AL-Smadi},
  booktitle = {Proceedings of the Third Workshop on Patient-Oriented Language Processing (CL4Health) @ LREC 2026},
  year = {2026},
  doi = {10.63317/2kuc94aqpqj8},
  url = {https://doi.org/10.63317/2kuc94aqpqj8}
}

References

35 references, 9 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

  1. Takuya Akiba, Shotaro Sano, Toshihiko Yanase, Takeru Ohta, and Masanori Koyama. 2019. Optuna: A next-generation hyperparameter optimization framework. In Proceedings of the 25th ACM SIGKDD International Conference on Knowledge Discovery & Data Mining, pages 2623–2631.
  2. Emily Alsentzer, John R. Murphy, Willie Boag, Wei-Hung Weng, Di Jin, Tristan Naumann, and Matthew B. A. McDermott. 2019. Publicly available clinical BERT embeddings. In Proceedings of the 2nd Clinical Natural Language Processing Workshop, pages 72–78. Association for Computational Linguistics.
  3. Alan R. Aronson and François-Michel Lang. 2010. An overview of MetaMap: Historical perspective and recent advances. Journal of the American Medical Informatics Association, 17(3):229–236. find
  4. Nitesh V. Chawla, Kevin W. Bowyer, Lawrence O. Hall, and W. Philip Kegelmeyer. 2002. SMOTE: Synthetic minority over-sampling technique. Journal of Artificial Intelligence Research, 16:321– 357.
  5. Tianqi Chen and Carlos Guestrin. 2016. XGBoost: A scalable tree boosting system. In Proceedings of the 22nd ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, pages 785–794. ACM. find
  6. Matthew M. Churpek, Richa Adhikari, and Dana P. Edelson. 2016. The value of vital sign trends for detecting clinical deterioration on the wards. Resuscitation, 102:1–5. find
  7. Cemal Cingi and Nuray Bayar Muluk. 2017. Quick Guide to Good Clinical Practice. Springer. find
  8. Jacob Devlin, Ming-Wei Chang, Kenton Lee, and Kristina Toutanova. 2019. BERT: Pre-training of deep bidirectional transformers for language understanding. In Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies…
  9. Charles Elkan. 2001. The foundations of cost-sensitive learning. In Proceedings of the 17th International Joint Conference on Artificial Intelligence - Volume 2, IJCAI’01, page 973–978, San Francisco, CA, USA. Morgan Kaufmann Publishers Inc. find
  10. Sohrab Ferdowsi, Félicien Hêche, Anthony Yazdani, Edward Choi, Sara Sansaloni-Pastor, and Douglas Teodoro. 2026. Overview of the ct-deb’26 shared task on predicting dosing errors in interventional clinical trials. In Proceedings of the Third Workshop on Patient-Oriented Language Processing (CL4Healt… find
  11. Sohrab Ferdowsi, Julien Knafou, Nikolay Borissov, David Vicente Alvarez, Rahul Mishra, Poorya Amini, and Douglas Teodoro. 2023. Deep learning-based risk prediction for interventional clinical trials based on protocol design: A retrospective study. Patterns, 4(3). find
  12. Yu Gu, Robert Tinn, Hao Cheng, Michael Lucas, Naoto Usuyama, Xiaodong Liu, Tristan Naumann, Jianfeng Gao, and Hoifung Poon. 2021. Domain-specific language model pretraining for biomedical natural language processing.
  13. ACM Transactions on Computing for Healthcare, 3(1):1–23. find
  14. Rave Harpaz, William DuMouchel, Nigam H. Shah, David Madigan, Patrick Ryan, and Carol Friedman. 2012. Novel data-mining methodologies for adverse drug event discovery and analysis. Clinical Pharmacology & Therapeutics, 91(6):1010– 1021. find
  15. Pengcheng He, Xiaodong Liu, Jianfeng Gao, and Weizhu Chen. 2021. DeBERTa: Decoding-enhanced BERT with disentangled attention. In International Conference on Learning Representations (ICLR).
  16. Félicien Hêche, Sohrab Ferdowsi, Anthony Yazdani, Sara Sansaloni-Pastor, and Douglas Teodoro. 2026. Early risk stratification of dosing errors in clinical trials using machine learning. arXiv preprint arXiv:2602.22285. — [arXiv:2602.22285](https://arxiv.org/abs/2602.22285) find
  17. ICH Expert Working Group. 2001. ICH harmonised tripartite guideline: Guideline for good clinical practice E6 (R1). International Council for Harmonisation (ICH) / regulators. Commonly accessed via regulator repositories; no Crossref DOI. find
  18. Abhyuday Jagannatha, Feifan Liu, Weisong Liu, and Hong Yu. 2019. Overview of the first natural language processing challenge for extracting medication, indication, and adverse drug events from electronic health record notes (made 1.0). Drug safety, 42(1):99–111. find
  19. Abhyuday N. Jagannatha and Hong Yu. 2016. Structured prediction models for RNN based sequence labeling in clinical text. In Proceedings of the 2016 Conference on Empirical Methods in Natural Language Processing (EMNLP), pages 856–865, Austin, Texas, USA. Association for Computational Linguistics. find
  20. Alistair E. W. Johnson, Tom J. Pollard, Lu Shen, Li-Wei H. Lehman, Mengling Feng, Mohammad Ghassemi, Benjamin Moody, Peter Szolovits, Leo Anthony Celi, and Roger G. Mark. 2016. MIMIC-III, a freely accessible critical care database. Scientific Data, 3(1):160035.
  21. Leila R. Kalankesh and Elham Monaghesh. 2024. Utilization of ehrs for clinical trials: a systematic review. BMC Medical Research Methodology, 24(1):70. find
  22. Guolin Ke, Qi Meng, Thomas Finley, Taifeng Wang, Wei Chen, Weidong Ma, Qiwei Ye, and Tie-Yan Liu. 2017. Lightgbm: A highly efficient gradient boosting decision tree. In Advances in Neural Information Processing Systems, volume 30. Curran Associates, Inc. find
  23. Jinhyuk Lee, Wonjin Yoon, Sungdong Kim, Donghyeon Kim, Sunkyu Kim, Chan Ho So, and Jaewoo Kang. 2020. BioBERT: A pre-trained biomedical language representation model for biomedical text mining. Bioinformatics, 36(4):1234–1240.
  24. Fei Li, Weisong Liu, and Hong Yu. 2018. Extraction of information related to adverse drug events from electronic health record notes: Design of an end-to-end model based on deep learning. JMIR Medical Informatics, 6(4). find
  25. Takeo Nakayama. 2007. What are “clinical practice guidelines”? Journal of Neurology, 254(Suppl 5):2–7. find
  26. Alvin Rajkomar, Eyal Oren, Kai Chen, Andrew M. Dai, Nissan Hajaj, Michaela Hardt, Peter J. Liu, Xiaobing Liu, Jake Marcus, Mimi Sun, Patrik Sundberg, Hector Yee, Kun Zhang, Yi Zhang, Gerardo Flores, Gavin E. Duggan, Jamie Irvine, Quoc Le, Kurt Litsch, Alexander Mossin, Justin Tansuwan, De Wang, Jame… find
  27. Nils Reimers and Iryna Gurevych. 2019. Sentence-BERT: Sentence embeddings using siamese BERT-networks. In Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing (EMNLP-IJCNLP), pages 3982–3992,…
  28. Yuqi Si, Jingqi Wang, Hua Xu, and Kirk Roberts. 2019. Enhancing clinical concept extraction with contextual embeddings. Journal of the American Medical Informatics Association, 26(11):1297– 1304. find
  29. Özlem Uzuner, Brett R. South, Shuying Shen, and Scott L. DuVall. 2011. 2010 i2b2/va challenge on concepts, assertions, and relations in clinical text. Journal of the American Medical Informatics Association, 18(5):552–556. find
  30. Wenhui Wang, Furu Wei, Li Dong, Hangbo Bao, Nan Yang, and Ming Zhou. 2020. MiniLM: Deep self-attention distillation for task-agnostic compression of pre-trained transformers. In Advances in Neural Information Processing Systems, volume 33, pages 5776–5788. find
  31. Yannan Yuan, Yun Mei, Shuhua Zhao, Shenglong Dai, Xiaohong Liu, Xiaojing Sun, Zhiying Fu, Liheng Zhou, Jie Ai, Liheng Ma, et al. 2024. Data flow construction and quality evaluation of electronic source data in clinical trials: pilot study based on hospital electronic medical records in china. JMIR M… find
  32. Andrew J. Zimolzak, Li Wei, Usman Mir, Ashish Gupta, Viralkumar Vaghani, Devika Subramanian, and Hardeep Singh. 2024. Machine learning to enhance electronic detection of diagnostic errors. JAMA Network Open, 7(9):e2431982. find
  33. Emma Richard and Bhargava Reddy. 2021. Text classification for clinical trial operations: evaluation and comparison of natural language processing techniques. Therapeutic Innovation & Regulatory Science, 55(2):447–453. find
  34. Gerard Salton and Christopher Buckley. 1988. Term-weighting approaches in automatic text retrieval. Information Processing & Management, 24(5):513–523. find
  35. Guergana K. Savova, James J. Masanz, Philip V. Ogren, Jiaping Zheng, Sunghwan Sohn, Karin C. Kipper-Schuler, and Christopher G. Chute. 2010. Mayo clinical text analysis and knowledge extraction system (cTAKES): Architecture, component evaluation and applications. Journal of the American Medical Info… find

Updated

↑ back to top

This page in Markdown → https://md.research.elamur.ai/free/en/papers/smadi-2026-automated-detection-dosing-errors-clinical/