Elamur Research⌕
☰
home / papers / 2026 / Understanding Structured Financial Data with LLMs: A Case Study on Fraud Detection

Understanding Structured Financial Data with LLMs: A Case Study on Fraud Detection

Paper · 2026 · Xuwei Tan, Yao Ma and Xueru Zhang

Summary

Tabular fraud detectors such as gradient-boosted trees depend on laborious feature engineering and provide little insight into individual decisions. LLMs can produce readable rationales, but prompting them directly on raw transaction tables gives near-random results. The difficulties are many numeric features, rare positive cases (often under 1%), and differing definitions of fraud across institutions. The authors propose FinFRE-RAG, a training-free pipeline with two stages:

  1. Offline: a conventional ML model ranks features by importance, and only the top-ranked numeric and categorical attributes are turned into natural-language descriptions.
  2. Online: similar labeled historical transactions are retrieved by cosine similarity and placed in the prompt as in-context examples. The LLM then outputs a 1–5 risk score with a short explanation for a human reviewer.

The method was tested on four public fraud datasets with three families of open-weight LLMs. Compared with direct prompting, it raises F1 and MCC substantially and sometimes matches strong tabular baselines, though specialized classifiers generally remain ahead. The work suggests LLMs can serve as interpretable assistants for fraud analysts, not as replacements for dedicated models.

Key points

  • Datasets: IEEE-CIS, CCF, CC Fraud and PaySim. Only IEEE-CIS has a rich feature set, and IEEE-CIS and CCF use anonymized feature names.
  • Analyses: the paper studies how many features and how many retrieved neighbors are needed, the effect of prediction granularity, and a comparison with LoRA-fine-tuned LLMs.
  • Gemma 3: it performs especially well on CCF and PaySim, apparently by following the majority label among closely clustered neighbors. This advantage disappears on the more heterogeneous IEEE-CIS data.
  • Retrieval bias: the PaySim test split contains just 13 fraud cases. For 11 of them every retrieved neighbor was fraudulent, and for the other 2 every neighbor was legitimate, so recall is driven mainly by retrieval.
  • Stated limitations: no production-scale or domain-specialized model comparisons, no latency or cost evaluation, and no systematic human review. The model's rationales should not be read as causal explanations.

Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.

 Institutions

🎓The Ohio State UniversityColumbus · United States82 works · 18,502 citations

 Fields, topics and concepts

📚 Shelves 2The field of science this work belongs to (OpenAlex fields).Computer Science · 16,667 worksArtificial Intelligence · 16,194 works

Cite

Cite as Xuwei Tan, Yao Ma and Xueru Zhang. 2026. Understanding Structured Financial Data with LLMs: A Case Study on Fraud Detection. Proceedings of the 64th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers).

The paper itself, not this page. BibTeX from the library's record of the work. DOI 10.18653/v1/2026.acl-long.1071

@inproceedings{tan2026understanding,
  title = {{Understanding Structured Financial Data with LLMs: A Case Study on Fraud Detection}},
  author = {Xuwei Tan, Yao Ma and Xueru Zhang},
  booktitle = {Proceedings of the 64th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers)},
  year = {2026},
  doi = {10.18653/v1/2026.acl-long.1071},
  url = {https://doi.org/10.18653/v1/2026.acl-long.1071}
}

References

53 references, 16 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

  1. Sandhini Agarwal, Lama Ahmad, Jason Ai, Sam Altman, Andy Applebaum, Edwin Arbus, Rahul K Arora, Yu Bai, Bowen Baker, Haiming Bao, and 1 others. 2025. gpt-oss-120b & gpt-oss-20b model card. arXiv preprint arXiv:2508.10925.
  2. Toyin D Aguda, Suchetha Siddagangappa, Elena Kochkina, Simerjot Kaur, Dongsheng Wang, and Charese Smiley. 2024. Large language models as financial data annotators: A study on effectiveness and efficiency. In Proceedings of the 2024 Joint International Conference on Computational Linguistics, Languag… find
  3. Takuya Akiba, Shotaro Sano, Toshihiko Yanase, Takeru Ohta, and Masanori Koyama. 2019. Optuna: A next-generation hyperparameter optimization framework. In Proceedings of the 25th ACM SIGKDD international conference on knowledge discovery & data mining, pages 2623–2631. 23383 9
  4. Tom Brown, Benjamin Mann, Nick Ryder, Melanie Subbiah, Jared D Kaplan, Prafulla Dhariwal, Arvind Neelakantan, Pranav Shyam, Girish Sastry, Amanda Askell, and 1 others. 2020. Language models are few-shot learners. Advances in neural information processing systems, 33:1877–1901.
  5. Tianqi Chen and Carlos Guestrin. 2016. Xgboost: A scalable tree boosting system. In Proceedings of the 22nd acm sigkdd international conference on knowledge discovery and data mining, pages 785– 794. find
  6. Davide Chicco and Giuseppe Jurman. 2020. The advantages of the matthews correlation coefficient (mcc) over f1 score and accuracy in binary classification evaluation. BMC genomics, 21(1):6. find
  7. Michael Han Daniel Han and Unsloth team. 2023. Unsloth. find
  8. Yingtong Dou, Zhiwei Liu, Li Sun, Yutong Deng, Hao Peng, and Philip S Yu. 2020. Enhancing graph neural network-based fraud detectors against camouflaged fraudsters. In Proceedings of the 29th ACM international conference on information & knowledge management, pages 315–324. find
  9. Duanyu Feng, Yongfu Dai, Jimin Huang, Yifang Zhang, Qianqian Xie, Weiguang Han, Zhengyu Chen, Alejandro Lopez-Lira, and Hao Wang. 2023. Empowering many, biasing a few: Generalist credit scoring through large language models. arXiv preprint arXiv:2310.00566. — [arXiv:2310.00566](https://arxiv.org/abs/2310.00566) find
  10. Ugo Fiore, Alfredo De Santis, Francesca Perla, Paolo Zanetti, and Francesco Palmieri. 2019. Using generative adversarial networks for improving classification effectiveness in credit card fraud detection. Information Sciences, 479:448–455. find
  11. João Gama, Indrė Žliobaitė, Albert Bifet, Mykola Pechenizkiy, and Abdelhamid Bouchachia. 2014. A survey on concept drift adaptation. ACM computing surveys (CSUR), 46(4):1–37. find
  12. Yury Gorishniy, Akim Kotelnikov, and Artem Babenko. 2024. Tabm: Advancing tabular deep learning with parameter-efficient ensembling. arXiv preprint arXiv:2410.24210. — [arXiv:2410.24210](https://arxiv.org/abs/2410.24210) find
  13. Kelvin Guu, Kenton Lee, Zora Tung, Panupong Pasupat, and Mingwei Chang. 2020. Retrieval augmented language model pre-training. In International conference on machine learning, pages 3929–3938. PMLR.
  14. Waleed Hilal, S Andrew Gadsden, and John Yawney. 2022. Financial fraud: a review of anomaly detection techniques and recent advances. Expert systems With applications, 193:116429. find
  15. Addison Howard and Bernadette Bouchon-Meunier. 2019. Ieee cis, inversion, john lei, lynn@ vesta, marcus2010, hussein abbass. IEEE-CIS Fraud Detection, Kaggle. find
  16. Edward J Hu, yelong shen, Phillip Wallis, Zeyuan Allen-Zhu, Yuanzhi Li, Shean Wang, Lu Wang, and Weizhu Chen. 2022. LoRA: Low-rank adaptation of large language models. In International Conference on Learning Representations.
  17. Jun Hu, Wenwen Xia, Xiaolu Zhang, Chilin Fu, Weichang Wu, Zhaoxin Huan, Ang Li, Zuoli Tang, and Jun Zhou. 2024. Enhancing sequential recommendation via llm-based semantic embedding learning. In Companion Proceedings of the ACM Web Conference 2024, pages 103–111. find
  18. Tairan Huang and Yili Wang. 2025. Can llms find fraudsters? multi-level llm enhanced graph fraud detection. arXiv preprint arXiv:2507.11997. — [arXiv:2507.11997](https://arxiv.org/abs/2507.11997) find
  19. Jing Jin and Yongqing Zhang. 2025. The analysis of fraud detection in financial market under machine learning. Scientific Reports, 15(1):29959. find
  20. SK Kamaruddin and Vadlamani Ravi. 2016. Credit card fraud detection using big data analytics: use of psoaann based one-class classification. In Proceedings of the international conference on informatics and analytics, pages 1–8. find
  21. Vladimir Karpukhin, Barlas Oguz, Sewon Min, Patrick SH Lewis, Ledell Wu, Sergey Edunov, Danqi Chen, and Wen-tau Yih. 2020. Dense passage retrieval for open-domain question answering. In EMNLP (1), pages 6769–6781.
  22. Guolin Ke, Qi Meng, Thomas Finley, Taifeng Wang, Wei Chen, Weidong Ma, Qiwei Ye, and Tie-Yan Liu. 2017. Lightgbm: A highly efficient gradient boosting decision tree. Advances in neural information processing systems, 30. find
  23. Seunghee Kim, Changhyeon Kim, and Taeuk Kim. 2025. Fcmr: Robust evaluation of financial cross-modal multi-hop reasoning. In Proceedings of the 63rd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 23352– 23380, Vienna, Austria. Association for Computatio… find
  24. Patrick Lewis, Ethan Perez, Aleksandra Piktus, Fabio Petroni, Vladimir Karpukhin, Naman Goyal, Heinrich Küttler, Mike Lewis, Wen-tau Yih, Tim Rocktäschel, and 1 others. 2020. Retrieval-augmented generation for knowledge-intensive nlp tasks. Advances in neural information processing systems, 33:9459–…
  25. Haohang Li, Yupeng Cao, Yangyang Yu, Shashidhar Reddy Javaji, Zhiyang Deng, Yueru He, Yuechen Jiang, Zining Zhu, K.p. Subbalakshmi, Jimin Huang, Lingfei Qian, Xueqing Peng, Jordan W. Suchow, and Qianqian Xie. 2025. Investorbench: A benchmark for financial decision-making tasks with llm-based agent.…
  26. Kaidi Li, Tianmeng Yang, Min Zhou, Jiahao Meng, Shendi Wang, Yihui Wu, Boshuai Tan, Hu Song, Lujia Pan, Fan Yu, and 1 others. 2024. Sefraud: Graph-based self-explainable fraud detection via interpretative mask learning. In Proceedings of the 30th ACM SIGKDD Conference on Knowledge Discovery and Data… find
  27. E Lopez-Rojas. 2017. Synthetic financial datasets for fraud detection. Kaggle. Available online: https://www.kaggle. com/datasets/ealaxi/paysim1 (accessed on 29 July 2023). ↗
  28. Junyu Luo, Zhizhuo Kou, Liming Yang, Xiao Luo, Jinsheng Huang, Zhiping Xiao, Jingshu Peng, Chengzhong Liu, Jiaming Ji, Xuanzhe Liu, Sirui Han, Ming Zhang, and Yike Guo. 2025. Finmme: Benchmark dataset for financial multi-modal reasoning evaluation. In Proceedings of the 63rd Annual Meeting of the As…
  29. Sewon Min, Xinxi Lyu, Ari Holtzman, Mikel Artetxe, Mike Lewis, Hannaneh Hajishirzi, and Luke Zettlemoyer. 2022. Rethinking the role of demonstrations: What makes in-context learning work? arXiv preprint arXiv:2202.12837.
  30. Yansong Ning, Shuowei Cai, Wei Li, Jun Fang, Naiqiang Tan, Hua Chai, and Hao Liu. 2025. Dima: An llm-powered ride-hailing assistant at didi. In Proceedings of the 31st ACM SIGKDD Conference on Knowledge Discovery and Data Mining V. 2, pages 4728–4739. find
  31. Liudmila Prokhorenkova, Gleb Gusev, Aleksandr Vorobev, Anna Veronika Dorogush, and Andrey Gulin. 2018. Catboost: unbiased boosting with categorical features. Advances in neural information processing systems, 31. find
  32. Ori Ram, Yoav Levine, Itay Dalmedigos, Dor Muhlgay, Amnon Shashua, Kevin Leyton-Brown, and Yoav Shoham. 2023. In-context retrieval-augmented language models. Transactions of the Association for Computational Linguistics, 11:1316–1331.
  33. Gurjot Singh, Prabhjot Singh, and Maninder Singh. 2025. Advanced real-time fraud detection using rag-based llms. arXiv preprint arXiv:2501.15290. — [arXiv:2501.15290](https://arxiv.org/abs/2501.15290) find
  34. Makram Soui, Ines Gasmi, Salima Smiti, and Khaled Ghédira. 2019. Rule-based credit risk assessment model using multi-objective evolutionary algorithms. Expert systems with applications, 126:144–157. find
  35. Xuwei Tan, Ziyu Hu, and Xueru Zhang. 2026. Benchmarking bias mitigation toward fairness without harm from vision to LVLMs. In The Fourteenth International Conference on Learning Representations. find
  36. Gemma Team, Aishwarya Kamath, Johan Ferret, Shreya Pathak, Nino Vieillard, Ramona Merhej, Sarah Perrin, Tatiana Matejovicova, Alexandre Ramé, Morgane Rivière, and 1 others. 2025. Gemma 3 technical report. arXiv preprint arXiv:2503.19786. find
  37. Hugo Touvron, Thibaut Lavril, Gautier Izacard, Xavier Martinet, Marie-Anne Lachaux, Timothée Lacroix, Baptiste Rozière, Naman Goyal, Eric Hambro, Faisal Azhar, and 1 others. 2023a. Llama: Open and efficient foundation language models. arXiv preprint arXiv:2302.13971.
  38. Hugo Touvron, Louis Martin, Kevin Stone, Peter Albert, Amjad Almahairi, Yasmine Babaei, Nikolay Bashlykov, Soumya Batra, Prajjwal Bhargava, Shruti Bhosale, and 1 others. 2023b. Llama 2: Open foundation and fine-tuned chat models. arXiv preprint arXiv:2307.09288.
  39. Chengrui Wang, Qingqing Long, Meng Xiao, Xunxin Cai, Chengjun Wu, Zhen Meng, Xuezhi Wang, and Yuanchun Zhou. 2024. Biorag: A rag-llm framework for biological question reasoning. arXiv preprint arXiv:2408.01107. — [arXiv:2408.01107](https://arxiv.org/abs/2408.01107) find
  40. Jianling Wang, Yifan Liu, Yinghao Sun, Xuejian Ma, Yueqi Wang, He Ma, Zhengyang Su, Minmin Chen, Mingyan Gao, Onkar Dalal, and 1 others. 2025. User feedback alignment for LLM-powered exploration in large-scale recommendation systems. In Proceedings of the 63rd Annual Meeting of the Association for C… find
  41. Neng Wang, Hongyang Yang, and Christina Wang. Fingpt: Instruction tuning benchmark for open-source large language models in financial datasets. In NeurIPS 2023 Workshop on Instruction Tuning and Instruction Following. find
  42. Shijie Wu, Ozan Irsoy, Steven Lu, Vadim Dabravolski, Mark Dredze, Sebastian Gehrmann, Prabhanjan Kambadur, David Rosenberg, and Gideon Mann. 2023. Bloomberggpt: A large language model for finance. arXiv preprint arXiv:2303.17564.
  43. Qianqian Xie, Weiguang Han, Zhengyu Chen, Ruoyu Xiang, Xiao Zhang, Yueru He, Mengxi Xiao, Dong Li, Yongfu Dai, Duanyu Feng, and 1 others. 2024. Finben: A holistic financial benchmark for large language models. Advances in Neural Information Processing Systems, 37:95716–95743. find
  44. Qianqian Xie, Weiguang Han, Xiao Zhang, Yanzhao Lai, Min Peng, Alejandro Lopez-Lira, and Jimin Huang. 2023. Pixiu: A comprehensive benchmark, instruction dataset and large language model for finance. Advances in Neural Information Processing Systems, 36:33469–33484. find
  45. An Yang, Anfeng Li, Baosong Yang, Beichen Zhang, Binyuan Hui, Bo Zheng, Bowen Yu, Chang Gao, Chengen Huang, Chenxu Lv, and 1 others. 2025a. Qwen3 technical report. arXiv preprint arXiv:2505.09388.
  46. Chengdong Yang, Hongrui Liu, Daixin Wang, Zhiqiang Zhang, Cheng Yang, and Chuan Shi. 2025b. Flag: Fraud detection with llm-enhanced graph neural network. In Proceedings of the 31st ACM SIGKDD Conference on Knowledge Discovery and Data Mining V. 2, pages 5150–5160. 23385 11 find
  47. Hongyang Yang, Xiao-Yang Liu, and Christina Dan Wang. 2023. Fingpt: Open-source financial large language models. FinLLM Symposium at IJCAI 2023.
  48. Shu Yang, Shenzhe Zhu, Zeyu Wu, Keyu Wang, Junchi Yao, Junchao Wu, Lijie Hu, Mengdi Li, Derek F Wong, and Di Wang. 2025c. Fraud-r1: A multi-round benchmark for assessing the robustness of llm against augmented fraud and phishing inducements. arXiv preprint arXiv:2502.12904. find
  49. Yi Yang, Mark Christopher Siy UY, and Allen Huang. 2020. Finbert: A pretrained language model for financial communications. Preprint, arXiv:2006.08097. find
  50. Michihiro Yasunaga, Xinyun Chen, Yujia Li, Panupong Pasupat, Jure Leskovec, Percy Liang, Ed H. Chi, and Denny Zhou. 2024. Large language models as analogical reasoners. In The Twelfth International Conference on Learning Representations. find
  51. Jianke Yu, Hanchen Wang, Xiaoyang Wang, Zhao Li, Lu Qin, Wenjie Zhang, Jian Liao, and Ying Zhang. 2023. Group-based fraud detection network on ecommerce platforms. In Proceedings of the 29th ACM SIGKDD conference on knowledge discovery and data mining, pages 5463–5475. find
  52. Yangyang Yu, Zhiyuan Yao, Haohang Li, Zhiyang Deng, Yuechen Jiang, Yupeng Cao, Zhi Chen, Jordan Suchow, Zhenyu Cui, Rong Liu, and 1 others. 2024. Fincon: A synthesized llm multi-agent system with conceptual verbal reinforcement for enhanced financial decision making. Advances in Neural Information P… find
  53. CCF CC F RAUD IEEE-CIS P AY S IM \\\# Numerical Features \\\# Categorical Features \\\# Positive \\\# Negative Imbalanced (%) 30 5 373 5 -2 20 3 490 596,014 20,663 8,213 284,317 9,403,986 569,877 6,354,407 0.172 5.98 3.50 0.13 License DbCL v1.0 DbCL v1.0 Competition Data CC BY-SA 4.0 *Table 5: Dataset summa… find

Updated

↑ back to top

This page in Markdown → https://md.research.elamur.ai/free/en/papers/tan-2026-understanding-structured-financial-data-llms/