Elamur Research⌕
☰
home / papers / 2026 / Exploring Cross-Lingual Transfer in Transformer-Based Fraud Detection Models

Exploring Cross-Lingual Transfer in Transformer-Based Fraud Detection Models

Paper · 2026 · Iván Martínez Murillo, Robiert Sepúlveda-Torres and Juan Pablo Consuegra-Ayala

Summary

Most automated fraud detectors are built and tested on English text, and many address only one threat, usually spam. This paper asks whether a classifier fine-tuned in a single language can still recognize fraud written in another. The authors, working within the ALIA framework, build three binary detectors: spam (SMS and email), phishing, and fake job postings. Each detector is built on two multilingual encoders, MrBERT and mROBERTa. Training data came from openly licensed corpora on Hugging Face and Kaggle, with label schemes unified across sources. Spanish was used for the spam and phishing tasks. Because labeled Valencian resources are scarce, part of the job-posting data was translated into Valencian and checked by a language expert, giving a parallel English–Valencian set. Models trained only on English are almost flawless on English but transfer poorly, with phishing in Spanish suffering most. Adding target-language examples raises scores to near-ceiling levels without harming English. Transfer works best when the data in both languages are parallel. The study argues that non-English training data are essential for multilingual cybersecurity tools.

Key points

  • Encoders compared:
    • MrBERT: 308M parameters, ModernBERT-based, pretrained on 6.1T tokens in 35 European languages plus code, 8,192-token context.
    • mROBERTa: 283M parameters, RoBERTa-based, pretrained on 12.8TB covering 35 European and 92 programming languages, 256K SentencePiece vocabulary, 512-token context.
  • English-only training: F1 is about 0.98 on English test sets. On Spanish, F1 drops to roughly 0.44 for phishing and about 0.85 for spam.
  • Multilingual training: Adding Spanish or Valencian data lifts target-language F1 to about 0.95–0.99. English performance is kept or even improved, apart from a slight English drop for mROBERTa.
  • Fake job postings (parallel English–Valencian data): mROBERTa transferred surprisingly well to Valencian without Valencian training data, while MrBERT lagged.
  • Data selection: Only datasets under permissive licenses were kept (CC0, CC-BY, Apache, and MIT when clearly specified). Corpora with restrictive or unclear terms were dropped.

Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.

 Fields, topics and concepts

📚 Shelves 2The field of science this work belongs to (OpenAlex fields).Computer Science · 16,667 worksArtificial Intelligence · 16,194 works

🧭 Topics 1Research subjects and languages the work is about.Multilingual & cross-lingual NLP · 828 works

Cite

Cite as Iván Martínez Murillo, Robiert Sepúlveda-Torres and Juan Pablo Consuegra-Ayala. 2026. Exploring Cross-Lingual Transfer in Transformer-Based Fraud Detection Models. Proceedings of the Second International Conference on Natural Language Processing and Artificial Intelligence for Cyber Security.

The paper itself, not this page. BibTeX from the library's record of the work.

@inproceedings{murillo2026exploring,
  title = {{Exploring Cross-Lingual Transfer in Transformer-Based Fraud Detection Models}},
  author = {Iván Martínez Murillo, Robiert Sepúlveda-Torres and Juan Pablo Consuegra-Ayala},
  booktitle = {Proceedings of the Second International Conference on Natural Language Processing and Artificial Intelligence for Cyber Security},
  year = {2026},
  url = {https://aclanthology.org/2026.nlpaics-1.1/}
}

References

29 references, 1 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

  1. Mehraj Ibne Halim, Mohammad Zibran Hasan, Md Humayun Kabir, Mohammad Nadib Hasan, Hassan Jaki, Ahmad, and Hasibul Hasan. 2025. Enhancing phishing detection: A machine learning approach to predicting malicious emails, urls, and sms messages. Applied Computational Intelligence and Soft Computing, 2025… find
  2. Faisal Akbar, Junaid Hussain, Muhammad Babar Usman, and Jamil Afzal. 2025. The impact of financial scams on consumer trust in the banking sector: A qualitative analysis. International Journal of Discovery in Social Sciences, 1(1). find
  3. Vitali Herrera-Semenets, Lázaro Bustio-Martínez, Yamel Pérez-Guadarramas, Jorge Ángel González-Ordiano, and Jan van den Berg. 2024. Unmasking phishing attempts: A study on detection in spanish emails. In Iberoamerican Congress on Pattern Recognition, pages 1–15. Springer. find
  4. Mohammed Rasol Al Saidat, Suleiman Y Yerima, and Khaled Shaalan. 2024. Advancements of sms spam detection: A comprehensive survey of nlp and ml techniques. Procedia Computer Science, 244:248– 259. find
  5. Naser Abdullah Alam. 2024. Phishing email dataset. find
  6. Bodduru Keerthana, Anumala Reethika Reddy, and Avantika Tiwari. 2021. Accurate prediction of fake job offers using machine learning. In Machine Intelligence and Soft Computing: Proceedings of ICMISC 2020, pages 101–112. Springer. find
  7. Tiago A. Almeida, José María G. Hidalgo, and Akebo Yamakami. 2011. Contributions to the study of sms spam filtering: new collection and results. In Proceedings of the 11th ACM Symposium on Document Engineering, DocEng ’11, page 259–262, New York, NY, USA. Association for Computing Machinery. find
  8. Shashikant Kumar, Sadiya Yasmeen, Prabhat Kumar, et al. 2025. The growing threat of fake job postings: A review of machine learning and nlp-based detection approaches. Revolutionary Advances in Computing and Electronics: An International Journal, pages 41–51. find
  9. Pradeep Kumar B P, Bhagya D J, Gagana G, and Mani N. 2025. Fake job post detection using machine learning. In 2025 International Conference on Computing for Sustainability and Intelligent Future (COMP-SIF), pages 1–6. find
  10. Yinhan Liu, Myle Ott, Naman Goyal, Jingfei Du, Mandar Joshi, Danqi Chen, Omer Levy, Mike Lewis, Luke Zettlemoyer, and Veselin Stoyanov. 2019. Roberta: A robustly optimized bert pretraining approach. arXiv preprint arXiv:1907.11692.
  11. B Eswar Babu, Sathya Prakash Racharla, Pasupuleti Pavani, Yadaiah Balagoni, Mohan Ajmeera, et al. 2025. A performance evaluation of transformer models and recurrent neural networks models in efficient text classification tasks. In 2025 8th International Conference on Computing Methodologies and Comm… find
  12. Sandhya Mishra and Devpriya Soni. 2023. Sms phishing dataset for machine learning and pattern recognition. In Proceedings of the 14th International Conference on Soft Computing and Pattern Recognition (SoCPaR 2022), pages 597–604, Cham. Springer Nature Switzerland. find
  13. Shivam Bansal. Real / Fake Job Posting Prediction — kaggle.com. https://www.kaggle.com/datasets/shivamb/ real-or-fake-fake-jobposting-prediction/ data. [Accessed 16-03-2026]. ↗
  14. Chidimma Opara, Paolo Modesti, and Lewis Golightly. 2025. Evaluating spam filters and stylometric detection of ai-generated phishing emails. Expert systems with applications, 276:127044. find
  15. Lazaro Bustio-Martinez, Viviana Fuentes-Fuentes, Luisa Fernanda Agudelo-Fuentes, Vitali Herrera-Semenets, Darian Llanes-Guilarte, Felipe Antonio Trujillo-Fernández, Carlos Antonio Cardeña-Matamoros, Carlos Francisco Betancourt-Moreno, Joshua Ismael Haase-Hernández, Andrés Guillermo Molano-Jiménez, a… find
  16. Jude Osamor, Moses Ashawa, Alireza Shahrabi, Anand Phillip, and Celestine Iwend. 2025. The evolution of phishing and future directions: A review. In International Conference on Cyber Warfare and Security, pages 361–368. Academic Conferences International Limited. find
  17. Devsmit Ranparia, Shaily Kumari, and Ashish Sahani. 2020. Fake job prediction using sequential network. In 2020 IEEE 15th international conference on industrial and information systems (ICIIS), pages 339–343. IEEE. find
  18. Juan Manuel Diaz, Alfonso Martínez Cruz, and Lazaro Bustio Martinez. 2026. Spear-Phishing en español: dataset de correos dirigidos con señales de personal-ización (SpearPhishMX). find
  19. Tarini Saka, Kami Vaniea, and Nadin Kökciyan. 2025. Sok: Grouping spam and phishing email threats for smarter security. IEEE Access. find
  20. Sultana Umme Habiba, Md Khairul Islam, and Farzana Tasnim. 2021. A comparative study on fake job post prediction using different data mining techniques. In 2021 2nd international conference on robotics, electrical and signal processing techniques (ICREST), pages 543–546. IEEE. find
  21. Said Salloum, Tarek Gaber, Sunil Vadera, and Khaled Shaalan. 2022. A systematic literature review on phishing email detection using natural language processing techniques. Ieee Access, 10:65703–65727. find
  22. Robiert Sepúlveda-Torres, Iván Martínez-Murillo, Estela Saquete, Elena Lloret, and Manuel Palomar. 2025. To write or not to write as a machine? that’s the question. IEEE Transactions on Big Data. softecapps. 2024. spam_ham_spanish. find
  23. Cheekati Srikanth, M Rashmi, S Ramu, and Ram Mohana Reddy Guddeti. 2022. A novel fake job posting detection: an empirical study and performance evaluation using ml and ensemble techniques. In International Conference on Security, Privacy and Data Analytics, pages 219–234. Springer. find
  24. Daniel Tamayo, Iñaki Lacunza, Paula Rivera-Hidalgo, Severino Da Dalt, Javier Aula-Blasco, Aitor Gonzalez-Agirre, and Marta Villegas. 2026. Mrbert: Modern multilingual encoders via vocabulary, domain, and dimensional adaptation. arXiv preprint arXiv:2602.21379. — [arXiv:2602.21379](https://arxiv.org/abs/2602.21379) find
  25. Khushboo Taneja, Jyoti Vashishtha, and Saroj Ratnoo. 2025. Fraud-bert: transformer based context aware online recruitment fraud detection. Discover Computing, 28(1):9. find
  26. Kutub Thakur, Md Liakat Ali, Muath A Obaidat, and Abu Kamruzzaman. 2023. A systematic review on deep-learning-based phishing email detection. Electronics, 12(21):4545. find
  27. Mohammad Amaz Uddin, Md Mahiuddin, and Iqbal H Sarker. 2026. An explainable transformer-based model for phishing email detection: A large language model approach. Computer Networks, page 112061. find
  28. Soumitra Utturkar, Shravani Sonawane, Aditya Shah, Shruti Palange, Rupesh Jaiswal, and Mousami Munot. 2026. Phishing detection using machine learning for english and multilingual. In Proceedings of International Conference on Computational Intelligence and Information Retrieval: ICCIIR 2025, Volume… find
  29. Gabriela Zuñiga. 2024. dataset. find

Updated

↑ back to top

This page in Markdown → https://md.research.elamur.ai/free/en/papers/murillo-2026-exploring-cross-lingual-transfer-transformer/