Elamur Research⌕
☰
home / papers / 2025 / HAMSA: Hijacking Aligned Compact Models via Stealthy Automation

HAMSA: Hijacking Aligned Compact Models via Stealthy Automation

Summary

Safety-aligned compact LLMs can still be steered into harmful output by jailbreak prompts. Hand-written attacks do not scale, and earlier automated ones often produce garbled text that perplexity filters catch. HAMSA is an automated red-teaming framework that aims to generate fluent, hard-to-detect attack prompts for such models. It extends AutoDAN-style evolutionary search with temperature-controlled mutation and wraps requests in a structured "policy file" disguise. It also adds a retrieval-backed strategy library. A short warm-up phase records which attack variants worked. A summarizer LM then condenses the contrast between weaker and stronger attempts into named strategies, which a lifelong phase retrieves and reuses. Evaluation uses In-The-Wild Jailbreak Prompts on LLMs plus a new 100-prompt Moroccan Darija Arabic set adapted by native speakers. The boosted variant beats AutoDAN on most topics for Mistral, Qwen and Vicuna. Darija prompts drew more harmful replies than English ones. The work matters mainly as evidence that alignment is weaker in low-resource dialects, which argues for safety testing beyond English.

Key points

  • English data: In-The-Wild Jailbreak Prompts on LLMs, with 390 prompts over 13 safety topics. Only 20 prompts form the warm-up set that seeds the strategy library.
  • New resource: a 100-prompt Moroccan Darija subset translated and adapted by native speakers. Its topics include fraud, hate speech, malware and privacy.
  • Victim models are Mistral, Qwen and Vicuna, comparing AutoDAN with AutoDAN-Boost. The paper reports per topic the success rate, a 1–10 quality score and the mean number of attempts. The authors claim gains on 10 of 13 topics, but the table shows the baseline ahead in several Vicuna topics.
  • Strategy reuse depends on the score gap between paired attempts. A gap above 5 means the single best strategy is used, a gap of 2–5 means moderate strategies are combined, and a gap below 2 triggers exploration of new ones.
  • The harmfulness score was 1.28 for English and 2.24 for Darija. The authors attribute this to thin dialect training data and to Arabic morphology and idiom.

Interesting ideas

  • I1. The system pairs a low-scoring and a high-scoring attempt on the same input and has an LM state the difference as a named strategy with a definition and an example, which turns raw trial logs into reusable, human-readable knowledge. — For Mova (learn): Repair-rule induction could compare a failed and a corrected parse or answer for the same sentence and have the LLM teacher name the fix as an explicit rule with a definition and example.
  • I2. Library entries are retrieved by embedding similarity, and each entry stores the weak example, the strong example, the score gap and a summary, so every reuse carries its own evidence of benefit. — For Mova (answer): Solved maths word problems could be stored the same way, with the failing and working solution paths, so a new problem retrieves strategies backed by a recorded gain.
  • I3. A simple three-way rule uses the recorded improvement to choose between exploiting one strong strategy, combining several moderate ones, or exploring fresh ones, which gives a transparent exploit/explore policy without learned weights. — For Mova (learn): The same thresholds could decide whether to apply one high-gain repair rule, stack several mid-gain rules, or ask a teacher tier for a new rule when nothing helps enough.
  • I4. The protocol has a small warm-up set to build the library, a lifelong phase that keeps updating it, and a test phase with the library frozen and summarization off, which separates learning from honest measurement of transfer. — For Mova (learn): Self-play and silver-data loops could freeze the induced rule set before evaluation, so scores on held-out tests reflect transfer rather than leakage from rule updates.
  • I5. A dialectal subset built by native speakers exposed failures that the standard-language benchmark hid, showing that variety-specific test slices reveal blind spots. — For Mova (tokens): A small slice of dialectal or informal English, prepared by native speakers, could serve as a stress test for the tagger and lemmatizer alongside standard treebank text.

Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.

Fields: Linguistics & languages

Topics: Bias, fairness & safety

Cite

The paper itself, not this page. BibTeX from the library's record of the work. DOI 10.48550/arxiv.2508.16484 · arXiv 2508.16484

@misc{krylov2025hamsa,
  title = {{HAMSA: Hijacking Aligned Compact Models via Stealthy Automation}},
  author = {Krylov, Alexey and Vagizov, Iskander and Korzh, Dmitrii and Douiba, Maryam and Guezzaz, Azidine and Kokh, Vladimir and Erokhin, Sergey D. and Tutubalina, Elena V. and Rogov, Oleg Y.},
  year = {2025},
  doi = {10.48550/arxiv.2508.16484},
  eprint = {2508.16484},
  archivePrefix = {arXiv},
  url = {https://doi.org/10.48550/arxiv.2508.16484}
}

References

43 references, 18 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

  1. A. Grattafiori et al., “The llama 3 herd of models,” 2024. [Online]. Available: https://arxiv.org/abs/2407.21783 ↗
  2. OpenAI et al., “Gpt-4 technical report,” 2024. [Online]. Available: https://arxiv.org/abs/2303.0 8774 ↗ our card
  3. DeepSeek-AI et al., “Deepseek-r1: Incentivizing reasoning capability in llms via reinforcement learning,” 2025. [Online]. Available: https://arxiv.org/abs/2501.12948 ↗ our card
  4. L. Ouyang, J. Wu, X. Jiang, D. Almeida, C. L. Wainwright, P. Mishkin, C. Zhang, S. Agarwal, K. Slama, A. Ray, J. Schulman, J. Hilton, F. Kelton, L. Miller, M. Simens, A. Askell, P. Welinder, P. Christiano, J. Leike, and R. Lowe, “Training language models to follow instructions with human feedback,” 2022. [Online]. Available: https://arxiv.org/abs/2203.02155 ↗ our card
  5. D. M. Ziegler, N. Stiennon, J. Wu, T. B. Brown, A. Radford, D. Amodei, P. Christiano, and G. Irving, “Fine-tuning language models from human preferences,” 2020. [Online]. Available: https://arxiv.org/abs/1909.08593 ↗
  6. W. Zhao, Y. Hu, Z. Li, Y. Deng, Y. Zhao, B. Qin, T.-S. Chua, and T. Liu, “Towards comprehensive and efficient post safety alignment of large language models via safety patching,” 2024. [Online]. Available: https://openreview.net/forum?id=09JVxsEZPf ↗
  7. M. Salnikov et al., “Geopolitical biases in llms: what are the ”good” and the ”bad” countries according to contemporary language models,” 2025. [Online]. Available: https://arxiv.org/abs/2506.06751 ↗
  8. Y. Mao, T. Cui, P. Liu, D. You, and H. Zhu, “From llms to mllms to agents: A survey of emerging paradigms in jailbreak attacks and defenses within llm ecosystem,” 2025. [Online]. Available: https://arxiv.org/abs/2506.15170 ↗
  9. B. Li, H. Xing, C. Tian, C. Huang, J. Qian, H. Xiao, and L. Feng, “Structuralsleight: Automated jailbreak attacks on large language models utilizing uncommon text-organization structures,” 2025. [Online]. Available: https://arxiv.org/abs/2406.08754 ↗
  10. G. Alon and M. Kamfonas, “Detecting language model attacks with perplexity,” 2023. [Online]. Available: https://arxiv.org/abs/2308.14132 ↗ our card
  11. P. Lewis, E. Perez, A. Piktus, F. Petroni, V. Karpukhin, N. Goyal, H. Küttler, M. Lewis, W. tau Yih, T. Rocktäschel, S. Riedel, and D. Kiela, “Retrieval-augmented generation for knowledge-intensive nlp tasks,” 2021. [Online]. Available: https://arxiv.org/abs/2005.11401 ↗
  12. OpenAI, “Gpt-4 technical report,” 2023. find our card
  13. D. Ganguli, L. Lovitt, J. Kernion, A. Askell, Y. Bai, S. Kadavath, B. Mann, E. Perez, N. Schiefer, K. Ndousse et al., “Red teaming language models to reduce harms: Methods, scaling behaviors, and lessons learned,” arXiv preprint arXiv:2209.07858, 2022. ↗ our card
  14. H. Touvron, L. Martin, K. Stone, P. Albert, A. Almahairi, Y. Babaei, N. Bashlykov, S. Batra, P. Bhargava, S. Bhosale et al., “Llama 2: Open foundation and fine-tuned chat models,” arXiv preprint arXiv:2307.09288, 2023. ↗
  15. K. Ethayarajh, Y. Choi, and S. Swayamdipta, “Understanding dataset difficulty with V-usable information,” in International Conference on Machine Learning. PMLR, 2022, pp. 5988–6008. find
  16. A. Havrilla, https://huggingface.co/datasets/Dahoas/synthetic-instruct-gptj-pairwise, 2023, accessed: 2023-09-28. ↗
  17. S. H. Bach et al., “Promptsource: An integrated development environment and repository for natural language prompts,” 2022. find our card
  18. Y. Wang, S. Mishra, P. Alipoormolabashi, Y. Kordi, A. Mirzaei, A. Arunkumar, A. Ashok, A. S. Dhanasekaran, A. Naik, D. Stap et al., “Super-naturalinstructions: Generalization via declarative instructions on 1600+ nlp tasks,” arXiv preprint arXiv:2204.07705, 2022. ↗
  19. Y. Wang, Y. Kordi, S. Mishra, A. Liu, N. A. Smith, D. Khashabi, and H. Hajishirzi, “Self-instruct: Aligning language model with self generated instructions,” arXiv preprint arXiv:2212.10560, 2022. ↗
  20. J. Wu, L. Ouyang, D. M. Ziegler, N. Stiennon, R. Lowe, J. Leike, and P. Christiano, “Recursively summarizing books with human feedback,” arXiv preprint arXiv:2109.10862, 2021. ↗ our card
  21. L. Ouyang, J. Wu, X. Jiang, D. Almeida, C. Wainwright, P. Mishkin, C. Zhang, S. Agarwal, K. Slama, A. Ray et al., “Training language models to follow instructions with human feedback,” Advances in Neural Information Processing Systems, vol. 35, pp. 27 730–27 744, 2022. find our card
  22. D. Kang, X. Li, I. Stoica, C. Guestrin, M. Zaharia, and T. Hashimoto, “Exploiting programmatic behavior of llms: Dual-use through standard security attacks,” arXiv preprint arXiv:2302.05733, 2023. ↗
  23. J. Hazell, “Large language models can be used to effectively scale spear phishing campaigns,” arXiv preprint arXiv:2305.06972, 2023. ↗
  24. J. Christian, “Amazing “jailbreak” bypasses chatgpt’s ethics safeguards,” Futurism, February, vol. 4, p. 2023, 2023. find
  25. M. Burgess, “The hacking of chatgpt is just getting started,” Wired, 2023. find
  26. A. Albert, https://www.jailbreakchat.com/, 2023, accessed: 2023-09-28. ↗
  27. X. Shen, Z. Chen, M. Backes, Y. Shen, and Y. Zhang, “” do anything now”: Characterizing and evaluating in-the-wild jailbreak prompts on large language models,” in Proceedings of the 2024 on ACM SIGSAC Conference on Computer and Communications Security, 2024, pp. 1671–1685. find our card
  28. P. Chao, E. Debenedetti, A. Robey, M. Andriushchenko, F. Croce, V. Sehwag, E. Dobriban, N. Flammarion, G. J. Pappas, F. Tramèr, H. Hassani, and E. Wong, “Jailbreakbench: An open robustness benchmark for jailbreaking large language models,” in NeurIPS Datasets and Benchmarks Track, 2024. find our card
  29. Y. Liu, G. Deng, Z. Xu, Y. Li, Y. Zheng, Y. Zhang, L. Zhao, T. Zhang, and Y. Liu, “Jailbreaking chatgpt via prompt engineering: An empirical study,” 2023. find our card
  30. A. Wei, N. Haghtalab, and J. Steinhardt, “Jailbroken: How does llm safety training fail?” arXiv preprint arXiv:2307.02483, 2023. ↗ our card
  31. A. Zou, Z. Wang, J. Z. Kolter, and M. Fredrikson, “Universal and transferable adversarial attacks on aligned language models,” 2023. find our card
  32. G. Deng, Y. Liu, Y. Li, K. Wang, Y. Zhang, Z. Li, H. Wang, T. Zhang, and Y. Liu, “Jailbreaker: Automated jailbreak across multiple large language model chatbots,” 2023. find
  33. H. Li, D. Guo, W. Fan, M. Xu, J. Huang, F. Meng, and Y. Song, “Multi-step jailbreaking privacy attacks on chatgpt,” 2023. find our card
  34. R. Lapid, R. Langberg, and M. Sipper, “Open sesame! universal black box jailbreaking of large language models,” 2023. find
  35. X. Liu, N. Xu, M. Chen, and C. Xiao, “Autodan: Generating stealthy jailbreak prompts on aligned large language models,” arXiv preprint arXiv:2310.04451, 2023. ↗ our card
  36. X. Liu et al., “Autodan-turbo: A lifelong agent for strategy self-exploration to jailbreak llms,” 2025. [Online]. Available: https://arxiv.org/abs/2410.05295 ↗
  37. J. Ji et al., “Ai alignment: A comprehensive survey,” 2025. [Online]. Available: https://arxiv.org/abs/2310.19852 ↗
  38. X. Qi et al., “Safety alignment should be made more than just a few tokens deep,” 2024. [Online]. Available: https://arxiv.org/abs/2406.05946 ↗
  39. X. Shen et al., “Do anything now: Characterizing and evaluating in-the-wild jailbreak prompts on large language models,” 2024. [Online]. Available: https://arxiv.org/abs/2308.03825 ↗ our card
  40. J. Bai et al., “Qwen technical report,” arXiv preprint arXiv:2309.16609, 2023. ↗ our card
  41. A. Q. Jiang et al., “Mistral 7b,” 2023. [Online]. Available: https://arxiv.org/abs/2310.06825 ↗
  42. L. Zheng, W.-L. Chiang, Y. Sheng, S. Zhuang, Z. Wu, Y. Zhuang, Z. Lin, Z. Li, D. Li, E. P. Xing, H. Zhang, J. E. Gonzalez, and I. Stoica, “Judging llm-as-a-judge with mt-bench and chatbot arena,” 2023. [Online]. Available: https://arxiv.org/abs/2306.05685 ↗
  43. T. GigaChat et al., “Gigachat family: Efficient russian language modeling through mixture of experts architecture,” 2025. [Online]. Available: https://arxiv.org/abs/2506.09440 ↗

↑ back to top

This page in Markdown → https://md.research.elamur.ai/free/en/papers/krylov-2025-hamsa-hijacking-aligned-compact-models/