📄 Red Teaming Language Model Detectors with Language Models

Summary

Detectors that flag LLM-generated text are proposed as a safeguard against misuse such as plagiarism or mass misinformation, but it was unclear how well they resist deliberate evasion. The authors stress-test three detector families: a trained classifier (OpenAI's AI Text Classifier), the watermarking scheme of Kirchenbauer et al. (2023), and the likelihood-based DetectGPT. Their setting is harder than earlier paraphrasing attacks because the attacker's helper LLM is assumed to be protected by a detector too. They introduce two attacks. The first asks the auxiliary LLM for context-aware synonym candidates and chooses replacements either without consulting the detector or through a query-based evolutionary search. The second targets instruction-following models like ChatGPT. It uses a small training subset to find one fixed extra instruction that pushes the writing style toward text the classifier misses. On XSum and ELI5, every detector loses substantial AUROC and detection rate. Human raters still judge the attacked outputs as reasonably fluent and plausible. The work shows current detection tools are not yet trustworthy.

Key points

• Attacks cover all three detector types. Word substitution works against classifier, watermark, and likelihood detectors. Instructional prompts target classifier-based detection of instruction-tuned LLMs and need no test-time queries.
• The prompt search drew on a training set of n = 50 examples and ran for T = 5 rounds, considering K = 5 candidates in each round. The OpenAI classifier was evaluated at a 0.9 threshold for detection rate. The found prompts beat paraphrasing with ChatGPT.
• Watermarking proved the most robust of the three, which the authors attribute to its statistical nature. Query-based substitution generally gave the strongest attacks, while query-free substitution tended to preserve fluency better.
• Human evaluation ran on Amazon MTurk. It covered the first 20 test examples per dataset, with 3 workers each rating fluency and plausibility on a 1–3 scale. Quality dropped only slightly.
• Proposed defence: evading the watermark typically requires swapping about 20% of tokens from greenlist to redlist. Checking redlist-token likelihood or perplexity under the watermarked model could therefore expose such tampering. Code is available at github.com/shizhouxing/LLM-Detector-Robustness.