📄 Command-Line Obfuscation Detection in Real-World Telemetry under Extreme Class Imbalance

Summary

Attackers often abuse built-in Windows utilities (LOLBins) and disguise their commands by rewriting the syntax while keeping the behavior. Tricks include random casing, caret insertion, and string reassembly, and they defeat hash and signature matching. The authors, from Cisco, build a detector for such obfuscated command-lines that can run on real endpoint telemetry. Two constraints shape the design: obfuscated entries there are extremely rare, and daily log volumes are huge.

They train a small transformer (an ELECTRA-based model) from scratch, paired with a WordPiece tokenizer fitted to command-line logs. Training uses real benign traffic, roughly 1,500 obfuscated samples confirmed by threat hunters, and synthetic positives generated with public obfuscation tools.

Earlier methods were mostly tested on balanced data. Here, evaluation subsets are resampled to progressively harsher class ratios, and the model beats existing approaches, especially at extreme imbalance. A second test covers 24 million real command-lines, with flagged items reviewed by hand. The model keeps precision high, runs on a single GPU, and finds previously unseen malicious variants. This makes it useful for cutting analyst workload at scale.

Key points

• Telemetry: about 350 million unique execution logs from around 1,000 private networks over 20 days. They cover PowerShell, cmd, msiexec, rundll and explorer. The estimated rate of real obfuscation is below 1 in 100k.
• Training and evaluation sets: both use a 1:10 positive-to-negative ratio.
  • Training: 405k negatives, 1.1k real obfuscated samples and 39.4k artificial ones.
  • Evaluation: 129k negatives, 387 real and 12.5k artificial positives.
  • Synthetic positives come from Invoke-Obfuscation and Invoke-DOSfuscation.
• Tokenizer: a custom WordPiece tokenizer (Huggingface Tokenizers) trained on 300M logs, with a 20k vocabulary chosen. A 10k-vocabulary version already yields about 11% fewer tokens than the 32k bert-base-cased tokenizer.
• Real-world deployment: detections were manually labeled to measure precision. The model caught Raspberry Robin-style commands that published signatures would miss.
• Limitations and code: stated limitations are reliance on tool-generated samples, untested adversarial robustness, and Windows-only scope. Code is released on GitHub, but the data and tokenizer stay private.