📄 AI Steerability 360: A Toolkit for Steering Large Language Models

Summary

Many techniques now exist for lightweight control of LLM behavior. Each follows its own conventions, so comparing or combining them is hard, and so is attributing effects to a specific intervention. IBM Research presents AI Steerability 360, an open-source, Hugging Face–native Python library (Apache 2.0). It groups steering methods, called controls, by where they act: the prompt (input), the weights or architecture (structural), activations and attention at inference time (state), and decoding (output). Every control plugs into a shared SteeringPipeline object, which handles training through steer() and inference through generate(), and which can chain several controls together. Use case and Benchmark classes let users define tasks and compare pipelines with fixed or swept parameters, revealing side effects on behaviors that were not targeted. The demonstrations include reducing sycophancy in Llama-2-7b-chat with CAA. They also show that combining PASTA and DeAL yields better truthfulness–informativeness trade-offs on TruthfulQA than either method alone. The toolkit fills a gap, since earlier frameworks typically cover only one control surface and lack decoding-time steering.

Key points

• The taxonomy has four control surfaces: input, structural, state and output. Each has a base class with a required override: get_prompt_adapter, steer, get_hooks and generate, respectively.
• The CAA example uses Llama-2-7b-chat-hf with multiplier −10.0 at layer 15, trained on sycophancy contrastive pairs. It turns an agreeing answer into a neutral, balanced one.
• In composite steering, PASTA (state) plus DeAL (output) on Qwen2.5-1.5B-Instruct improves the truthfulness–informativeness balance on TruthfulQA compared with each control used alone.
• Activation steering is broken into four reusable parts: estimator, selector, transform and gate. ActAdd, ITI and CAA are built this way, with different estimators (e.g., MeanDifferenceEstimator, SinglePairEstimator, ProbeMassShiftEstimator).
• Limitations include slow inference with transformers compared with vLLM (vLLM.hook support is planned) and the difficulty of choosing optimal control parameters.