# First-Order Steering: Translating Weight Adaptation into Activation Steering

Preprint · Sri Pranav Kunda, Alexander Kurz, Tomas Dominik, Uri Maoz

- Archive id: arxiv:2610.04283
- Published: 2026-10-03
- arXiv: https://arxiv.org/abs/2610.04283
- Citations: 0
- Source URL: https://arxiv.org/abs/2610.04283
- PDF: https://arxiv.org/pdf/2610.04283
- Provenance: arxiv

## Abstract

Activation steering exploits interpretable directions in the residual stream to enable inference-time manipulation of model behavior. Composing steering vectors to apply multiple target behaviors simultaneously is important in various fields-including AI alignment and safety-but remains a challenge for existing activation steering methods. In contrast, prior work in model merging shows that target behaviors represented by learned weight adaptations can be combined with high accuracy. A method that translates weight adaptations into activation steering vectors could therefore extend prior work in model merging to generate composable steering vectors that better enable simultaneous inference-time behavioral control. For this, we introduce First-Order Steering, a formulation of activation steering as a first-order approximation of weight update matrices parameterized by a vector of steering strengths, and establish theoretical bounds on the approximation error of first-order steering. We then develop a novel model merging procedure, HeRD-Merging, which minimizes the first-order approximation error terms to enable higher first-order steering accuracy. Together, our method produces steering vectors that control both individual and composed behaviors more accurately than existing activation steering methods. Furthermore, HeRD-Merging matches the performance of conventional model-merging baselines, while producing weight adaptations that admit more accurate first-order steering vectors.

_Abstract reproduced under CC0-1.0 (arXiv descriptive metadata)._

Topics: [Mechanistic Interpretability](https://space-kitty.com/topics/interpretability)

---
Space Kitty Archive · https://space-kitty.com · API: https://space-kitty.com/api/v1 · MCP: https://space-kitty.com/mcp · Premium: $0.20/request via x402 (Base Sepolia (testnet)). Metadata: arXiv (CC0), OpenAlex (CC0), Europe PMC (CC BY/CC0 works), ClinicalTrials.gov.