# Calibrating Lightweight Sparse Autoencoder Feature Steering

Preprint · Ananya Joshi, Celia Cintas, Skyler Speakman

- Archive id: arxiv:2506.12576
- Published: 2025-06-14
- arXiv: https://arxiv.org/abs/2506.12576
- Source URL: https://arxiv.org/abs/2506.12576
- PDF: https://arxiv.org/pdf/2506.12576
- Provenance: arxiv

## Abstract

Sparse autoencoders (SAEs) can enable inference-time topic steering by modifying latent feature activations, but existing steering methods often fail when target-aligned features are not identified or are modified at the wrong scale. We introduce \\textsc{ContrastiveSteer} to address these respective failure modes. First, features are scored by how much more strongly they activate on target-domain text than on general text. Second, steering strength is set using a model-specific calibration. We also introduce \\emph{contamination}, a complementary heuristic that measures whether post-steering activations emphasize weakly aligned features, thus indicating off-target or nonsensical outputs. In our evaluations, across multiple LLM families and SAEs, \\textsc{ContrastiveSteer} improves topic alignment across domains up to 3.9$\\times$ over unsteered and steering baselines, and raises target-domain classification by up to 93\\%. Our activation-scale normalization method also recovers at least 93\\% of optimal domain-score gains across all tested model families, and contamination distinguishes successful steering, under-steering, and over-steering collapse.

_Abstract reproduced under CC0-1.0 (arXiv descriptive metadata)._

---
Space Kitty Archive · https://space-kitty.com · API: https://space-kitty.com/api/v1 · MCP: https://space-kitty.com/mcp · Premium: $0.20/request via x402 (Base Sepolia (testnet)). Metadata: arXiv (CC0), OpenAlex (CC0), Europe PMC (CC BY/CC0 works), ClinicalTrials.gov.