{"id":"arxiv:2609.31122","kind":"preprint","title":"LocUS: Head Selection and Subspace Projection for Targeted Activation Steering","authors":["Irene Tallini","Lorenzo Basile","Valentino Maiorca","Francesco Locatello","Alberto Cazzaniga"],"author_count":5,"year":2026,"published":"2026-09-25","venue":null,"cited_by":null,"doi":null,"arxiv_id":"2609.31122","nct_id":null,"source_url":"https://arxiv.org/abs/2609.31122","archive_url":"https://space-kitty.com/papers/arxiv:2609.31122","updated":"2026-09-25","pmid":null,"pmcid":null,"openalex_id":null,"pdf_url":"https://arxiv.org/pdf/2609.31122","abstract":"Activation steering is a powerful training-free paradigm for controlling large language models at inference time. However, standard approaches estimate a per-layer steering direction from contrastive data and apply it on the layer's entire representation space, which may couple the intervention to off-target properties present in the contrastive data and degrade unrelated capabilities. To mitigate this issue, we introduce LocUS (Localized Unembedding Steering), a method which grounds activation steering to the model's own output vocabulary subspace. By identifying a property-specific linear subspace within the unembedding matrix, LocUS enforces a geometric constraint that restricts the steering transformation to a specific subspace and at the same time localizes its application to a sparse subset of attention heads. Extensive evaluations across three model families on toxicity mitigation, sentiment redirection and sycophancy suppression show that LocUS matches or outperforms state-of-the-art baselines while intervening on under 6% of parameters and better preserving general capability.","abstract_license":"CC0-1.0 (arXiv descriptive metadata)","license":null,"is_open_access":false,"retracted":false,"peer_reviewed":false,"categories":["cs.CL","cs.LG"],"details":null,"provenance":"arxiv","attribution":"Abstract: arXiv descriptive metadata (CC0). Thank you to arXiv for use of its open access interoperability. This service was not reviewed or approved by, nor does it necessarily express or reflect the policies or opinions of, arXiv.","topics":[],"cited_in_dossiers":[]}