{"id":"arxiv:2605.10664","kind":"preprint","title":"Prompt-Activation Duality: Improving Activation Steering via Attention-Level Interventions","authors":["Diancheng Kang","Zheyuan Liu","Ningshan Ma","Yue Huang","Zhaoxuan Tan","Meng Jiang"],"author_count":6,"year":2026,"published":"2026-05-11","venue":null,"cited_by":null,"doi":null,"arxiv_id":"2605.10664","nct_id":null,"source_url":"https://arxiv.org/abs/2605.10664","archive_url":"https://space-kitty.com/papers/arxiv:2605.10664","updated":"2026-05-14","pmid":null,"pmcid":null,"openalex_id":null,"pdf_url":"https://arxiv.org/pdf/2605.10664","abstract":"Activation steering controls language model behavior by adding directions to internal representations at inference time, but standard residual-stream steering can fail in stateful dialogue. We identify KV-cache contamination as a key failure mode: steered token states are stored and repeatedly reused, turning a local perturbation into cumulative coherence degradation. To address this challenge, we propose Gated Cropped Attention-Delta steering (GCAD), which extracts steering signals from system-prompt contributions to self-attention and applies them with token-level gating. Across persona-steering experiments, GCAD preserves trait control while substantially improving long-horizon coherence. On the main multi-turn benchmark, GCAD improves average coherence drift from -18.6 to -1.9 and raises turn-10 trait expression from 78.0 to 93.1. These results suggest that activation steering becomes more reliable when interventions follow the prompt-mediated pathways that models already use for behavioral control.","abstract_license":"CC0-1.0 (arXiv descriptive metadata)","license":null,"is_open_access":false,"retracted":false,"peer_reviewed":false,"categories":["cs.CL","cs.AI"],"details":null,"provenance":"arxiv","attribution":"Abstract: arXiv descriptive metadata (CC0). Thank you to arXiv for use of its open access interoperability. This service was not reviewed or approved by, nor does it necessarily express or reflect the policies or opinions of, arXiv.","topics":[],"cited_in_dossiers":[]}