{"id":"arxiv:2605.23036","kind":"preprint","title":"Multilingual Steering by Design: Multilingual Sparse Autoencoders and Principled Layer Selection","authors":["Yusser Al Ghussin","Daniil Gurgurov","Tanja Baeumel","Josef van Genabith","Patrick Schramowski","Simon Ostermann"],"author_count":6,"year":2026,"published":"2026-05-21","venue":null,"cited_by":0,"doi":null,"arxiv_id":"2605.23036","nct_id":null,"source_url":"https://arxiv.org/abs/2605.23036","archive_url":"https://space-kitty.com/papers/arxiv:2605.23036","updated":"2026-05-21","pmid":null,"pmcid":null,"openalex_id":"W7162330427","pdf_url":"https://arxiv.org/pdf/2605.23036","abstract":"Sparse autoencoders (SAEs) enable feature-level mechanistic interpretability and activation steering in large language models (LLMs), but SAE-based language control remains unreliable in multilingual settings: most SAEs are trained on English-only data, and steering layers are chosen heuristically. We address these limitations by advancing a principled, mechanistic account of multilingual language steering with SAEs. First, we show that training SAEs on multilingual data consistently strengthens cross-lingual representations and yields more reliable, quality-preserving language control across layers and model families. Second, we introduce an \\emph{a priori} steering layer-selection rule based on the intersection of multilingual alignment and language separability, which predicts effective intervention depths without exhaustive layerwise search. We evaluate our approach on LLaMA-3.1-8B and Gemma-2-9B across machine translation and cross-lingual summarization (CrossSumm), using SpBLEU, ROUGE-L, COMET, and LaSE. Our results show that multilingual SAEs combined with intersection-selected layers stabilize the trade-off between language identification accuracy and generation quality, providing a principled, predictive, representation-level account of multilingual SAE steering.","abstract_license":"CC0-1.0 (arXiv descriptive metadata)","license":null,"is_open_access":false,"retracted":false,"peer_reviewed":false,"categories":["cs.CL"],"details":null,"provenance":"arxiv","attribution":"Abstract: arXiv descriptive metadata (CC0). Thank you to arXiv for use of its open access interoperability. This service was not reviewed or approved by, nor does it necessarily express or reflect the policies or opinions of, arXiv.","topics":[{"slug":"interpretability","name":"Mechanistic Interpretability","domain_slug":"ai"}],"cited_in_dossiers":[]}