{"id":"arxiv:2605.12225","kind":"preprint","title":"On the Interpretability of Whisper Encodings Using Sparse Autoencoders","authors":["Dan Pluth","Zachary Nicholas Houghton","Yu Zhou","Vijay K. Gurbani"],"author_count":4,"year":2026,"published":"2026-05-12","venue":null,"cited_by":null,"doi":null,"arxiv_id":"2605.12225","nct_id":null,"source_url":"https://arxiv.org/abs/2605.12225","archive_url":"https://space-kitty.com/papers/arxiv:2605.12225","updated":"2026-10-01","pmid":null,"pmcid":null,"openalex_id":null,"pdf_url":"https://arxiv.org/pdf/2605.12225","abstract":"While deep transformer-based models have advanced rapidly, their internal mechanisms remain largely a mystery. Recent work has prioritized understanding text-based transformer models, leaving ASR systems largely unexplored. In order to address this gap, we examine the internal representations of Whisper's encoder using a sparse autoencoder. We find diverse monosemantic features across linguistic and non-linguistic boundaries, spanning a hierarchy from phonetic to semantic representations, and conduct a causal feature-steering campaign across this hierarchy, including cross-lingual steering. We further find that steering is more reliable for higher-level features than lower-level ones, an asymmetry that may reflect redundant encoding of lower-level information. Altogether, this work demonstrates that Whisper's encoder represents a surprisingly rich hierarchy of linguistic information that extends well beyond what is strictly necessary for transcription.","abstract_license":"CC0-1.0 (arXiv descriptive metadata)","license":null,"is_open_access":false,"retracted":false,"peer_reviewed":false,"categories":["cs.CL"],"details":null,"provenance":"arxiv","attribution":"Abstract: arXiv descriptive metadata (CC0). Thank you to arXiv for use of its open access interoperability. This service was not reviewed or approved by, nor does it necessarily express or reflect the policies or opinions of, arXiv.","topics":[],"cited_in_dossiers":[]}