# Mechanistic Interpretability for Transformer-based Time Series Classification

Article · Matīss Kalnāre, Sofoklis Kitharidis, Thomas Bäck, Niki van Stein

- Archive id: arxiv:2511.21514
- Published: 2025-11-26
- Venue: Communications in computer and information science
- DOI: https://doi.org/10.1007/978-3-032-15638-9_15
- arXiv: https://arxiv.org/abs/2511.21514
- Citations: 0
- Source URL: https://arxiv.org/abs/2511.21514
- PDF: https://arxiv.org/pdf/2511.21514
- Provenance: arxiv

## Abstract

Transformer-based models have become state-of-the-art tools in various machine learning tasks, including time series classification, yet their complexity makes understanding their internal decision-making challenging. Existing explainability methods often focus on input-output attributions, leaving the internal mechanisms largely opaque. This paper addresses this gap by adapting various Mechanistic Interpretability techniques; activation patching, attention saliency, and sparse autoencoders, from NLP to transformer architectures designed explicitly for time series classification. We systematically probe the internal causal roles of individual attention heads and timesteps, revealing causal structures within these models. Through experimentation on a benchmark time series dataset, we construct causal graphs illustrating how information propagates internally, highlighting key attention heads and temporal positions driving correct classifications. Additionally, we demonstrate the potential of sparse autoencoders for uncovering interpretable latent features. Our findings provide both methodological contributions to transformer interpretability and novel insights into the functional mechanics underlying transformer performance in time series classification tasks.

_Abstract reproduced under CC0-1.0 (arXiv descriptive metadata)._

Topics: [Mechanistic Interpretability](https://space-kitty.com/topics/interpretability)

---
Space Kitty Archive · https://space-kitty.com · API: https://space-kitty.com/api/v1 · MCP: https://space-kitty.com/mcp · Premium: $0.20/request via x402 (Base Sepolia (testnet)). Metadata: arXiv (CC0), OpenAlex (CC0), Europe PMC (CC BY/CC0 works), ClinicalTrials.gov.