# RouterInterp: Understanding Superposed Specialisation in Mixture of Experts Routing

Preprint · Ilya Lasy, Nora Yinuo Cai, Kola Ayonrinde

- Archive id: arxiv:2610.11775
- Published: 2026-10-08
- arXiv: https://arxiv.org/abs/2610.11775
- Citations: 0
- Source URL: https://arxiv.org/abs/2610.11775
- PDF: https://arxiv.org/pdf/2610.11775
- Provenance: arxiv

## Abstract

Sparse Mixture of Experts (MoE) models scale more efficiently than dense models by routing tokens to modular expert networks that are only active for processing a fraction of tokens. A leading hypothesis for the performance of MoE models is that each expert specialises in a single, coherent domain. However, interpretability efforts that assume this hypothesis have generally been unsuccessful. We propose and present evidence for an alternative account that we call the Superposed Specialisation Hypothesis (SSH): experts specialise in a disjoint union of fine-grained features rather than one broad domain. Leveraging the SSH, we introduce RouterInterp, a method for interpreting expert routing that identifies Sparse Autoencoder features most predictive of routing decisions and produces unified natural language explanations. On gpt-oss-20b, RouterInterp explains expert routing with ${\\sim}65\\%$ higher detection accuracy than prior token statistics based methods. This work provides a scalable method for generating more accurate explanations of expert routing and increases our understanding of a previously uninterpretable component of foundation models.

_Abstract reproduced under CC0-1.0 (arXiv descriptive metadata)._

Topics: [LLM Architectures & Pretraining](https://space-kitty.com/topics/llm-architectures)

---
Space Kitty Archive · https://space-kitty.com · API: https://space-kitty.com/api/v1 · MCP: https://space-kitty.com/mcp · Premium: $0.20/request via x402 (Base Sepolia (testnet)). Metadata: arXiv (CC0), OpenAlex (CC0), Europe PMC (CC BY/CC0 works), ClinicalTrials.gov.