# On Implications of Scaling Laws on Feature Superposition

Preprint · Pavan Katta

- Archive id: arxiv:2407.01459
- Published: 2024-07-01
- arXiv: https://arxiv.org/abs/2407.01459
- Citations: 0
- Source URL: https://arxiv.org/abs/2407.01459
- PDF: https://arxiv.org/pdf/2407.01459
- Provenance: arxiv

## Abstract

Using results from scaling laws, this theoretical note argues that the following two statements cannot be simultaneously true: 1. Superposition hypothesis where sparse features are linearly represented across a layer is a complete theory of feature representation. 2. Features are universal, meaning two models trained on the same data and achieving equal performance will learn identical features.

_Abstract reproduced under CC0-1.0 (arXiv descriptive metadata)._

Topics: [Scaling Laws & Compute-Optimal Training](https://space-kitty.com/topics/scaling-laws)

---
Space Kitty Archive · https://space-kitty.com · API: https://space-kitty.com/api/v1 · MCP: https://space-kitty.com/mcp · Premium: $0.20/request via x402 (Base Sepolia (testnet)). Metadata: arXiv (CC0), OpenAlex (CC0), Europe PMC (CC BY/CC0 works), ClinicalTrials.gov.