# Foundations of the Scaling Law

Preprint · Tianyu Zhao, Zhixiang Ren

- Archive id: arxiv:2610.27789
- Published: 2026-10-20
- arXiv: https://arxiv.org/abs/2610.27789
- Citations: 0
- Source URL: https://arxiv.org/abs/2610.27789
- PDF: https://arxiv.org/pdf/2610.27789
- Provenance: arxiv

## Abstract

Empirical scaling laws relate loss to parameters $N$, training tokens $D$ and compute $C$. This technical note explains three mechanisms behind their exponents. Local approximation bounds function error at rate $N\_e^{-γs/d}$, with effective parameters $N\_e$, smoothness $s$, reuse exponent $γ$ and dimension $d$; the loss function determines the risk bound. Independent feature coverage gives $α\_D=(β-1)/ζ$ from predictive-weight decay $β$ and occurrence rarity $ζ$. Separable-loss optimization determines training allocations and deployment crossovers. Nonlinear scoring can produce apparent emergent abilities, while grokking concerns delayed generalization and verification constrains test-time scaling. For regular molecular configurations, the $\\mathrm{SE}(3)$ quotient gives $d\_{\\mathrm{int}}=d-6$ and relative approximation-exponent gain $6/d\_{\\mathrm{int}}$. Optimized dense-to-equivariant excess-loss ratios scale as $C^{p\_{\\mathrm{eq}}-p\_{\\mathrm{dense}}}$, where $p\_j$ are compute-loss exponents and the prefactor depends on loss amplitudes and FLOP costs. Domain examples also cover visual resolution, conformational coverage and correlated-sample precision.

_Abstract reproduced under CC0-1.0 (arXiv descriptive metadata)._

Topics: [Scaling Laws & Compute-Optimal Training](https://space-kitty.com/topics/scaling-laws)

---
Space Kitty Archive · https://space-kitty.com · API: https://space-kitty.com/api/v1 · MCP: https://space-kitty.com/mcp · Premium: $0.20/request via x402 (Base Sepolia (testnet)). Metadata: arXiv (CC0), OpenAlex (CC0), Europe PMC (CC BY/CC0 works), ClinicalTrials.gov.