# Efficient Training & Inference: State of the Field (October 2026)

Research dossier · v1 · updated 2026-10-30 · 5,130 words · 48 sources · 30 claims · verification: verified

An expert briefing, current to late October 2026, on how large models are made cheaper to train and serve: low-precision training and quantization, KV-cache compression, speculative decoding, pruning, distillation, parameter-efficient adaptation and kernel-level serving. The main takeaway is that cost gains depend on matching number formats, cache design and decoding strategy to a workload, and that perplexity alone can hide damage that matters for reasoning. The open questions are how to evaluate compression beyond perplexity, whether lossy caches can be made exact at useful speed, and whether falling cost per token is outpaced by demand.

## Contents

- Executive summary
- Background and core concepts
- Timeline of key milestones
- State of the art
  - Low-precision training
  - Post-training quantization
  - KV-cache compression
  - Speculative decoding
  - Pruning, sparsity and mixture-of-experts
  - Distillation, provenance and transfer
  - Kernels, hardware and serving
  - Quantitative benchmarks
- Open problems and active debates
- Major programs and players
- Outlook: what to watch (November 2026 to April 2028)
- Key papers
- Glossary

## Sample findings

- In an MXFP8 reinforcement-learning study on B200 GPUs, matching the sampler's number grid to the learner's lowered train-rollout KL by 11% or 16%, depending on how many linear layers were matched, in one engine at fixed weights.
- The MXFP8 study found the format paid off only above about 4,500 tokens per micro-batch in its dense setting.

## Get the full dossier

GET https://space-kitty.com/api/v1/dossiers/efficient-ai costs $0.20 via x402 (USDC on Base Sepolia (testnet)). Send `Accept: text/markdown` for Markdown or `application/json` for structured data (claims with citations, timeline, benchmarks, open problems, glossary, bibliography).
---
Space Kitty Archive · https://space-kitty.com · API: https://space-kitty.com/api/v1 · MCP: https://space-kitty.com/mcp · Premium: $0.20/request via x402 (Base Sepolia (testnet)). Metadata: arXiv (CC0), OpenAlex (CC0), Europe PMC (CC BY/CC0 works), ClinicalTrials.gov.