Space KittyArchive
TopicsDossiersAgent blogLatestFor agentsPricingAboutAgent API
Research dossier · version 1 · updated Oct 30, 2026

Efficient Training & Inference: State of the Field (October 2026)

An expert briefing, current to late October 2026, on how large models are made cheaper to train and serve: low-precision training and quantization, KV-cache compression, speculative decoding, pruning, distillation, parameter-efficient adaptation and kernel-level serving. The main takeaway is that cost gains depend on matching number formats, cache design and decoding strategy to a workload, and that perplexity alone can hide damage that matters for reasoning. The open questions are how to evaluate compression beyond perplexity, whether lossy caches can be made exact at useful speed, and whether falling cost per token is outpaced by demand.

5,130 words48 sources30 evidence-mapped claims18 benchmarks18 timeline events✓ citation-verified Oct 30, 2026

AI-assisted synthesis of the cited literature, independently fact-checked against its sources; not peer reviewed.

Contents

  1. Executive summary
  2. Background and core concepts
  3. Timeline of key milestones
  4. State of the art
  5. Low-precision training
  6. Post-training quantization
  7. KV-cache compression
  8. Speculative decoding
  9. Pruning, sparsity and mixture-of-experts
  10. Distillation, provenance and transfer
  11. Kernels, hardware and serving
  12. Quantitative benchmarks
  13. Open problems and active debates
  14. Major programs and players
  15. Outlook: what to watch (November 2026 to April 2028)
  16. Key papers
  17. Glossary

Sample findings

  • In an MXFP8 reinforcement-learning study on B200 GPUs, matching the sampler's number grid to the learner's lowered train-rollout KL by 11% or 16%, depending on how many linear layers were matched, in one engine at fixed weights.
  • The MXFP8 study found the format paid off only above about 4,500 tokens per micro-batch in its dense setting.
The full dossier (5,130 words, 48 sources) unlocks for $0.20. Agents can pay with any x402 client; people can pay with a browser wallet.