{"slug":"efficient-ai","topic":"efficient-ai","title":"Efficient Training & Inference: State of the Field (October 2026)","summary":"An expert briefing, current to late October 2026, on how large models are made cheaper to train and serve: low-precision training and quantization, KV-cache compression, speculative decoding, pruning, distillation, parameter-efficient adaptation and kernel-level serving. The main takeaway is that cost gains depend on matching number formats, cache design and decoding strategy to a workload, and that perplexity alone can hide damage that matters for reasoning. The open questions are how to evaluate compression beyond perplexity, whether lossy caches can be made exact at useful speed, and whether falling cost per token is outpaced by demand.","version":1,"published_at":"2026-10-29","updated_at":"2026-10-30","stats":{"words":5130,"citations":48,"claims":30,"timeline":18,"benchmarks":18,"open_problems":6,"key_papers":15,"glossary":18},"verification":{"status":"verified","method":"independent AI fact-check against cited sources (evidence pack + web checks); automated citation resolution","claims_checked":30,"issues_fixed":25,"verified_at":"2026-10-30"},"ai_assisted":true,"price":{"usd":"0.20","protocol":"x402","network":"eip155:84532","asset":"USDC"},"url":"https://space-kitty.com/dossiers/efficient-ai","preview_url":"https://space-kitty.com/api/v1/dossiers/efficient-ai/preview","purchase_url":"https://space-kitty.com/api/v1/dossiers/efficient-ai","toc":[{"level":2,"text":"Executive summary","anchor":"executive-summary"},{"level":2,"text":"Background and core concepts","anchor":"background-and-core-concepts"},{"level":2,"text":"Timeline of key milestones","anchor":"timeline-of-key-milestones"},{"level":2,"text":"State of the art","anchor":"state-of-the-art"},{"level":3,"text":"Low-precision training","anchor":"low-precision-training"},{"level":3,"text":"Post-training quantization","anchor":"post-training-quantization"},{"level":3,"text":"KV-cache compression","anchor":"kv-cache-compression"},{"level":3,"text":"Speculative decoding","anchor":"speculative-decoding"},{"level":3,"text":"Pruning, sparsity and mixture-of-experts","anchor":"pruning-sparsity-and-mixture-of-experts"},{"level":3,"text":"Distillation, provenance and transfer","anchor":"distillation-provenance-and-transfer"},{"level":3,"text":"Kernels, hardware and serving","anchor":"kernels-hardware-and-serving"},{"level":3,"text":"Quantitative benchmarks","anchor":"quantitative-benchmarks"},{"level":2,"text":"Open problems and active debates","anchor":"open-problems-and-active-debates"},{"level":2,"text":"Major programs and players","anchor":"major-programs-and-players"},{"level":2,"text":"Outlook: what to watch (November 2026 to April 2028)","anchor":"outlook-what-to-watch-november-2026-to-april-2028"},{"level":2,"text":"Key papers","anchor":"key-papers"},{"level":2,"text":"Glossary","anchor":"glossary"}],"sample_claims":["In an MXFP8 reinforcement-learning study on B200 GPUs, matching the sampler's number grid to the learner's lowered train-rollout KL by 11% or 16%, depending on how many linear layers were matched, in one engine at fixed weights.","The MXFP8 study found the format paid off only above about 4,500 tokens per micro-batch in its dense setting."],"includes":["markdown","claims","timeline","benchmarks","open_problems","key_papers","glossary","bibliography"],"citation_count":48,"how_to_buy":"GET https://space-kitty.com/api/v1/dossiers/efficient-ai with an x402 client ($0.20, USDC on Base Sepolia (testnet))."}